At Under Armour, I recently overhauled the enterprise-level SEO strategy by pioneering an automated SEO testing framework. This shift from manual audits to data-driven experimentation resulted in a +71% jump in identifying regression issues and new codebase bugs, substantially cutting technical errors within six to nine months, and establishing a scalable foundation for sustained search leadership.
The methodology was to automate the validation of meta tags and schema, ensuring 100% compliance before and after site releases.
How we transformed the SEO testing process from manual to automated?
The Old Way
- Partnered with expensive SEO agencies
- Relied exclusively on external SEO tools
- Time-consuming manual testing
- SEO automated testing did not exist
- Testing was REACTIVE
- Took weeks or more to uncover issues
The New Way
- In-house 'build vs buy' SEO testing framework
- Robust SEO automation of 100+ test cases
- Testing became PROACTIVE
- Issues caught within hours of code deployment
- Dev Engineering immediately alerted for resolution
- CI/CD pipeline integration across all environments
Massive Scale-Up in Test Coverage
Our enterprise-level testing capabilities grew exponentially as we introduced more test cases into each test suite and scaled our overall testing solution.
To ensure a scalable and robust testing environment, the team integrated the following technologies:
Core Languages
Java PythonCI/CD & Version Control
GitHub Actions CI/CD PipelineTest Reporting & Monitoring
Zebrunner Real-Time Dashboards KPI TrackingWorkflow Management
Jira ConfluenceDynamic Validation
Our testing adapts to real-time content, validating thousands of elements per locale. A sample of the test conditions used:
- Validate that links on each page are crawlable
- Validate product image links are accessible in the code
- Validate each page has a Title and Meta Description
- Validate canonical tags are set correctly across environments
- Validate schema markup is present and valid
- Validate hreflang implementation for international pages
* It validates how search bots crawl with JavaScript disabled, to ensure maximum content discoverability.
Environment-Aware Testing
Robust testing was performed across the entire development lifecycle to catch issues early, improving the user experience in production.
* This allowed discovery of bugs and regression issues before they impacted consumers.
Locale-Specific Depth & Visual Integrity
Following the creation of a Proof of Concept (POC) in the US market, the commitment to localized quality assurance was demonstrated throughout North American markets.
Locale-Specific Depth & Visual Integrity
Demonstrating our commitment to localized quality assurance across markets. Click a card to expand.
What's next for this type of testing?
With an arsenal of approximately 107 high-impact SEO test scripts, the goal is to build highly targeted test suites to further refine and optimize SEO capabilities:
- Functional: Regression testing for all core SEO elements on every release
- Performance: Speed and Core Web Vitals monitoring integrated into the pipeline
- International: Expanded hreflang and locale validation for additional markets
- AI-era: Structured data validation for AI Overviews and rich result eligibility
Work Samples
Frequently Asked Questions
You can utilize Java or Python to develop custom scripts that validate critical SEO elements such as meta-tags, canonicals, and schema against a staging environment. This automation ensures that code deployments do not trigger SEO regressions, moving the process from manual sampling to 100% data coverage.
Integrating SEO tests into the CI/CD pipeline means every code deployment automatically triggers a full suite of SEO checks. This allows the engineering team to catch and resolve SEO-impacting regressions within hours of a commit, rather than discovering them weeks later through manual audits or traffic drops.
Locale-specific validation involves running the full test suite against each market's URL set, verifying hreflang implementation, canonical tag configuration, image availability, and link crawlability per locale. Starting with a US Proof of Concept, the framework was expanded across all North American markets to ensure parity in technical SEO quality.
The 'build vs buy' evaluation weighs the cost of licensing enterprise SEO testing tools against the long-term value of an in-house framework tailored to your specific site architecture and testing needs. At Under Armour, the decision to build a custom framework delivered significantly better coverage, deeper CI/CD integration, and lower ongoing cost than available third-party solutions.
Key KPIs include: test pass/fail rate over time, bug discovery velocity (how quickly new issues are identified after deployment), mean time to resolution, and the reduction in SEO-related production incidents. Tracking a sustained 98%+ pass rate over a 12-month period demonstrates both framework stability and the team's commitment to technical SEO excellence.