Skip to content

AI grades its own homework: why demand for QA is about to come back

Mykhailo Poliarush5 min read

Teams now ship more code than ever and understand less of it, and the usual safety net is asking the same AI to write the tests. That is why demand for QA is likely to come back hard: in the DORA 2025 report, AI adoption raises delivery throughput but still increases delivery instability, and 30% of technology professionals report little or no trust in AI-generated code. Someone has to close that gap, and it is not the model that wrote the code.

Why does everything look great on paper?

The dashboards are full of good news. In DORA 2025, 90% of technology professionals use AI at work and more than 80% believe it has increased their productivity. Telemetry from Faros AI (a vendor, analyzing more than 10,000 developers on 1,255 teams) shows developers on high-adoption teams completing 21% more tasks and merging 98% more pull requests.

Testing looks covered too. Among people whose job includes creating test cases, 62% already use AI to write them, according to DORA 2025. Coverage numbers go up with every merge.

Where is the gap quietly getting wider?

Look one level down and the same data tells a different story. In the Faros telemetry, PR review time rose 91%, average PR size grew 154% and bugs per developer rose 9%, and Faros found no significant correlation between AI adoption and improvements at the company level. Teams produce more change than they can understand.

Teams with high AI adoption: reported change by metric

Average PR size
154%
Pull requests merged
98%
PR review time
91%
Tasks completed
21%
Bugs per developer
9%

Source: Faros AI, AI Productivity Paradox Report 2025 (vendor; 10,000+ developers, 1,255 teams)

Developers feel it. In the Stack Overflow Developer Survey 2025, 66% name AI answers that are "almost right, but not quite" as a frustration, and 45.2% say debugging AI-generated code takes longer. Only 33% trust the accuracy of AI output, while 46% distrust it (Stack Overflow 2025).

Perception is not a reliable gauge either. In a randomized trial by METR, experienced open-source developers were 19% slower with AI tools while believing they had been 20% faster. If teams cannot feel their own speed correctly, they will not feel quality slipping either.

How developers experience AI-generated code

How developers experience AI-generated code
CriterionShare of respondents
Use or plan to use AI tools84%
Frustrated by answers that are almost right, but not quite66%
Distrust the accuracy of AI output46%
Say debugging AI-generated code takes longer45.2%
Trust the accuracy of AI output33%

Source: Stack Overflow Developer Survey 2025, AI section (each question had its own respondent base)

What is wrong with "AI writes the code, AI writes the tests"?

The same model that wrote the code is grading its own homework. Tests generated from the change confirm the model's reading of the ticket; if that reading was wrong, code and tests are wrong together and the build stays green. It looks like a four-eyes check, but there is only one pair of eyes.

Research points the same way. A 2024 study, LLM Evaluators Recognize and Favor Their Own Generations, describes self-preference: an LLM evaluator scores its own outputs higher than others' while human annotators consider them of equal quality. The authors also found that models such as GPT-4 and Llama 2 can recognize their own outputs, and that this self-recognition correlates with the strength of the bias. A test suite written by the model that wrote the code inherits the same blind spot.

The risk is not only functional. In Veracode's 2025 study (Veracode sells security testing), AI-generated code introduced security vulnerabilities in 45% of cases. Tests that only restate the implementation will not catch what the implementation never considered.

Developers test their intent. Who tests the user's reality?

A developer, human or AI, tests what they meant to build. QA tests what the user will actually do. That is a different job:

  • Questioning the requirement before a line is written, when a wrong assumption is cheapest to fix.
  • Thinking off the happy path: the odd input, the slow network, the half-finished form, the user who clicks back twice.
  • Exploring after the build, looking for problems nobody wrote a ticket for.
  • Owning independent checks, so the person or model that wrote the code is not the only one deciding it works.

Writing automated test cases has never been easier. What still cannot be automated is someone who thinks like your users and goes hunting for the weird edge cases.

Is quality debt already building up?

There are early signs. Bitrise reports that the share of teams experiencing any test flakiness grew from 10% in 2022 to 26% in 2025, across more than 10 million builds (Bitrise Mobile Insights 2025, vendor data). In the World Quality Report 2025 (Capgemini, Sogeti and OpenText), 89% of organizations are piloting or deploying generative AI in quality engineering, but only 15% have reached enterprise-wide implementation.

The bill for poor quality was already large before AI coding took off: CISQ estimated the cost of poor software quality in the US at a minimum of $2.41 trillion in 2022. Quality debt behaves like technical debt. It is invisible on the sprint board and very visible in incidents, churn and support tickets.

Will QA demand swing back?

Our prediction is yes, once businesses notice quality slipping. The labor data is at least consistent with that: the US Bureau of Labor Statistics projects employment of software developers, quality assurance analysts and testers (one combined group) to grow 10% from 2025 to 2035, much faster than the average, with about 106,100 openings a year. That is a forecast for the whole group, not proof of a QA boom, but it does not describe a profession being automated away.

The QA role that comes back will look different. Less time typing scripts, which AI does well. More time on requirements, risk, exploratory testing, independent verification of AI-written changes, and test strategy that decides what is worth automating at all.

How do you keep AI speed without losing quality?

Keep the speed and add an independent check:

  1. Separate the writer from the checker. Acceptance tests come from the requirement, written or reviewed by someone other than the author of the change.
  2. Test the user's reality. Schedule exploratory sessions on every significant feature, not just regression runs.
  3. Make tests readable. Scenario-style tests that a product owner can read are easier to challenge than generated assertions.
  4. Watch the quality signals: escaped defects, change failure rate, flaky tests and review time, not only coverage.
  5. Use AI where it is strong: drafting tests, test data and page objects, with a person deciding what the tests must prove.

Want a second pair of eyes that is not the model?

SDCLabs has built and tested software since 2011 and created the open-source testing frameworks Codeception and CodeceptJS. Our engineers use AI to draft tests faster, and keep humans in charge of what the tests prove. Talk to us about QA and test automation, manual and exploratory testing or testing AI-powered products, or send us a short description of where your quality is slipping.

Sources checked on 2026-11-17.

Frequently asked questions

Can AI write my tests?

It can draft them well, and 62% of people who create test cases already use AI for it (DORA 2025). But tests written by the model that wrote the code mostly confirm its own reading of the requirement, so someone independent should decide what the tests must prove.

Why is AI-generated code a quality risk?

DORA 2025 finds AI adoption still increases delivery instability, and in Faros AI's telemetry PR review time rose 91% and bugs per developer 9% on high-adoption teams. Teams ship more change than they can review.

Will AI replace QA engineers?

The evidence does not point that way. The US Bureau of Labor Statistics projects 10% growth from 2025 to 2035 for software developers, QA analysts and testers combined; the QA role shifts toward requirements, exploratory testing and independent verification.

What is the difference between developer testing and QA?

Developers test their intent: that the code does what they meant. QA tests the user's reality: what people will actually do, including the edge cases nobody wrote a ticket for.

Related posts

Get an estimate for your product.