Skip to content

Generative AI fails after the demo, not before it

Mykhailo PoliarushUpdated 5 min read

In 2026, generative AI is everywhere in companies, but real business value is still rare and concentrated in a few organizations. In McKinsey's State of AI 2025, 88% of respondents said their organizations use AI regularly, yet only 39% attributed any EBIT impact to it. Building a demo is easy; most efforts fail between the pilot and a product that works in production.

How widely are companies using generative AI in 2026?

Almost everyone uses it, and few have scaled it. In the McKinsey survey of 1,993 participants, about one-third of respondents said their companies had begun to scale AI programs, and most of those reporting EBIT impact put it at less than 5% of EBIT. A Gartner survey of 1,303 respondents in early 2026 found that only 22% of organizations have scaled AI across multiple business units.

Spending keeps rising anyway. Menlo Ventures estimates that companies spent $37 billion on generative AI in 2025, up from $11.5 billion in 2024, and that 76% of AI use cases are now bought rather than built. Menlo is an investor in AI companies, including Anthropic, so treat these as one investor's estimates. For model benchmarks, agents and enterprise model spend, see The state of AI engineering in 2026.

Did Gartner's prediction about abandoned GenAI projects come true?

In July 2024, Gartner predicted that at least 30% of generative AI projects would be abandoned after proof of concept by the end of 2025, citing poor data quality, inadequate risk controls, escalating costs and unclear business value. That deadline has passed, and Gartner has not published a final count on that release. Its newer data points the same way.

In the October 2026 Gartner survey of 2,002 CIOs and technology executives, only 13% said their enterprise had realized significant or transformational value from AI, and the likelihood that an AI use case succeeds was still one in five. Only 31% of CIOs had rules for stopping a pilot before production, and only 17% had rules for abandoning low-value AI already in production. That survey covers generative AI, traditional AI and agents together.

AI value and governance as reported by CIOs

More pressure to deliver cost improvement from AI
58%
Expect AI to increase total cost of ownership
51%
Have rules for stopping a pilot before production
31%
Have rules for abandoning low-value AI in production
17%
Realized significant or transformational value
13%
Have plans to redesign roles for humans and agents
4%

Source: Gartner, 2027 CIO and Technology Executive Agenda survey (n=2,002, May-July 2026)

The next forecast is already out. Gartner predicts that over 40% of agentic AI projects will be canceled by the end of 2027, for the same reasons. Forecasts are not measurements, but the pattern is consistent: projects fail on value, cost and risk, not on whether the model can produce text.

What are the main challenges between a generative AI pilot and production?

Because a pilot proves that a model can answer, not that a product can work inside real workflows. The MIT NANDA report from July 2025 found that 60% of organizations evaluated task-specific generative AI tools, 20% reached a pilot and only 5% reached production with a sustained impact. General-purpose chatbots did much better, because they are easy to try and need no integration.

From evaluation to production: share of organizations at each stage

From evaluation to production: share of organizations at each stage
CriterionStageGeneral-purpose LLMsTask-specific GenAI tools
Investigated80%60%
Piloted50%20%
Successfully implemented40%5%

Source: MIT NANDA, The GenAI Divide: State of AI in Business 2025 (July 2025; 52 organizations interviewed, 153 survey responses; directional)

The same report claims that 95% of organizations get zero return from generative AI. Its authors call their figures "directionally accurate", based on interviews with 52 organizations and 153 survey responses, so read the headline with care. Its more useful finding is about approach: in its sample, external partnerships with customized tools reached deployment about 67% of the time, against about 33% for internal builds. The authors blame brittle workflows, missing context and poor fit with daily work, not model quality.

Does retrieval-augmented generation stop wrong answers?

It reduces them, but it does not remove them. Retrieval-augmented generation (RAG) is now standard: Menlo Ventures reports that prompt design is the most common customization technique, followed by RAG. Grounding helps, but only when retrieval finds the right sources and the model uses them correctly.

The best public test of that claim comes from law. A preregistered study by Magesh and colleagues found that RAG-based legal research tools from LexisNexis and Thomson Reuters hallucinated between 17% and 33% of the time, fewer errors than GPT-4 but far from the "hallucination-free" marketing. The study dates from 2024 and the tools have changed since, but the lesson holds: you have to measure accuracy on your own questions.

Wrong answers already cost companies. In McKinsey's 2025 survey, 51% of respondents from organizations using AI reported at least one negative consequence, and nearly one-third reported consequences from AI inaccuracy. Security adds more failure modes. The OWASP Top 10 for LLM Applications 2025 ranks prompt injection first and lists vector and embedding weaknesses, a risk specific to RAG systems.

What does a generative AI product cost to build and run?

More than the pilot suggests. Gartner's 2024 release put the cost of building generative AI into business models at $5 million to $20 million for those deployment approaches. In the 2026 CIO survey, 51% of CIOs expected AI to increase the total cost of ownership of technology, and 58% faced more pressure to deliver cost savings from it. About 11% of organizations in Gartner's other 2026 survey did not know what their function spent on AI in 2025.

Unit prices are falling fast. Epoch AI measured LLM inference prices dropping between 9x and 900x per year, depending on the performance level, though it notes the fastest declines may not last. Cheaper tokens do not make a product cheap: usage grows, retrieval and evaluation add calls, and OWASP lists unbounded consumption as a top risk. Cost control is a design task, with budgets, caching and model routing built in from the start.

What do teams that get value from generative AI do differently?

They treat it as engineering, with clear rules for stopping. McKinsey's AI high performers, about 6% of respondents, are nearly three times as likely as others to have fundamentally redesigned workflows. They are also more likely to have defined processes for when model outputs need human validation. In Gartner's 2026 survey, organizations with rules for stopping pilots were 2.3 times more likely to be "vanguards", the CIOs reporting both better business outcomes and high perceived AI value.

A practical way to start:

  1. Pick one task with a measurable outcome, such as answer accuracy, handling time or deflection rate, and record the baseline.
  2. Build an evaluation set from real cases before you build the feature, and run it on every change to prompts, models or retrieval.
  3. Ground answers in your data with citations, and test retrieval quality on its own, not only the final answer.
  4. Test for prompt injection and data leaks using the OWASP list as a checklist.
  5. Set a token budget and a stop rule. Decide in advance what result moves the pilot to production and what result ends it.

How can SDCLabs help you ship a generative AI product?

SDCLabs builds generative AI features the same way it builds other software: AI agents draft the code, prompts and tests, senior engineers review and decide what ships, and automated tests check every change. As founders of Codeception and CodeceptJS, we treat model output as something to test. Every feature gets an evaluation suite, guardrails and cost monitoring. Engagements start in 1 to 2 weeks.

See our generative AI development services or contact us. If you need to test an AI system you already run, see AI testing services.

Sources checked on 2026-11-17.

Frequently asked questions

Did 30% of generative AI projects get abandoned by the end of 2025?

Gartner predicted in July 2024 that at least 30% would be abandoned after proof of concept by the end of 2025, but has not published a final count on that release. Its October 2026 survey found that only one in five AI use cases succeeds.

How many companies get real value from generative AI?

Few. Only 13% of CIOs in Gartner's 2026 survey report significant or transformational value from AI, and only 39% of McKinsey's 2025 respondents attribute any EBIT impact to it.

Does RAG eliminate hallucinations?

No. A preregistered study by Magesh and colleagues found that RAG-based legal research tools still hallucinated between 17% and 33% of the time, so accuracy has to be measured with evaluations on real questions.

Why do generative AI pilots fail to reach production?

In the MIT NANDA report, only 5% of organizations took task-specific generative AI tools to production, with brittle workflows, missing context and poor fit with daily work as the main causes.

Related posts

Get an estimate for your product.