AI Test Case Generation: Four Approaches and When to Use Each
Four different products call themselves “AI test case generation,” and they disagree about what a test case even is. One means a paragraph of text describing steps a human will later automate. Another means executable code committed straight into your repository. A third means a running test, already executed, with a bug report attached. Comparing these as if they compete for the same job is why so many pilots end in a shrug.
The disagreement traces back to one split: authoring versus execution. Authoring turns a requirement into a structured test case. Execution turns that test case into something that runs against your application and keeps running as the UI changes. Some tools do only the first job. Others do both in the same pass. The four approaches below split along exactly this line, and the split is the reason “which one is best” is the wrong question. The right one is which job your backlog actually needs done.
Table of contents
The four bets, side by side
| Approach | What it produces | Best for |
|---|---|---|
| Prompting a general-purpose LLM | A text draft of a test case | Early-stage teams, fast edge-case discovery |
| Agent-native generation (coding assistant) | Executable test code in your repo | Engineering-led QA with clean, documented codebases |
| Dedicated AI test platforms | An executable test on the vendor's runtime | Teams needing execution, reporting, and non-technical contributors in one product |
| MCP-powered workflows | A generated test, executed, with results read back | Teams wanting a closed generation-to-execution loop without platform lock-in |
Each row solves a different bottleneck. The rest of this comparison walks through the three places where these four genuinely pull apart.
What each one actually produces
Prompting a general-purpose LLM against a ticket or PRD produces text. Someone still has to translate that text into code that runs. This is the lowest-cost entry point (no tool, no integration, just a well-structured prompt) and it’s a reasonable place to start before spending on anything else. Boldare’s QA team documented this exact workflow in a practical guide on generating test cases from Jira tickets using Claude Code, including how ticket structure changes output quality.
Agent-native generation, using a coding agent like Claude Code, skips the translation step. The agent works with repository-level context rather than an isolated prompt, so it writes runnable test code directly into the project, following whatever conventions already exist there. The catch is that output quality now depends on the repository. A well-documented codebase with consistent testing patterns gets tests that fit right in. An undocumented one gets tests the agent has to guess at, and guessing shows up later as review overhead.
Dedicated AI test platforms go a step further: natural language in, an executable test out, running on the vendor’s own infrastructure across browsers or devices. This is the broadest single product, and the one non-developers can use directly. The output, however, lives inside a proprietary format. Portability is the price of breadth.
MCP-powered workflows close the loop furthest. An agent generates a test case, executes it through a browser automation server such as Playwright MCP, reads the actual result, and can file a bug report with logs and screenshots, all in one session. Boldare’s MCP Server Development team has built this pattern for clients moving coding agents past code generation into execution. What it doesn’t give you is a packaged product. Someone has to wire the pieces together.
What happens when the UI changes next sprint
This is where the four approaches diverge hardest, because it’s the moment maintenance cost either appears or doesn’t. Text drafts from a general-purpose LLM don’t maintain themselves at all. Whoever automated the original draft owns updating it, same as any hand-written test.
Agent-native generation inherits your existing test suite’s maintenance model. If your repo’s tests break on UI changes today, agent-generated ones will too, unless the conventions the agent follows already account for that.
Dedicated AI platforms are built specifically to answer this question: self-healing locators adjust automatically when a selector changes. That’s a genuine advantage, and also a genuine risk. A threshold tuned too loosely can wave through an actual regression as a harmless UI tweak. Self-healing needs the same scrutiny as any other quality gate, not blind trust because it’s automated.
MCP-powered workflows don’t self-heal in the same sense, but the agent re-reads the live page state on every run, so a changed UI produces a fresh, current interaction rather than a stale, brittle one. The trade-off is that this only works for flows the agent is actively re-running, not a dormant regression suite sitting untouched between releases.
How closed is the loop
Ranking the four by how much happens without a human in between: plain prompting closes nothing (text in, text out, human does the rest). Agent-native generation closes the authoring-to-code gap but stops at the repository boundary. Dedicated platforms close generation-to-execution-to-reporting inside their own walls. MCP workflows close the same loop across open tooling, which is the advantage, but also why they need the most setup investment before they run unattended.
None of the four should run fully unattended on flows that touch production data. Read-heavy, low-risk work (smoke tests, exploratory sessions) is where autonomous generation and execution are genuinely production-ready today. Anything that mutates real data still wants a human approval step in the loop.
Where the comparison stops mattering
All four approaches share one limit that no amount of tooling comparison fixes: none of them decide what’s worth testing. An AI can generate ten variations of a login flow in seconds, but if login isn’t your highest-risk path, that volume doesn’t improve quality. Generated coverage and meaningful coverage are different things, and confusing them is the most common reason teams report disappointment after adopting any of the four.
Volume also creates its own debt. A suite that grows from 200 to 2,000 cases because generation got cheap needs someone deciding which of those cases earn a permanent place in the regression suite. AI makes writing tests faster. It doesn’t make that judgment call for you.
Edge cases are the sharpest limit for the two least-execution-connected approaches (plain prompting and agent-native generation): an LLM can invent a plausible edge case that has nothing to do with how the system actually behaves, especially around business logic no one documented anywhere the model can read. Treat that output the way you’d treat a junior tester’s first draft.
If you’re not sure which bottleneck is slowing your team down, an AI Test Acceleration Audit usually answers that faster than piloting all four in parallel.
FAQ
Which of the four should we pilot first? Whichever matches your actual bottleneck. If test cases pile up unwritten, start with prompting. If they’re written but never automated, look at agent-native generation. If maintenance is eating your QA time, a self-healing platform addresses that directly. If you want generation and execution in one loop without vendor lock-in, MCP is the one built for that.
Does self-healing reduce false positives or create them? Both, depending on threshold tuning. It cuts failures from cosmetic changes but can also let a real regression through disguised as one. Tune it the way you’d tune any other quality gate.
Is MCP-based testing ready for production? For read-heavy, low-risk work like smoke tests and exploratory sessions, yes. For flows that mutate production data, keep a human approval step in the loop.
Can any of these replace manual QA judgment? No. All four remove mechanical effort, not the decisions about what matters, what risk looks like, or how to read an ambiguous requirement.
Do generated test cases still need review? Yes, across all four. The review burden moves from writing to reviewing. It doesn’t go away.
Share this article:
