How Agentic AI is changing testing landscape

How Agentic AI Is Changing the Testing Landscape

For a long time, test automation had a fairly simple equation: humans write tests, machines execute them. We got very good at the second part. Frameworks grew up around Selenium, Playwright, Appium, JUnit, TestNG, Cucumber and a long list of supporting libraries, and on top of them we built page objects, reusable functions, test data utilities, API clients and reporting layers.

But the expensive part was never really the execution. It was creating and maintaining the automation. A small UI change could send someone hunting for a broken locator. An API contract change could ripple through several tests. A new feature meant understanding the application, deciding what should be tested, writing the test, and then maintaining it as the product kept evolving.

Generative AI started changing this equation, and now agentic AI is pushing it a step further. The interesting change isn't that an AI can generate a Playwright test or write a JUnit class. That's useful, but it isn't fundamentally new. The bigger change is that an agent can start operating across the whole testing lifecycle: understand → plan → author → validate → repair → maintain.

That changes what we should expect from test automation, and it changes how we should build it. The rest of this post walks through what that looks like in practice: first for coded automation, then for no-code, and finally for the platform architecture that has to sit underneath both.

Part 1: Two Worlds, One Rule

I see the emerging landscape broadly dividing into two areas: coded automation, where AI agents work alongside developers and testers to produce executable code, and no-code automation, where agents work from higher-level business and application context to construct complete test flows.

What matters here is that these two approaches aren't really competing. They solve different problems. In both cases, though, one rule matters more than anything else:

Authoring can be agentic. Execution should be deterministic.

This is going to matter more, not less, as agents take on more of the work. An AI agent is probabilistic. A test execution engine should not be. You do not want an LLM deciding, in the middle of a regression run, whether it "thinks" the next step should be clicking a button, calling an API, or skipping an assertion.

The agent should help create and maintain the test. Once the test is created, execution should happen through a deterministic automation engine with no token dependency. Think of it as: AI for creating the test, code for running it. Everything below is really just this one idea, applied to different parts of the testing lifecycle.

Part 2: Coded Automation, Rebuilt Around Specifications

AI Becomes the Test Author

The first and most obvious transformation is happening in coded automation. Instead of asking an AI:

"Write me a Selenium test for this login page."

we can start giving it something far more useful:

"Here is the specification. Here are the business rules. Here is the API contract. Here are the acceptance criteria. Generate the tests."

That's a very different workflow. The important asset is no longer the prompt. It's the specification and context.

Unit and Integration Tests Start With a Good Specification

AI is already quite good at writing unit tests. Give a coding agent a function and ask it to generate tests, and it can produce dozens of test cases. The problem was never really whether the AI can write the test. It's whether it knows what the software is supposed to do. If the implementation is the only source of truth, the AI tends to reproduce the implementation's own assumptions.

That's where a specification-driven approach earns its keep. Instead of:

Code → AI → Tests

we move toward:

Specification
     ↓
Test Scenarios
     ↓
AI-generated Unit / Integration Tests
     ↓
Compilation + Execution
     ↓
Feedback
     ↓
AI refinement

The human's job moves up the abstraction layer: humans define the expected behaviour, and AI handles much of the mechanical work of translating that behaviour into tests. For example, a specification might say:

Given a customer with an active account

When the customer requests a withdrawal

Then the withdrawal should succeed if the available balance
is greater than or equal to the requested amount.

The balance must be reduced by the withdrawal amount.

A withdrawal greater than the available balance must be rejected.

A negative withdrawal amount must be rejected.

An agent can turn that into happy-path tests, boundary tests, negative tests and parameterized integration tests, complete with the mocks, fixtures and assertions around them. The important point is that the specification remains the human-controlled source of truth, which is much safer than asking AI to "look at the code and figure out what to test."

API Testing: Contracts Become the Starting Point

API testing is an even better candidate for this approach. We already have structured representations of API behaviour: OpenAPI specifications, request/response schemas, authentication requirements, status codes, validation rules, business rules, error conditions. Imagine handing an agent the OpenAPI contract together with the business behaviour, the authentication rules and the known edge cases, and asking it to generate the API test suite. The agent can then reason about combinations that would otherwise require a tester to manually enumerate.

For example, given:

POST /customers

Required:
- firstName
- lastName
- email

Rules:
- email must be unique
- email must be valid
- firstName cannot be empty
- customer cannot be created without authentication

the agent can generate a matrix that covers the valid request, missing authentication, each missing or invalid field, a duplicate email, empty and null values, unexpected fields and boundary values, all without anyone typing each case out by hand.

Again, the notable part isn't that AI can generate HTTP requests. Tools have been doing that for years. What matters is that an agent can take contracts, behaviour and context together and construct a meaningful test model from them.

E2E Testing: Where Agents Start to Feel Like Test Engineers

End-to-end testing has always had a higher authoring cost, because the test has to understand the application, the UI, navigation, locators, state, test data, authentication, the APIs underneath, browser behaviour and the assertions that tie it all together. This is where tools such as Playwright and MCP-based workflows earn their place.

A Playwright MCP server can give an agent access to a browser. The agent can observe the application, interact with it, and use that interaction to author a test. Other tools and MCP servers can extend the context further:

                ┌───────────────┐
                │   AI Agent    │
                └───────┬───────┘
                        │
        ┌───────────────┼────────────────┐
        ↓               ↓                ↓
   Playwright MCP    API tools      Application docs
        ↓               ↓                ↓
      Browser        Backend          Specs

The agent can use those sources together: discovering the UI through Playwright, inspecting an API contract, understanding the expected behaviour from a specification, and then producing a Playwright test from all of it. This starts to look like an AI test engineer rather than an AI code-completion tool.

Keep the Agent Out of the Execution Loop

This is one of the principles I'd strongly recommend: don't make every test execution an LLM interaction. Suppose the agent creates:

test("customer can transfer money", async ({ page }) => {
    await page.getByRole("button", { name: "Transfer" }).click();
    ...
});

Once that test has been authored, the execution should simply be CI/CD driving Playwright, driving a browser, producing assertions and a result. There should be no requirement to call an LLM for every step. Why? Because execution needs determinism, speed, reproducibility, predictable cost, security and auditability, and an LLM call in the loop reliably gives you none of those.

The agent belongs in authoring, maintenance, failure analysis and repair, not in re-deciding every test step on every run. In other words: make the authoring agentic, but keep the execution tokenless. This architecture also makes the economics of the whole system far more sensible.

Maintenance Is Where Agentic Automation Could Matter Most

Creating a test once is not the hardest problem. Maintaining thousands of them is. Consider a UI change:

Before:
<button>Submit</button>

After:
<button>Continue</button>

A traditional automation framework just sees a broken locator, and a human has to go investigate it. A locator-healing system might find the new element on its own. An agent can go further: it can understand why the test failed, inspect the changed UI, identify the replacement element, verify that the replacement represents the same intent, update the test, run it, check whether the change introduced a semantic problem, and open the result as a change for review.

That's a meaningful difference. Locator healing is not the same thing as test maintenance. A locator healer answers "which element is probably the same element?" An agent can potentially answer "what changed, why did the test fail, and what should the test look like now?" That's a far bigger question, and one that doesn't necessarily require generative AI at runtime. There's a lot that deterministic algorithms can already do around locator similarity, DOM structure and element attributes. AI becomes valuable when we need broader context and reasoning on top of that.

Part 3: No-Code Automation, Rebuilt Around Context

The Same Shift Is Happening Here Too

Now we get to the area I find most compelling. No-code automation traditionally tries to remove the coding layer, but removing code doesn't automatically remove complexity. A user still has to know what the application does, which flow to automate, which data to use, which assertions matter, how the different screens connect, and what the expected business outcome actually is.

So the next generation of no-code automation shouldn't simply be "click here and record this." It should become "understand this business scenario and construct the automation." Imagine a user says:

"Create a test for a customer registering for a new account, verifying their email and completing KYC."

That sentence alone isn't enough. A useful automation agent needs context: product documentation, API specifications, the application's structure, existing test cases, user roles, test data, business rules, authentication flows, reusable components, known dependencies, existing page objects. It combines those sources to construct the flow, which is very different from a generic chatbot generating steps.

The quality of the automation ends up being determined by the quality of the context supplied to the agent. That's why I think context engineering will become a core competency in AI-powered testing platforms.

The No-Code Platform Becomes an Agentic Orchestrator

A traditional no-code platform might look like:

User
 ↓
Recorder
 ↓
Steps
 ↓
Execution Engine

An agentic platform starts looking more like:

                    ┌──────────────────┐
                    │    AI Agent      │
                    └────────┬─────────┘
                             │
             ┌───────────────┼───────────────┐
             ↓               ↓               ↓
        Application       Business        Existing
          Context          Rules           Tests
             │               │               │
             └───────────────┼───────────────┘
                             ↓
                      Test Flow Model
                             ↓
                    Deterministic Engine
                             ↓
                       Test Execution

The platform can use agents to build the flow, but the same rule from coded automation still applies: the resulting flow should be represented in a deterministic model, not run through the agent itself.

Reusability Becomes Even More Important

This is another area where the architecture of a no-code platform matters. If every generated test is a completely independent sequence of actions, AI may simply generate a huge amount of duplication. A better architecture gives the agent reusable primitives (login, create customer, search customer, complete KYC, approve customer, generate token) that it then composes into flows.

Instead of:

Test A → 27 raw steps
Test B → 31 raw steps
Test C → 24 raw steps

we get:

Login
Create Customer
Complete KYC
    ↓
Test A

Login
Create Customer
Approve Customer
    ↓
Test B

This is essentially the same architectural thinking behind reusable steps and page-object-style abstractions in coded automation. AI doesn't eliminate good automation architecture. It makes good architecture even more important.

Part 4: What Doesn't Change

Engineering Principles Still Apply

There's a temptation with agentic AI to think "the agent can figure it out." That's exactly where testing platforms become unreliable. We still need abstraction, modularity, reusable components, contracts, versioning, deterministic execution, assertions, test data management, observability and traceability. AI should reduce the amount of manual work required to use these principles. It shouldn't eliminate the principles themselves.

The best AI testing systems will probably look less like autonomous magic and more like well-engineered systems with an intelligent authoring layer.

The Future Test Automation Stack, in Three Layers

I'd describe it as three layers.

Layer 1: Human intent

Humans provide the specifications, business rules, acceptance criteria, API contracts, application context and test strategy. Human judgment carries the most weight here.

Layer 2: Agentic authoring

Agents handle test design, scenario generation, code generation, flow construction, locator discovery, test maintenance, failure analysis and repair proposals. This is where the biggest productivity gains will happen.

Layer 3: Deterministic execution

The actual test runs through Playwright, Selenium, Appium, API clients, unit-test frameworks, integration frameworks and database or test-data utilities. No token is required to execute the test.

Put together, this gives us human intent → AI-generated automation → deterministic execution, a far more grounded model than "AI runs your tests."

Part 5: Building One Yourself

The Shape of the System

The natural question is: can we actually build such a system ourselves? Yes, and the architecture is surprisingly approachable. You do not need to build a giant autonomous AI platform. You can start with a workflow engine such as LangGraph and build a small set of specialized agents, for example:

Specification
     ↓
Requirement Analyzer
     ↓
Test Designer
     ↓
Tool Selector
     ↓
Test Author
     ↓
Validator
     ↓
Repair Agent
     ↓
Test Artifact

Each node has a specific responsibility, which is far easier to control than one giant prompt saying "you are an autonomous QA engineer, test this application."

A Simple LangGraph Model

A simplified implementation can look like this:

from typing import TypedDict
from langgraph.graph import StateGraph, END

class TestState(TypedDict):
    specification: str
    scenarios: list
    test_code: str
    validation_result: str

def analyze_requirement(state: TestState):
    # Call the LLM with the specification
    # and produce structured scenarios.
    return {
        "scenarios": [
            "successful customer registration",
            "duplicate email registration",
            "invalid email registration"
        ]
    }

def generate_test(state: TestState):
    # Generate executable test code
    # from the approved scenarios.
    return {
        "test_code": "// generated Playwright test"
    }

def validate_test(state: TestState):
    # Compile / execute / statically validate
    # the generated artifact.
    return {
        "validation_result": "passed"
    }

graph = StateGraph(TestState)

graph.add_node("analyze", analyze_requirement)
graph.add_node("generate", generate_test)
graph.add_node("validate", validate_test)

graph.set_entry_point("analyze")

graph.add_edge("analyze", "generate")
graph.add_edge("generate", "validate")
graph.add_edge("validate", END)

app = graph.compile()

This is deliberately simple. The important idea is the separation of responsibilities: the graph becomes the orchestration layer, the LLM provides reasoning where reasoning is useful, and the actual test remains an artifact that can be executed independently.

Add MCP: Giving the Agent Eyes and Hands

Now add tools. For an E2E scenario, the agent could have access to a Playwright MCP for browser interaction, an API MCP for inspecting and exercising endpoints, a documentation MCP for product context, a repository MCP for existing automation and source code, and a test management MCP for existing scenarios and requirements.

The agent is no longer answering questions from a prompt. It's operating against a tool-enabled testing environment. At that point, agentic testing starts becoming genuinely different from traditional AI-assisted coding.

Give Agents Boundaries

An agent with unrestricted access to a test environment is not automatically a good test engineer. We need guardrails. An agent should be free to inspect the application, generate tests, run them, inspect failures and propose changes. But it shouldn't be able to modify production data, change business configuration, silently delete tests, alter approved specifications, or bypass assertions.

The agent should also produce an audit trail: what it observed, what it inferred, what test it generated, what it changed, why it changed it, and what evidence supports the change. This matters more and more as AI starts modifying automation on its own.

Part 6: What This Means Going Forward

The Biggest Shift Is Not "AI Writes Tests"

That framing undersells it. The real shift is:

AI is moving testing from automation of execution toward automation of engineering work around execution.

We already automated execution decades ago. What's new is automating the requirement analysis, the test design, the authoring, the validation, the maintenance, the failure analysis and the repair that surround it. The test itself becomes almost a generated artifact, and the specification and context behind it matter more than the artifact itself.

What Happens to Test Engineers?

This doesn't mean testers disappear. It changes where their value sits. A tester who spends most of their time manually writing repetitive automation code is going to see a lot of that work automated. But a tester who understands product behaviour, risk, test strategy, specifications, system architecture, data, contracts, failure modes and automation architecture becomes even more valuable.

The role moves upward. Instead of asking:

"How do I write this test?"

the better question becomes:

"What should this system prove, and what context does an agent need to prove it?"

That's a far better problem to have.

The Future Is Not No-Code vs Code

I don't think coded automation dies while no-code automation wins. It's more likely to look like:

Human intent
       ↓
Context
       ↓
Agent
       ↓
Automation artifact
       ↓
Deterministic execution

Sometimes that artifact will be Java, sometimes TypeScript, sometimes Python, sometimes a no-code flow. The user shouldn't necessarily care. What matters is whether the generated artifact is understandable, maintainable, reusable, versioned, testable and deterministic.

Where I Think This Is Going

The testing tools we have today were largely designed around a world where humans were responsible for authoring everything. The next generation will be designed around a different assumption: humans define intent, and AI handles much of the translation into automation.

That doesn't mean giving an AI a browser and saying "go test this." It means building a system where the AI has the right context, the right tools and the right constraints. The architecture matters. The context matters. The specifications matter. And deterministic execution matters.

For me, the model that matters isn't agentic execution. It's agentic test engineering.

Let the agent explore. Let the agent reason. Let the agent author. Let the agent maintain. Let the agent investigate failures. But once the test is ready to run, take the AI out of the critical execution path. Run the test as code. Run it fast. Run it repeatedly. Run it without tokens.

That combination, agentic authoring with deterministic execution, is where I think the next major evolution of test automation will happen.