# AI-Assisted Testing with Agents, MCP, and CLI
Course: [Playwright](https://www.devtutorials.ai/learn/playwright) ยท Section: Advanced Playwright and AI
Source: https://www.devtutorials.ai/learn/playwright/advanced/ai-assisted-testing-with-agents-mcp-and-cli
> How AI coding agents work with Playwright, accessibility snapshots as the agent's view of a page, the Playwright MCP server, playwright-cli for coding agents, Playwright Test Agents (planner, generator and healer) with init-agents and seed tests, reviewing AI-written tests for weak locators, fixed waits and missing checks, and safe habits when agents drive browsers.
## AI-Assisted Testing with Agents, MCP, and CLI
- AI coding agents, like Claude Code, GitHub Copilot or Cursor, can now open a browser, click around and write Playwright tests.
- Playwright has three ways to connect them: the **Playwright MCP server**, **playwright-cli**, and **Playwright Test Agents**.
- AI writes tests fast, but not always good tests. In this lesson, you will set up the tools and learn to **review** what they write.
**Simple lesson:** AI can drive the browser and draft tests. You still decide if a test is right.
### A Real-World Example
- Asha asked her coding agent: *"Write a test that a shopper can filter the catalog to Clothing."*
- The agent opened the page through Playwright, read its structure, and wrote this test. Run it.
- This one is good: role-based locators, web-first assertions, a clear check at the end. Later you will see ones that are not.
### What You Will Learn
1. How an agent "sees" a page: accessibility snapshots.
2. The Playwright MCP server.
3. playwright-cli, the command line for coding agents.
4. Playwright Test Agents: planner, generator and healer.
5. Reviewing and fixing AI-written tests.
6. Using AI tools safely.
## How Agents See a Page
### Accessibility Snapshots, Not Pixels
- An agent does not look at screenshots by default. It reads an **accessibility snapshot**: the same role-and-name tree you saw with `ariaSnapshot()`.
- Each element gets a short **ref**, like `e8`. The agent says "click e8", and Playwright clicks that element.
- Snapshots are small and exact, so they are cheaper and more reliable than reading pixels.
The agent reads a snapshot, picks a ref, and Playwright acts on the real page. Then it reads the new snapshot.
**Simple lesson:** agents read the accessibility tree, pick an element by its ref, and Playwright does the action.
## The Playwright MCP Server
### What MCP Is
- **MCP** (Model Context Protocol) is a standard way to give an AI app new tools.
- The **Playwright MCP server** (`@playwright/mcp`) gives the agent browser tools: navigate, take a snapshot, click, type, fill forms, take a screenshot, and more.
- You add it once to your AI app. Then you can say *"open the shop and check that login works"* in plain words.
### Adding It to Your Agent
- Most AI apps read a JSON config like this:
```json
{
"mcpServers": {
"playwright": {
"command": "npx",
"args": ["@playwright/mcp@latest"]
}
}
}
```
- Claude Code has a command for it: `claude mcp add playwright npx @playwright/mcp@latest`.
- Useful options: `--browser firefox`, `--headless`, `--isolated` (a fresh profile in memory), and `--storage-state playwright/.auth/user.json` to start logged in.
- By default the server opens a visible browser with a **persistent profile**, so logins survive between sessions.
An agent with a browser can click anything a person can, including **Buy** and **Delete**. Point it at test servers and test accounts, never at production with real data.
## playwright-cli for Coding Agents
### Commands Instead of Tools
- **playwright-cli** (`@playwright/cli`) does the same job as a command line. The agent runs shell commands like `playwright-cli click e8`.
- After each command it prints the Playwright code it ran and a new snapshot.
- It sends less text to the model than MCP, which keeps tool lists and snapshots in the conversation. That makes it a good fit for coding agents that already have a terminal.
- `playwright-cli install --skills` adds a skill file that teaches the agent the commands.
- Look at the **Ran Playwright code** lines. They are the steps of a test, written with role-based locators.
### WebMCP: Pages That Offer Their Own Tools
- **WebMCP** is an experimental browser feature. A page can register its own tools, like `search_catalog`, for AI agents to call directly.
- Playwright 1.64 added `page.webmcp.tools()` and `page.webmcp.callTool()`. The MCP server exposes page tools as `webmcp_`, and playwright-cli adds `webmcp-list` and `webmcp-call`.
- `page.getByRef('e8')` finds an element by the ref an AI snapshot gave it, so agent steps can turn into test code.
**Simple lesson:** MCP gives an AI app browser tools. playwright-cli gives a coding agent browser commands.
## Playwright Test Agents
### Planner, Generator and Healer
- Playwright ships three ready-made agent definitions for test work:
- **Planner** explores the app and writes a test **plan** in Markdown, in the `specs/` folder.
- **Generator** turns a plan into Playwright test files, checking each locator in a real browser.
- **Healer** runs failing tests, finds the cause, and fixes the test. If the app itself is broken, it can mark the test as skipped and explain why.
- `npx playwright init-agents --loop=claude` writes them for Claude Code. `--loop=vscode` and `--loop=opencode` target those tools.
- Run it again after updating Playwright, so the agents match the new version.
### The Seed Test
- `tests/seed.spec.ts` is the **seed test**. The agents run it first to reach a starting point, with your fixtures, login and setup.
- A good seed test makes the agents' tests look like yours.
### From Plan to Test
- A plan is plain Markdown that people can read and edit before any code is written:
```text
# Catalog filter
## Scenario: shopper filters by category
Seed: tests/seed.spec.ts
1. Open /catalog. Expect "8 products".
2. Choose "Clothing" in the Category list.
3. Expect 2 products: Cotton T-Shirt and Rain Jacket.
```
- The generator turns each step into code, like the first box on this page.
Edit the plan, not only the code. A wrong step in the plan becomes a wrong test. Reviewing a short Markdown plan is much faster than reviewing many test files.
**Simple lesson:** planner writes the plan, generator writes the test, healer fixes failures. The seed test sets the start.
## Reviewing AI-Written Tests
### What to Look For
- **Weak locators**: CSS classes, `nth-child`, long XPath. Ask for roles, labels and test ids.
- **Fixed waits**: `waitForTimeout()`. Replace them with web-first assertions.
- **No real check**: a test that clicks around but never asserts the result.
- **Checks that cannot fail**: `expect(true).toBe(true)`, or checking text the test itself typed.
- **Too much in one test**: one long journey that hides which part broke.
### A Weak Test That Passes
- This agent-written test passes. Read it carefully. What does it prove?
- It waits 3 seconds for nothing. Its locator breaks if the list order changes. And it never checks the cart. It would pass even if **Add to cart** did nothing.
### A Better Version
### When the Healer Steps In
- A test fails after a change. The healer reruns it, reads the error and the page snapshot, and proposes a fix.
- Here the generated test looked for a **Sign in** button, but the page says **Log in**. The healer would update the locator.
- Always check *why* it changed a test. If the page is truly broken, the right fix is in the app, not the test.
- Read the call log. Then change `'Sign in'` to `'Log in'` and run again.
**Simple lesson:** treat AI tests like a teammate's pull request. Read them, run them, and ask what they really prove.
## Using AI Safely
### Good Habits
- **Test environments only.** Give agents test servers, test accounts and fake payment cards.
- **Secrets stay secret.** Use storage state files and environment variables. Do not paste passwords into prompts.
- **Small steps.** Ask for one scenario at a time, then review it.
- **Keep the rules in writing.** A short file in your repo (like `CLAUDE.md` or `AGENTS.md`) can tell agents your conventions: use page objects, prefer `getByRole`, never use `waitForTimeout`.
- **A human merges.** CI runs the AI tests like any others, and a person approves them.
These AI tools change quickly. Check the Playwright docs for the current commands and options before you set them up.
**Simple lesson:** AI on test data, secrets out of prompts, small reviewed steps, and a human in charge.
## Try It in Your Project
- These steps take the lesson from the lab to your own app.
- Connect the Playwright MCP server to your AI assistant, pointed at a test environment.
- Ask the planner for a plan of one feature, and review the plan before generating tests.
- Add a short rules file for agents with your locator and fixture conventions.
**Simple lesson:** use one idea from this lesson on your own app this week.
## TL;DR
- Agents see pages as accessibility snapshots with refs, not as pixels.
- `@playwright/mcp` gives AI apps browser tools. `claude mcp add playwright npx @playwright/mcp@latest` adds it to Claude Code.
- `playwright-cli` gives coding agents browser commands and prints the Playwright code it ran.
- `npx playwright init-agents --loop=claude` adds the planner, generator and healer agents, plus a seed test.
- Review AI tests for weak locators, fixed waits and missing checks.
- Use test environments, keep secrets out of prompts, and keep a human in charge.
## Common mistakes
| Mistake | What happens | Fix |
|---|---|---|
| Merging AI tests without reading them | Tests that prove nothing reach `main` | Review like any pull request |
| Letting an agent use production | Real orders, emails or deleted data | Test servers and test accounts only |
| Accepting every healer change | Real app bugs get "fixed" in the tests | Ask why it changed. Fix the app when it is wrong |
| Pasting passwords into the chat | Secrets end up in logs and history | Storage state and environment variables |
| No seed test or conventions file | Agent tests look nothing like yours | Add a seed test and a short rules file |
## Check Yourself
## Exercises
## What to Remember
- Agents act through accessibility snapshots and refs.
- MCP for AI apps, playwright-cli for coding agents, init-agents for planner, generator and healer.
- A seed test and a short rules file shape what agents write.
- Review AI tests: real checks, good locators, no fixed waits.
- Test data only, secrets out of prompts, a human approves.
Next: **Capstone: Test the Practice Shop**