Reporting
By Syncfusion®, Inc. · Last updated
What test reports are for, the list, line and dot terminal reporters, the HTML report and show-report, JSON and JUnit reports for other tools, the GitHub Actions reporter, using several reporters at once, annotations, tags and attachments in reports, and writing a custom reporter.
Reporting
- A test run produces results. A reporter decides how you see them.
- You see a short list while you work. Your team sees a web page after a CI run. Other tools read a file.
- In this lesson, you will try Playwright's reporters, read the HTML report, and write a reporter of your own.
Simple lesson: reporters turn results into something people or tools can read.
A Real-World Example
- After a CI run, the team opens the HTML report in a browser. It lists every test, with errors, steps and traces.
- This project uses the
htmlreporter next tolist. Run it, then open the Report tab.
- Click the Failed chip, then the failing test. The report shows its error, its steps and its output.
What You Will Learn
- The terminal reporters: list, line and dot.
- The HTML report, and
npx playwright show-report. - JSON and JUnit reports for other tools.
- The GitHub Actions reporter.
- Using several reporters at once.
- Annotations, tags and attachments in reports.
- Writing a custom reporter.
Terminal Reporters
list, line and dot
- list prints one line per test with a ✓ or ✘ and the time. It is the default on your computer.
- line prints one line that updates, plus failures. Good for big suites in a terminal.
- dot prints one character per test:
·passed,Ffailed,×failed but will retry,±flaky,°skipped. It is the default on CI. - Choose one on the command line with
--reporter=dot, or in the config withreporter: 'dot'.
- Change the command to
--reporter=listor--reporter=lineand run again to compare.
Simple lesson: list while you work, dot or line for long runs.
The HTML Report
What Is in It
- The
htmlreporter writes a folder,playwright-report, with anindex.htmlyou open in a browser. - The top shows counts: All, Passed, Failed, Flaky and Skipped. Click one to filter. The search box filters by title, file or tag.
- Each test page shows the error, the steps with timings, attachments, screenshots, videos and the trace.
Animation: the HTML report shows filter chips with counts. The Failed chip is clicked, the list narrows to one failing test, and its error and trace link open
All 4 Passed 2 Failed 1 Skipped 1
home page loads @smoke cart › starts empty cart › shows a total cart › coupon codes
cart › shows a total
Error: expect(locator).toHaveText(expected) failed Expected: "Total: ₹899" Received: "Your cart is empty" Traces: trace.zip ▸ open
Filter by status, open a test, read its error and open its trace.
Opening the Report
npx playwright show-reportserves the last report on a local web server and opens it.- By default, the report opens by itself when tests fail on your computer.
open: 'never'turns that off. CI never opens it. outputFolder: 'reports/html'changes the folder.- In the lab, type
npx playwright show-reportin the command bar of the first box after a run. It opens the Report tab, even if the run did not use the html reporter.
The report folder is a complete static website. On CI, upload it as an artifact or publish it, so anyone on the team can open it without running the tests.
Simple lesson: html writes a report website. show-report opens it.
Reports for Other Tools
JSON and JUnit
- json writes every result as JSON. Scripts and dashboards read it.
- junit writes JUnit XML. Most CI systems (Jenkins, Azure DevOps, GitLab) show it as a test tab.
- Give each an
outputFile. Without one, they print to the terminal.
- Open the Report tab and scroll to Report files. Click
jsonandjunitto read them.
The GitHub Actions Reporter
githubprints special::errorlines. GitHub Actions turns them into notes on the changed lines of a pull request.- Use it only on CI:
reporter: process.env.CI ? [['github'], ['html']] : 'list'.
A JUnit or JSON file is only useful if CI keeps it. Point your CI's test-results step at the outputFile, or upload it as an artifact.
Simple lesson: json and junit for tools, github for pull requests.
Making Reports More Useful
Annotations, Tags and Attachments
- Tags in titles (
@smoke) or in{ tag: '@smoke' }show as labels and can be searched. - Annotations add notes:
test.info().annotations.push({ type: 'issue', description: 'https://...' }).test.fixme()andtest.fail()add their own. - Attachments add files or text:
await testInfo.attach('name', { body, contentType }). They are listed on the test page.
Custom Reporters
Writing a Reporter Class
- A custom reporter is a class with hooks. Playwright calls them during the run.
onBegin(config, suite)runs once at the start.onTestEnd(test, result)runs after each test.onEnd(result)runs at the end.- Teams use them to post to chat, send results to a dashboard, or print a summary they like.
- List the file in
reporter, next to a built-in one.
- The options object in the config (
{ slowMs: 200 }) reaches the constructor. - In the lab, custom reporter output appears after the run. A real run calls the hooks while tests run.
Before writing a reporter, search the Playwright docs and npm. Reporters for Allure, Slack, Microsoft Teams, TestRail and many dashboards already exist.
Simple lesson: a reporter is a class with onBegin, onTestEnd and onEnd.
Try It in Your Project
- These steps take the lesson from the lab to your own app.
- Use
dotorlistplushtmlon CI, and upload the report folder. - Add
junitif your CI system has a test results tab. - Add tags and annotations with ticket links to tests that track known bugs.
Simple lesson: use one idea from this lesson on your own app this week.
TL;DR
list,lineanddotprint to the terminal.--reporter=dotpicks one.htmlwrites a report website.npx playwright show-reportopens it.jsonandjunitwrite files for tools. Give them anoutputFile.githubturns failures into pull request notes.reportertakes a list, so you can use several at once.- Tags, annotations and attachments make reports clearer. Custom reporters do the rest.
Common mistakes
| Mistake | What happens | Fix |
|---|---|---|
json or junit without outputFile | The report floods the terminal | Set outputFile |
Only the html reporter on CI | The CI log shows almost nothing | Add dot or list too |
| Not uploading the report folder on CI | Nobody can see why tests failed | Upload playwright-report and test-results |
| The report opening on every local failure when you do not want it | A browser tab pops up each run | ['html', { open: 'never' }] |
| A custom reporter that throws | Its output is lost | Wrap risky code in try and catch |
Check Yourself
Exercises
What to Remember
- Terminal:
listwhile you work,dotorlineon CI. htmlplusshow-reportfor people.jsonandjunitwithoutputFilefor tools.githubadds notes to pull requests.- Use several reporters at once. Add tags, annotations and attachments.
- A custom reporter is a class with a few hooks.
Next: CI/CD and Scaling