Reporting

By Syncfusion®, Inc. · Last updated

What test reports are for, the list, line and dot terminal reporters, the HTML report and show-report, JSON and JUnit reports for other tools, the GitHub Actions reporter, using several reporters at once, annotations, tags and attachments in reports, and writing a custom reporter.

Reporting

  • A test run produces results. A reporter decides how you see them.
  • You see a short list while you work. Your team sees a web page after a CI run. Other tools read a file.
  • In this lesson, you will try Playwright's reporters, read the HTML report, and write a reporter of your own.

Simple lesson: reporters turn results into something people or tools can read.

A Real-World Example

  • After a CI run, the team opens the HTML report in a browser. It lists every test, with errors, steps and traces.
  • This project uses the html reporter next to list. Run it, then open the Report tab.
  • Click the Failed chip, then the failing test. The report shows its error, its steps and its output.

What You Will Learn

  1. The terminal reporters: list, line and dot.
  2. The HTML report, and npx playwright show-report.
  3. JSON and JUnit reports for other tools.
  4. The GitHub Actions reporter.
  5. Using several reporters at once.
  6. Annotations, tags and attachments in reports.
  7. Writing a custom reporter.

Terminal Reporters

list, line and dot

  • list prints one line per test with a ✓ or ✘ and the time. It is the default on your computer.
  • line prints one line that updates, plus failures. Good for big suites in a terminal.
  • dot prints one character per test: · passed, F failed, × failed but will retry, ± flaky, ° skipped. It is the default on CI.
  • Choose one on the command line with --reporter=dot, or in the config with reporter: 'dot'.
  • Change the command to --reporter=list or --reporter=line and run again to compare.

Simple lesson: list while you work, dot or line for long runs.

The HTML Report

What Is in It

  • The html reporter writes a folder, playwright-report, with an index.html you open in a browser.
  • The top shows counts: All, Passed, Failed, Flaky and Skipped. Click one to filter. The search box filters by title, file or tag.
  • Each test page shows the error, the steps with timings, attachments, screenshots, videos and the trace.

Animation: the HTML report shows filter chips with counts. The Failed chip is clicked, the list narrows to one failing test, and its error and trace link open

All 4 Passed 2 Failed 1 Skipped 1

home page loads @smoke cart › starts empty cart › shows a total cart › coupon codes

cart › shows a total

Error: expect(locator).toHaveText(expected) failed Expected: "Total: ₹899" Received: "Your cart is empty" Traces: trace.zip ▸ open

Filter by status, open a test, read its error and open its trace.

Opening the Report

  • npx playwright show-report serves the last report on a local web server and opens it.
  • By default, the report opens by itself when tests fail on your computer. open: 'never' turns that off. CI never opens it.
  • outputFolder: 'reports/html' changes the folder.
  • In the lab, type npx playwright show-report in the command bar of the first box after a run. It opens the Report tab, even if the run did not use the html reporter.

The report folder is a complete static website. On CI, upload it as an artifact or publish it, so anyone on the team can open it without running the tests.

Simple lesson: html writes a report website. show-report opens it.

Reports for Other Tools

JSON and JUnit

  • json writes every result as JSON. Scripts and dashboards read it.
  • junit writes JUnit XML. Most CI systems (Jenkins, Azure DevOps, GitLab) show it as a test tab.
  • Give each an outputFile. Without one, they print to the terminal.
  • Open the Report tab and scroll to Report files. Click json and junit to read them.

The GitHub Actions Reporter

  • github prints special ::error lines. GitHub Actions turns them into notes on the changed lines of a pull request.
  • Use it only on CI: reporter: process.env.CI ? [['github'], ['html']] : 'list'.

A JUnit or JSON file is only useful if CI keeps it. Point your CI's test-results step at the outputFile, or upload it as an artifact.

Simple lesson: json and junit for tools, github for pull requests.

Making Reports More Useful

Annotations, Tags and Attachments

  • Tags in titles (@smoke) or in { tag: '@smoke' } show as labels and can be searched.
  • Annotations add notes: test.info().annotations.push({ type: 'issue', description: 'https://...' }). test.fixme() and test.fail() add their own.
  • Attachments add files or text: await testInfo.attach('name', { body, contentType }). They are listed on the test page.

Custom Reporters

Writing a Reporter Class

  • A custom reporter is a class with hooks. Playwright calls them during the run.
  • onBegin(config, suite) runs once at the start. onTestEnd(test, result) runs after each test. onEnd(result) runs at the end.
  • Teams use them to post to chat, send results to a dashboard, or print a summary they like.
  • List the file in reporter, next to a built-in one.
  • The options object in the config ({ slowMs: 200 }) reaches the constructor.
  • In the lab, custom reporter output appears after the run. A real run calls the hooks while tests run.

Before writing a reporter, search the Playwright docs and npm. Reporters for Allure, Slack, Microsoft Teams, TestRail and many dashboards already exist.

Simple lesson: a reporter is a class with onBegin, onTestEnd and onEnd.

Try It in Your Project

  • These steps take the lesson from the lab to your own app.
  • Use dot or list plus html on CI, and upload the report folder.
  • Add junit if your CI system has a test results tab.
  • Add tags and annotations with ticket links to tests that track known bugs.

Simple lesson: use one idea from this lesson on your own app this week.

TL;DR

  • list, line and dot print to the terminal. --reporter=dot picks one.
  • html writes a report website. npx playwright show-report opens it.
  • json and junit write files for tools. Give them an outputFile.
  • github turns failures into pull request notes.
  • reporter takes a list, so you can use several at once.
  • Tags, annotations and attachments make reports clearer. Custom reporters do the rest.

Common mistakes

MistakeWhat happensFix
json or junit without outputFileThe report floods the terminalSet outputFile
Only the html reporter on CIThe CI log shows almost nothingAdd dot or list too
Not uploading the report folder on CINobody can see why tests failedUpload playwright-report and test-results
The report opening on every local failure when you do not want itA browser tab pops up each run['html', { open: 'never' }]
A custom reporter that throwsIts output is lostWrap risky code in try and catch

Check Yourself

Exercises

What to Remember

  • Terminal: list while you work, dot or line on CI.
  • html plus show-report for people. json and junit with outputFile for tools.
  • github adds notes to pull requests.
  • Use several reporters at once. Add tags, annotations and attachments.
  • A custom reporter is a class with a few hooks.

Next: CI/CD and Scaling