How do I run a subset of tests with Playwright Test Runner using grep functionality?

When running your playwright tests depending on how large your suite is, you will get to a point where you may only want to run a subset of tests. You can achieve this goal through a few different ways (setting up different projects, different repos, etc..), but today I'll cover how you do this utilizing grep and tags within your test names all within the same repository using grep.

For grep functionality there are two places in which you can configure what parameters are passed in. In the playwright.config.ts file or by passing --grep or -g in command line parameters. Both can be useful depending on what your objectives are.

Using @Tags in Your Tests

The grep functionality works very similarly to the Linux grep command where it will search through the entire text of each of your describe titles and test titles. If there are matches they are added to the list of tests that are executed. The Playwright documentation recommends it a best practices to use the @ sign in your tag to indicate this is being used as a tag for grouping tests. One thing to note is the test.step titles are not searched for the grep functionality.

Below we'll create a basic test to show off how the tagging can be used with results.

//grep.spec.ts

import { test, expect } from "@playwright/test";

test.describe("Grep Describe @1", async () => {

  test("Test A @2", async ({ page, baseURL }) => {
    await page.goto(baseURL + "/login");
    expect(page.getByTestId("login-email")).toBeEmpty;
  });

  test("Test B @3", async ({ page, baseURL }) => {
    await page.goto(baseURL + "/login");
    expect(page.getByTestId("login-email")).toBeEmpty;
  });
});

Running the tests via CLI

The below examples show off different ways to trigger different tests based on different inputs.

// Runs both Test A and Test B
$ npx playwright test -g @1

// Runs only Test A
$ npx playwright test -g @2

// Runs only Test B
$ npx playwright test -g @3

// Runs only Test A
$ npx playwright test -g @1 --grep-invert @3

// Runs only Test B (note grep is case sensative)
$ npx playwright test -g B

// Runs  both Test A and Test B (this triggered off of the test.describe title)
$ npx playwright test -g scribe

// runs both Test A and Test B
$ npx playwright test -g "@2|@3"

Each of these examples can be added to the package.json file with different triggers.

In my day to day work with Playwright I've found these to be some helpful tags.

• @happy - used for any tests we want to be run at a minimum (Happy Path Tests!). We run these against our preview environments (feature branches) before code is merged into our trunk environment.
• @unsatisfactory - we use this tag for items that are flakey or need attention. We use the --grep-invert to remove these tests from our CI pipeline and run these tests once a day on a schedule. These will either be refactored or deleted.
• @smoke - used for tests that are 'Read Only', don't create data, that can be run against production environments without creating any data.
• @scenario - we use these for tests that are multi step, multi page work flows, such as a customer journey.
• @feature_name - replace feature_name with whatever feature you want to be able to run in isolation. This can also be achieved by adding sub folders, but what is nice about tags is you are able to run all tags across the entire suite of tests.

Grep in playwright.config.ts

You are also able to add grep as an option in the playwright.config.ts file. I don't recommend this is something that should be used, as when you add grep to your playwright config your cli grep already has those constraints. If using the below config grep: /@happy|@smoke/ with the above test example, none of the tests would run even just running npx playwright test as they are excluded from the playwright.config.ts level.

//playwright.config.ts

const config: PlaywrightTestConfig = {
  globalSetup: "./global-setup",
  globalTeardown: "./global-teardown",
  use: {
    baseURL: process.env.APP_URL,
    browserName: "chromium",
    headless: true,
  },
  testDir: "./tests",
  
  workers: 2,
  grep: /@happy|@smoke/,
};

Utilizing package.json to make your life easier

Last year when I learned about the power of the pacakge.json file my life was changed for the better. Being able to remember every single fully typed out command to run your tests is hard, and utilizing the pacakge.json will make your life easier.

You can read more about how this works here, but the easiest way to explain this is npm exposes a key value relationship, where you can use the command npm run {key} replacing key with the name of your script. This will then execute the value. See the example below.

//package.json

{
 "scripts": {
    "1": "npx playwright test -g @1",
    "2": "npx playwright test -g @2",
    "3": "npx playwright test -g @3",
    "AB": "npx playwright test -g '@2|@3'",
    "A": "npx playwright test -g @1 --grep-invert @3",
    "test:local": "BASE_URL=http://localhost:80 npx playwright test"
  },
}

// This will run bottom most script named "A"
$ npm run A

// What is executed from the above command
> npx playwright test -g @1 --grep-invert @3

Where this gets really powerful is you can build out the different types of ways you'd like to run your playwright commands including passing in different hard coded environment variables into your tests. Example code can be found in my playwright-demo repository.

Some Useful Links

• Playwright Test Config + Test config parameter
• Playwright Command Line interface
• Useful discussion as a GitHub issue, that I found on my learning journey.

Thanks for reading! If you found this helpful, reach out and let me know on LinkedIn or consider buying me a cup of coffee. If you want more content delivered to you in your inbox subscribe below.