How To Use Playwright's testInfo.retry To Deal With Flakey Environments

The environment I am working with can be unstable sometimes. To ensure I don't have false negative tests, I need to rerun the failed checks. Playwright makes this easy by allowing you to pass --retries followed by the number of retries.

Skipping certain tests if retried

The problem was that I didn't want to retry certain checks when running my automation suite (1500 checks in total). The solution was found in testInfo. testInfo provides useful information about currently running specs. One of its properties is retry, which returns the current retry number (0 for the initial run). I realized that, in my case, if retry is greater than 0, I could just skip the test.

test.describe("How testInfo works", async () => {  
    test("Using .retry to skip a test", async ({}, testInfo) => {    
        if (testInfo.retry) test.skip(true, "Retries are not supported"); 
        expect(1).toBe(2);  
    });
});

When the test is run:
– the test fails because 1 is not 2.
– testInfo is updated, and the retry property now equals 1.
– the if block says to skip the test if testInfo.retry is null, false, or undefined.
* for better readability, you can write the if block as if (testInfo.retry > 0).

Adding a delay to your test if retried

If your environment is acting up and can't handle the load, you can add artificial waits with .retry. You can increase the wait based on the retry number.

import { waitFor } from "../lib/helpers/waitFor";

test.describe("How testInfo works", async () => {
  test.beforeAll(async ({}, testInfo) => {
    await waitFor(testInfo.retry * 2000);
  });

  test("Check", async () => {
    expect(1).toBe(2);
  });
});

When the test is run:
– the initial run doesn't wait for anything because 0 * 2 = 0.
– if the test fails, the first retry will wait for 2 seconds because 1 * 2 = 2, and so on.
* modify the seconds to your needs.

Here is the code for the waitFor() function:

export async function waitFor(milliseconds = 100) {
  await new Promise((r) => setTimeout(r, milliseconds));
}

Cleaning up data after test retries

testInfo.retry can also be used to clean up data. For example, if you want to delete certain data created by retries but keep what's created by the initial run, you could write the code this way:

test.describe("How testInfo works", async () => {
  test.afterAll(async ({}, testInfo) => {
    if (testInfo.retry > 0) await cleanUpEnvironment();
  });

  test("Check", async () => {
    expect(1).toBe(2);
  });
});

You are only limited by your imagination

These are just examples I used, but you can utilize testInfo.retry for things like names or IDs of the data you create. This might help you track (if needed) what data was created during which retry.

let user = `${randomName} ${testInfo.retry}`
customerId = `${customerId}_${testInfo.retry}`

Let your imagination go wild! =)

The official Playwright docs for retries can be found below!

Retries | Playwright

Introduction

Playwright

I hope you find this useful, and if you did, please ❤️ and subscribe below to receive more useful tips. If you want to reach out to me personally, feel free to connect or message on LinkedIn.