The Definitive Guide to API Test Automation With Playwright: Part 14 - Creating Custom Assertions Through Extending Expect
This week I'll be going back and refactoring some tests adding custom assertions to my repo. This is something I considered implementing early on but decided against it, as I wanted to keep things simple. with the release of 1.39 the Playwright team has released some easier ways to extend your test and expect fixtures through 1 fixture file. See the release notes below.
If you are just now joining us for the first time feel free to check out the introduction post and the playwright-api-test-demo repository which in which all code examples are included.
Why Should You Care?
In the end if what you are doing is working for you, all good you don't have to implement custom assertions in your project. But if you find yourself making the same lengthy assertion over and over again, a custom expect may come in handy.
For example the below assertions can be converted to be much simpler and easier to read.
// I am asserting on each booking in the report array
body.report.forEach((booking) => {
//old
expect(isValidDate(booking.start)).toBe(true);
expect(isValidDate(booking.end)).toBe(true);
expect(typeof booking.title).toBe("string");
// new
expect(booking.start).toBeValidDate();
expect(booking.end).toBeValidDate();
expect(booking.title).toBeString();
});
With the latest 1.39 release, the ability to extend the expect via a fixture + the ability to mergeExpects and mergeTests this simplifies importing fixtures across all your tests! Before now the custom assertions were added to the playwrightconfig.ts file. This can be seen in the below article.
Implementing Custom Expects
So let's begin with implementing some custom expects via fixtures. We'll start with toBeValidDate(). In a previous example we created a helper that we were calling and validating that we received a true back if the date was parsable, today we'll extend the expect file to include this as a custom expect.
// lib/fixtures/toBeValidDate.ts
import { expect as baseExpect } from "@playwright/test";
export { test } from "@playwright/test";
export const expect = baseExpect.extend({
toBeValidDate(received: any) {
const pass = Date.parse(received) && typeof received === "string" ? true : false;
if (pass) {
return {
message: () => "passed",
pass: true,
};
} else {
return {
message: () => `toBeValidDate() assertion failed.\nYou expected '${received}' to be a valid date.\n`,
pass: false,
};
}
},
});
The logic on this custom expect is straight forward, take the received data from the expect, and validate that when using Date.parse(received) parses (and doesn't return NaN, which is a falsey value). From there we pass back the details needed when overwriting an expect.
Take note I am exporting both test and expect in this fixture in order that I have access to test when utilizing this fixture in my test. This is a decision I made, that you don't have to make in your tests. This does allow me to only have 1 test/expect import, rather than importing test from @playwright/test.
// tests/auth/login.post.spec.ts
// If I didn't export test
import { expect } from "lib/fixtures/fixtures"; (more on fixtures below)
import { test } from "@playwright/test";
// Since I did export test I can do this
import { test, expect } from "lib/fixtures/fixtures";
test.describe("auth/login POST requests", async () => {
...
test("POST with no body", async ({ request }) => {
const response = await request.post(`auth/login`, {});
expect(response.status()).toBe(400);
const body = await response.json();
expect(body.timestamp).toBeValidDate();
expect(body.status).toBe(400);
expect(body.error).toBe("Bad Request");
expect(body.path).toBe(`/auth/login`);
});
});
I've also gone ahead and added a few more custom expects some of which won't be used in this project but could be useful for others. The below custom assertion has been used in a big way in my repo at work as we have an API with a lot of potential values depending on the input. When making assertions on a large GET items request with multiple arrays that are returned, we can create more generic assertions for multiple values.
// lib/fixtures/toBeOneOfValues.ts
import { expect as baseExpect } from "@playwright/test";
export { test } from "@playwright/test";
export const expect = baseExpect.extend({
toBeOneOfValues(received: any, array: any[]) {
const pass = array.includes(received);
if (pass) {
return {
message: () => "passed",
pass: true,
};
} else {
return {
message: () => `toBeOneOfValues() assertion failed.\nYou expected [${array}] to include '${received}'\n`,
pass: false,
};
}
},
});
The below custom expects makes asserting that the response is the correct type super easy!
// lib/fixtures/typesExpects.ts
import { expect as baseExpect } from "@playwright/test";
export { test } from "@playwright/test";
export const expect = baseExpect.extend({
toBeOneOfTypes(received: any, array: string[]) {
const pass = array.includes(typeof received) || (array.includes(null) && received == null);
if (pass) {
return {
message: () => "passed",
pass: true,
};
} else {
return {
message: () =>
`toBeOneOfTypes() assertion failed.\nYou expected '${
received == null ? "null" : typeof received
}' type to be one of [${array}] types\n${
array.includes(null)
? `WARNING: [${array}] array contains 'null' type which is not printed in the error\n`
: null
}`,
pass: false,
};
}
},
toBeNumber(received: any) {
const pass = typeof received == "number";
if (pass) {
return {
message: () => "passed",
pass: true,
};
} else {
return {
message: () =>
`toBeNumber() assertion failed.\nYou expected '${received}' to be a number but it's a ${typeof received}\n`,
pass: false,
};
}
},
toBeString(received: any) {
const pass = typeof received == "string";
if (pass) {
return {
message: () => "passed",
pass: true,
};
} else {
return {
message: () =>
`toBeString() assertion failed.\nYou expected '${received}' to be a string but it's a ${typeof received}\n`,
pass: false,
};
}
},
toBeBoolean(received: any) {
const pass = typeof received == "boolean";
if (pass) {
return {
message: () => "passed",
pass: true,
};
} else {
return {
message: () =>
`toBeBoolean() assertion failed.\nYou expected '${received}' to be a boolean but it's a ${typeof received}\n`,
pass: false,
};
}
},
toBeObject(received: any) {
const pass = typeof received == "object";
if (pass) {
return {
message: () => "passed",
pass: true,
};
} else {
return {
message: () =>
`toBeObject() assertion failed.\nYou expected '${received}' to be an object but it's a ${typeof received}\n`,
pass: false,
};
}
},
});
In the below spec you can see all the different custom expects used in one test.
// tests/test.spec.ts
import { test, expect } from "from "lib/fixtures/fixtures"; // Import the custom matchers definition
test.describe("Custom Assertions", async () => {
test("with fixtures", async ({ request }) => {
const response = await request.post(`auth/login`, {});
expect(response.status()).toBe(400);
const body = await response.json();
expect(body.timestamp).toBeValidDate();
const dateStr = "2021-01-01";
expect(dateStr).toBeValidDate();
const number = 123;
expect(number).toBeNumber();
const boolean = true;
expect(boolean).toBeBoolean();
const string = "string";
expect(string).toBeString();
expect(body.status).toBeOneOfValues([400, 401, 403]);
expect(body.status).toBeOneOfTypes(["number", "null"]);
});
});
MergeExpects Fixture
If you were paying attention in the above example you probably noticed that I only had 1 import import { test, expect } from "@fixtures/fixtures"; for all the different fixtures we've added. With the 1.39 release, the playwright team introduced an easy way to merge expect.extend and test.extend allowing you to make your imports less verbose and super clean! The Release notes can be found here.
For our example I created a fixtures.ts file with the below content. I am importing in mergeExpects() which is a new addition with the latest release, along with all the other expect.extend fixtures. I am then creating and exporting a new expect variable setting it equal to the response of mergeExpects(fixture1, fixture2, fixture3, etc). This will create a single fixture that can be imported into all my tests that use these custom assertions.
mergeExpects you will need to update Playwright to atleast 1.39 in your package.json file.// lib/fixtures/fixtures.ts
import { mergeExpects } from "@playwright/test";
import { expect as toBeOneOfValuesExpect } from "lib/fixtures/toBeOneOfValues";
import { expect as toBeValidDate } from "lib/fixtures/toBeValidDate";
import { expect as typesExpects } from "lib/fixtures/typesExpects";
export { test } from "@playwright/test";
export const expect = mergeExpects(toBeOneOfValuesExpect, toBeValidDate, typesExpects);
But before we start importing the fixture, let's update our tsconfig.json and add the @fixtures relative path, and updat the previous tests and fixtures.ts file with the new imports.
// tsconfig.json
{
"compilerOptions": {
"baseUrl": ".",
"esModuleInterop": true,
"paths": {
"@datafactory/*": ["lib/datafactory/*"],
"@helpers/*": ["lib/helpers/*"],
"@fixtures/*": ["lib/fixtures/*"]
}
}
}
Importing should look like this with the above change
// new
import { test, expect } from "@fixtures/fixtures";
// old
import { test, expect } from "lib/fixtures/fixtures";
You can see all the changes that were added in the below pull request, across the repository. I didn't do a find and replace on "@playwright/test" though I could have.
playwrightsolutionsWeird Error I Experience in VSCode
I am still a bit perplexed on one error I continue to get when attempting toe use some of the custom expects. See below.
Property 'toBeValidDate' does not exist on type 'MakeMatchers<void, any,
{ toBeOneOfValues(this: State, received: any, array: any[]):
{ message: () => string; pass: true; }
| { message: () => string; pass: false; }; }
& { toBeValidDate(this: State, received: any):
{ ...; }
| { ...; }; }
& { ...; }>'.ts(2339) any
I suspect this may be a bug with the Playwright codebase as it only shows this error when there is an expect(any) type, in the below example body.timestamp is type any, as it doesn't get set until the response async call is made. If I change body.timestamp with a string of the timestamp the IDE error goes away. If you have any ideas reach out to me on linked (see below for a link), and let me know about it!
A Little Extra: Asserting API Request Duration
I spent a good bit of time attempting to find a way to measure the duration of an API call. I initially tried creating a request fixture but could never get this to work as I could capture the duration but there was no way to pass this duration calculation to the request object to be used in the response assertions, I could however print this out out console.log within the assertion. So instead I added a duration to the test, a way to do this can be found below. It is very verbose and I don't love it, but it at least is a path forward.
// tests/auth/login.post.spec.ts
//COVERAGE_TAG: POST /auth/login
import { test, expect } from "@fixtures/fixtures";
import Env from "@helpers/env";
test.describe("auth/login POST requests", async () => {
const username = Env.ADMIN_NAME;
const password = Env.ADMIN_PASSWORD;
test("POST with valid credentials", async ({ request }) => {
// Calculating Duration
const start = Date.now();
const response = await request.post(`auth/login`, {
data: {
username: username,
password: password,
},
});
// Calculating Duration
const end = Date.now();
const duration = end - start;
// Asserting Duration
expect(duration).toBeLessThan(1000);
expect(response.status()).toBe(200);
const body = await response.text();
expect(body).toBe("");
expect(response.headers()["set-cookie"]).toContain("token=");
});
});
Wrapping Up
The latest features in the 1.39 release should make managing imports for fixtures way easier, as we can merge fixtures to our hearts content! I believe this will have a far greater impact on extending test via fixtures than expect as this could make managing page objects through fixtures even easier.
Thanks for reading! If you found this helpful, reach out and let me know on LinkedIn or consider buying me a cup of coffee. If you want more content delivered to you in your inbox subscribe below, and be sure to leave a ❤️ to show some love.