The Definitive Guide to API Test Automation With Playwright: Part 2 - Adding More In-Depth Checks
Welcome Back! If you missed the Introduction and Part 1 make sure you understand the context of what we are doing and why. In this article I will focus on adding depth to our automated api checks. This will allow us to cover both positive and negative scenarios for each endpoint was want to check.
If you are following along with the codebase on GitHub you will notice in the P/R that I am going ahead and taking steps to build out tools in order to create test data in way that allows my tests to run in parallel. I had planned to wait to add this at a later time but my code was getting out of hand, and needed to be wrangled a bit. I'll cover the helpers in detail in a future blog post but feel free to check out the code if you'd like.
playwrightsolutionsWe used the docs for the booking endpoint to understand what we were testing and came up with a plan to get started. As we continue down our journey feel free to have the docs open and available.
Let's add some depth to our coverage!
On the past few projects I've worked on, I've found a really good strategy that can be used when starting from scratch, is to first work towards adding coverage breadth first. What I mean by that is to add the most happy path scenarios you can think of for each endpoint, and at some point or when you get close to 70-80% of the breadth coverage, start adding deep or depth coverage to your most critical endpoints, and eventually to all of them. This should allow you to get decent coverage and then focus your efforts on specific risks. This strategy is useful for UI automation as well.
Taking this approach will help give you a sense of the pain points you will face, areas where you may want to add data factories and helper functions so you can keep your code DRY.
Let's jump into the code!
In part 1 we finished with 1 file in the tests/booking/ folder named booking.get.spec.ts. I did this for a specific reason as it will help keep our tests organized versus putting all our files in 1 big folder. The naming also follows a convention, {name}.{method}.spec.ts. When we finished part 1 we had 3 different GET tests in our spec file, in this article we will finish with 8 in that directory.
The first thing worth calling out in the in the GET spec file is I've taken a the function we had to check if a date was valid, and moved it to a folder under /lib/helpers and imported that file along with creating an auth api call that I imported under the /lib/datafactory folder.
test.beforeAll()
The beginning of my test now looks like this, at the highest level I go ahead and create a let variable for cookies, in which I update in the beforeAll block. This method takes in a username and password, and returns the cookies string so that when I need to add an authorization header it will look like headers: { cookie: cookies }
import { test, expect } from "@playwright/test";
import { auth } from "../../lib/datafactory/auth";
import { isValidDate } from "../../lib/helpers/date";
test.describe("booking/ GET requests", async () => {
let cookies;
test.beforeAll(async ({ request }) => {
cookies = await auth("admin", "password");
});
...
Additional GET Request Checks
For the booking GET api calls there are only 3 different paths which are
booking/summary?roomid=1(GET booking summary with specific room id)-
booking/(GET all bookings with details, requires auth) booking/1(GET booking by id with details, requires auth)
For the first booking/summary?roomid={id} I added the following tests
- With valid room id
- With number room id that doesn't exist
- With no room id provided
test("GET booking summary with specific room id", async ({ request }) => {
const response = await request.get("booking/summary?roomid=1");
expect(response.status()).toBe(200);
const body = await response.json();
expect(body.bookings.length).toBeGreaterThanOrEqual(1);
expect(isValidDate(body.bookings[0].bookingDates.checkin)).toBe(true);
expect(isValidDate(body.bookings[0].bookingDates.checkout)).toBe(true);
});
test("GET booking summary with specific room id that doesn't exist", async ({
request,
}) => {
const response = await request.get("booking/summary?roomid=999999");
expect(response.status()).toBe(200);
const body = await response.json();
expect(body.bookings.length).toBe(0);
});
test("GET booking summary with specific room id that is empty", async ({
request,
}) => {
const response = await request.get("booking/summary?roomid=");
expect(response.status()).toBe(500);
const body = await response.json();
expect(isValidDate(body.timestamp)).toBe(true);
expect(body.status).toBe(500);
expect(body.error).toBe("Internal Server Error");
expect(body.path).toBe("/booking/summary");
});
Before going too much further, I was very intentional about my spacing, as you can see I try and break up areas in my assertions, first a break between making the request and the first assertion. Then after that any time I set a new variable that I want to assert against, I have a space above. Whether you adopt this styling or any other type of styling, as you build out more tests, having consistent styling can make working with a codebase a delight.
In each of these scenarios I asserted both on the response.status() and made some assertions on the response body. For these specific tests, I kept them very generic as with GET requests, I don't have 100% control over the data that gets returned, i.e. what if someone else is using the system and creating data that my test doesn't know about, it is unable to assert against the specific data, but can do things like assert on the type of information that is returned: isValidDate, is a string, is greater than 1, is a boolean, is a number, etc. In our below example we don't have a lot of things to test against, and I also know that this checkin date of 2022-02-01 will always be present as it is data that is created by the platform, if this didn't exist we would need to make a POST call to create data in a beforeAll or beforeEach to assert against.
// booking/summary?roomid=1 response body
{
"bookings": [
{
"bookingDates": {
"checkin": "2022-02-01",
"checkout": "2022-02-05"
}
}
]
}
The next set of tests GET booking/ actually requires authentication. there isn't any additional parameters we can pass in so I keep the checks simple
- Get all bookings with details with authentication
- Get all bookings with details without authentication
test("GET all bookings with details", async ({ request }) => {
const response = await request.get("booking/", {
headers: { cookie: cookies },
});
expect(response.status()).toBe(200);
const body = await response.json();
expect(body.bookings.length).toBeGreaterThanOrEqual(1);
expect(body.bookings[0].bookingid).toBe(1);
expect(body.bookings[0].roomid).toBe(1);
expect(body.bookings[0].firstname).toBe("James");
expect(body.bookings[0].lastname).toBe("Dean");
expect(body.bookings[0].depositpaid).toBe(true);
expect(isValidDate(body.bookings[0].bookingdates.checkin)).toBe(true);
expect(isValidDate(body.bookings[0].bookingdates.checkout)).toBe(true);
});
test("GET all bookings with details with no authentication", async ({
request,
}) => {
const response = await request.get("booking/", {
headers: { cookie: "test" },
});
expect(response.status()).toBe(403);
const body = await response.text();
expect(body).toBe("");
});
In these tests we have some hard coded values that are used as static data, bookingId #1. So in our assertions we did go ahead and hard code this data. Short term I think this is an ok approach, but there is some risk with it.
What happens if someone makes a put request on bookingId #1, and changes any of the details? If you guessed our test would fail you are correct. In this scenario the better practice would be to refactor this to create a booking, and then assert on that created booking, We'll do this later, but for now knowing that the data gets reset every 10 minutes to a default state it is a pretty low risk.
The final set of checks in this file are GET booking/{id} which also requires authentication.
booking/1(GET booking by id with details with authentication)booking/999999(GET booking by id that doesn't exist with authentication)booking/1(GET booking by id without authentication)
test("GET booking by id with details", async ({ request }) => {
const response = await request.get("booking/1", {
headers: { cookie: cookies },
});
expect(response.status()).toBe(200);
const body = await response.json();
expect(body.bookingid).toBe(1);
expect(body.roomid).toBe(1);
expect(body.firstname).toBe("James");
expect(body.lastname).toBe("Dean");
expect(body.depositpaid).toBe(true);
expect(isValidDate(body.bookingdates.checkin)).toBe(true);
expect(isValidDate(body.bookingdates.checkout)).toBe(true);
});
test("GET booking by id that doesn't exist", async ({ request }) => {
const response = await request.get("booking/999999", {
headers: { cookie: cookies },
});
expect(response.status()).toBe(404);
const body = await response.text();
expect(body).toBe("");
});
test("GET booking by id without authentication", async ({ request }) => {
const response = await request.get("booking/1");
expect(response.status()).toBe(403);
const body = await response.text();
expect(body).toBe("");
});
The first check in this set is also checking specifics of the static data that we are expecting to be there. One thing to note is we are asserting on every single key in the response body. I've found this is a great practice to follow, assert as specific as you can, in this scenario I decided to only assert on the booking date is a valid date, not the actual hardcoded value.
At this point we have 8 API checks(115 lines of code) with decent coverage around all the GET booking endpoints, there is always ways to go deeper, but I feel confident if there were any breaking changes made to these endpoints, we would be alerted via a check failing, and can take action to explore deeper.
POST Request Checks
Alright let's get 🌶️ with a POST request. I won't go for depth on this set of tests yet as there are still PUT and DELETE requests in this article to go. The following check took some time to put together because there were some helper and data factory functions that needed to get created to make the post spec simpler. I won't go into the details of those functions but high level.
createRandomBookingBody- requires a room id, checkin date and checkout date and creates a body that we can use to POST to thebooking/endpoint. 💡We can also use this to assert on the response body!!!futureOpenCheckinDate- requires a room id and returns a date string2023-03-31T00:00:00.000Z(this uses the get bookings api and does some quick maths)stringDateByDays- requires a date string and an optional number. It will take the number and add or subtract days based on todays date and return a date string2023-03-24
import { test, expect } from "@playwright/test";
import {
createRandomBookingBody,
futureOpenCheckinDate,
} from "../../lib/datafactory/booking";
import { stringDateByDays } from "../../lib/helpers/date";
test.describe("booking/ POST requests", async () => {
let requestBody;
let roomId = 1;
test.beforeEach(async ({ request }) => {
let futureCheckinDate = await futureOpenCheckinDate(roomId);
let checkInString = futureCheckinDate.toISOString().split("T")[0];
let checkOutString = stringDateByDays(futureCheckinDate, 2);
requestBody = await createRandomBookingBody(
roomId,
checkInString,
checkOutString
);
});
test("POST new booking with full body", async ({ request }) => {
const response = await request.post("booking/", {
data: requestBody,
});
expect(response.status()).toBe(201);
const body = await response.json();
expect(body.bookingid).toBeGreaterThan(1);
const booking = body.booking;
expect(booking.bookingid).toBe(body.bookingid);
expect(booking.roomid).toBe(requestBody.roomid);
expect(booking.firstname).toBe(requestBody.firstname);
expect(booking.lastname).toBe(requestBody.lastname);
expect(booking.depositpaid).toBe(requestBody.depositpaid);
const bookingdates = booking.bookingdates;
expect(bookingdates.checkin).toBe(requestBody.bookingdates.checkin);
expect(bookingdates.checkout).toBe(requestBody.bookingdates.checkout);
});
});
With each of these functions in the beforeEach block, I am able to add additional tests and have new request body data, and available dates to check in for each test I want to write. The actual POST request test will be making a new booking with the full body filled out (this includes required and optional parameters), note this API call does not require auth, so no headers are passed in, and I pass in the requestBody which was generated through our createRandomDataBody() function.
What's really nice about creating our Body the way we did (not hardcoding it) is now I can use the variable and data we set for the postBody, and use it in our assertions.
// POST booking response body
{
"bookingid": 2,
"booking": {
"bookingid": 2,
"roomid": 1,
"firstname": "Testy",
"lastname": "McTesterSon",
"depositpaid": true,
"bookingdates": {
"checkin": "2023-05-10",
"checkout": "2023-05-11"
}
}
}
The below assertions are also in the same order as the response body, this is on purpose in order to stay organized! Note that in this scenario I take the body which is the full json response and create a new variable booking and use this for assertions. I like to do this as it helps me quickly visualize I'm asserting on the booking object, and below that the bookingdates object. For small API responses it may not be that big of a deal, but when you have a response body with 20-50 items in the json object or you want to iterate through an array of objects, you'll be really happy that you followed this pattern.
const body = await response.json();
expect(body.bookingid).toBeGreaterThan(1);
const booking = body.booking;
expect(booking.bookingid).toBe(body.bookingid);
expect(booking.roomid).toBe(requestBody.roomid);
expect(booking.firstname).toBe(requestBody.firstname);
expect(booking.lastname).toBe(requestBody.lastname);
expect(booking.depositpaid).toBe(requestBody.depositpaid);
const bookingdates = booking.bookingdates;
expect(bookingdates.checkin).toBe(requestBody.bookingdates.checkin);
expect(bookingdates.checkout).toBe(requestBody.bookingdates.checkout);
We could go further and add tests such as...
- POST booking with only required inputs
- POST booking missing a required input
- POST booking with bad types ("true" instead of true or "1" instead of 1)
- POST booking with authentication though its not required
- POST booking with dates that overlap with an existing booking
- POST booking with dates in the past
- etc...
but we've validated the endpoint with this set of parameters is doing what we expect, so lets move on!
DELETE Request Checks
For this set of checks again I am going to use the data factory functions I've created createFutureBooking and auth to setup the test. Notice I am creating let variables in the describe block, some that have values set, and others that will get their values set in the beforeAll (if it's not changing) or in the beforeEach block which will assign a new value to the variable each time before a test is run. I went ahead and hardcoded the roomId in this test as 1 as I know the system will have it available, but this is a risk because someone could delete roomId 1, in that case I would need to create a room, and a booking for that room to execute our test of deleting a booking. The 3 tests I'm including are:
- DELETE booking with specific room id
- DELETE booking with an id that doesn't exist
- DELETE booking id without authentication
import { test, expect } from "@playwright/test";
import { auth } from "../../lib/datafactory/auth";
import {
getBookingSummary,
createFutureBooking,
} from "../../lib/datafactory/booking";
test.describe("booking/{id} DELETE requests", async () => {
let cookies;
let bookingId;
let roomId = 1;
test.beforeAll(async () => {
cookies = await auth("admin", "password");
});
test.beforeEach(async () => {
let futureBooking = await createFutureBooking(roomId);
bookingId = futureBooking.bookingid;
});
test("DELETE booking with specific room id:", async ({ request }) => {
const response = await request.delete(`booking/${bookingId}`, {
headers: { cookie: cookies },
});
expect(response.status()).toBe(202);
const body = await response.text();
expect(body).toBe("");
const getBooking = await getBookingSummary(bookingId);
expect(getBooking.bookings.length).toBe(0);
});
test("DELETE booking with an id that doesn't exist", async ({ request }) => {
const response = await request.delete("booking/999999", {
headers: { cookie: cookies },
});
expect(response.status()).toBe(404);
const body = await response.text();
expect(body).toBe("");
});
test("DELETE booking id without authentication", async ({ request }) => {
const response = await request.delete(`booking/${bookingId}`);
expect(response.status()).toBe(403);
const body = await response.text();
expect(body).toBe("");
});
});
Within the first test, you can see that we first create a booking in the beforeEach block, we use that booking id that is passed into the delete method along with the headers as we need authorization for this. This is another pattern that I always use, whatever Endpoint that I plan to do the main assertions against, will always have the response variable assigned to it. This gives us advantages as we continue to build out or tests in the future.
const response = await request.delete(`booking/${bookingId}`, {
headers: { cookie: cookies },
});
For this first test I make 2 assertions, 1 on the response.status() and the other expecting the response.text() to be an empty string "". For me I wanted to validate further that the booking was actually deleted, so rather than use the playwright request method in the test I went ahead and created another data factory to getBookingSummary(bookingId) which returns the body of the GET booking/summary?roomid=${bookingId} as something I can assert against. In this case I want to ensure the bookings.length toBe 0.
PUT Request Checks
This next section I left somewhat verbose in that I am creating a new putBody within each test section, there are lots of ways to organize the code. I could have abstracted this away similar to the way I create a request body on the post new booking endpoint. In this situation I wanted it to be super clear what information was getting passed in, as one of my tests, I try and make a PUT request without the first name present.
One pattern I am following, is I am creating all of my variables within the describe block. This allows me to have access to the variables within the test and test.steps for assertions. Currently I am hard coding some of the variables as well. I could use a tool like faker to make the data unique, if I did this I would set the variable within the beforeEach() so that each tests gets a new name. It would look like this.
test.describe("booking/{id} PUT requests", async () => {
let firstname;
test.beforeEach(async ({ request }) => {
firstname = faker.name.firstName()
...
});
The checks that I am putting in place are:
- PUT booking with specific room id + Verify booking was updated
- PUT booking without firstname in putBody
- PUT booking with an id that doesn't exist
- PUT booking id that is text
- PUT booking id with invalid authentication
- PUT booking id without authentication
- PUT booking id without put body
In my opinion this is one of the more interesting endpoints in that there are a lot of different combinations of things you can test for.
One specific area I'd like to call out is in the first test PUT booking with specific room id I use the test.step() method to help break up the test. In my test.step I am actually using the getBookingById() a data factory method I created to return the current body for the Booking ID sent. 💡 IMPORTANT: if you use test.step, make sure you use await in front of test.step, if you miss this, like I did while writing my tests the first time, you will be banging your head on the keyboard trying to figure out what is going on.
For me, I found I was missing the await by trying to make one of the assertions in the test.step to fail.... I couldn't in my testing and went back to the Playwright docs and found this issue.
await test.step("Verify booking was updated", async () => {
const getBookingBody = await getBookingById(bookingId);
expect(getBookingBody.bookingid).toBeGreaterThan(1);
expect(getBookingBody.bookingid).toBe(bookingId);
expect(getBookingBody.roomid).toBe(putBody.roomid);
expect(getBookingBody.firstname).toBe(putBody.firstname);
expect(getBookingBody.lastname).toBe(putBody.lastname);
expect(getBookingBody.depositpaid).toBe(putBody.depositpaid);
const getBookingDates = getBookingBody.bookingdates;
expect(getBookingDates.checkin).toBe(putBody.bookingdates.checkin);
expect(getBookingDates.checkout).toBe(putBody.bookingdates.checkout);
});
When I was first writing the automation for these tests, I ran into a lot of 409 error messages from the application under test. The code stands for conflict. The specific conflict was around checkin and checkout dates that were already in use which caused me to go ahead and create the data factory methods that made our lives easier writing these tests. Without the functions I would have spent a lot of time manually updating dates in my troubleshooting, and we would have hardcoded data which would eventually cause more 409s and inconsistency in our tests.
These data factory methods allowed me to have the test create the data it needs independently of any other test data or tests.
You shouldn't rely on Test 1 to setup Test 2s data. This is a trap!!
I'm specifically proud of futureOpenCheckinDate() and createFutureBooking(). I've included the jsdoc description I created for each below. (I just learned about jsdoc and I am loving it!!!)
All the PUT request tests can be found below.
import { test, expect } from "@playwright/test";
import { auth } from "../../lib/datafactory/auth";
import {
getBookingById,
futureOpenCheckinDate,
createFutureBooking,
} from "../../lib/datafactory/booking";
import { isValidDate, stringDateByDays } from "../../lib/helpers/date";
test.describe("booking/{id} PUT requests", async () => {
let cookies;
let bookingId;
let roomId = 1;
let firstname = "Happy";
let lastname = "McPathy";
let depositpaid = false;
let email = "[email protected]";
let phone = "5555555555555";
let futureBooking;
let futureCheckinDate;
test.beforeAll(async () => {
cookies = await auth("admin", "password");
});
test.beforeEach(async ({ request }) => {
futureBooking = await createFutureBooking(roomId);
bookingId = futureBooking.bookingid;
futureCheckinDate = await futureOpenCheckinDate(roomId);
});
test(`PUT booking with specific room id`, async ({ request }) => {
let putBody = {
bookingid: bookingId,
roomid: roomId,
firstname: firstname,
lastname: lastname,
depositpaid: depositpaid,
email: email,
phone: phone,
bookingdates: {
checkin: stringDateByDays(futureCheckinDate, 0),
checkout: stringDateByDays(futureCheckinDate, 1),
},
};
const response = await request.put(`booking/${bookingId}`, {
headers: { cookie: cookies },
data: putBody,
});
expect(response.status()).toBe(200);
const body = await response.json();
expect(body.bookingid).toBeGreaterThan(1);
const booking = body.booking;
expect(booking.bookingid).toBe(bookingId);
expect(booking.roomid).toBe(putBody.roomid);
expect(booking.firstname).toBe(putBody.firstname);
expect(booking.lastname).toBe(putBody.lastname);
expect(booking.depositpaid).toBe(putBody.depositpaid);
const bookingdates = booking.bookingdates;
expect(bookingdates.checkin).toBe(putBody.bookingdates.checkin);
expect(bookingdates.checkout).toBe(putBody.bookingdates.checkout);
await test.step("Verify booking was updated", async () => {
const getBookingBody = await getBookingById(bookingId);
expect(getBookingBody.bookingid).toBeGreaterThan(1);
expect(getBookingBody.bookingid).toBe(bookingId);
expect(getBookingBody.roomid).toBe(putBody.roomid);
expect(getBookingBody.firstname).toBe(putBody.firstname);
expect(getBookingBody.lastname).toBe(putBody.lastname);
expect(getBookingBody.depositpaid).toBe(putBody.depositpaid);
const getBookingDates = getBookingBody.bookingdates;
expect(getBookingDates.checkin).toBe(putBody.bookingdates.checkin);
expect(getBookingDates.checkout).toBe(putBody.bookingdates.checkout);
});
});
test("PUT booking without firstname in putBody", async ({ request }) => {
let putBody = {
bookingid: bookingId,
roomid: roomId,
lastname: lastname,
depositpaid: depositpaid,
email: email,
phone: phone,
bookingdates: {
checkin: stringDateByDays(futureCheckinDate, 0),
checkout: stringDateByDays(futureCheckinDate, 1),
},
};
const response = await request.put(`booking/${bookingId}`, {
headers: { cookie: cookies },
data: putBody,
});
expect(response.status()).toBe(400);
const body = await response.json();
expect(body.error).toBe("BAD_REQUEST");
expect(body.errorCode).toBe(400);
expect(body.errorMessage).toContain(
"Validation failed for argument [0] in public org.springframework.http.ResponseEntity"
);
expect(body.fieldErrors[0]).toBe("Firstname should not be blank");
});
test("PUT booking with an id that doesn't exist", async ({ request }) => {
let putBody = {
bookingid: bookingId,
roomid: roomId,
firstname: firstname,
lastname: lastname,
depositpaid: depositpaid,
email: email,
phone: phone,
bookingdates: {
checkin: stringDateByDays(futureCheckinDate, 0),
checkout: stringDateByDays(futureCheckinDate, 1),
},
};
const response = await request.delete("booking/999999", {
headers: { cookie: cookies },
data: putBody,
});
expect(response.status()).toBe(404);
const body = await response.text();
expect(body).toBe("");
});
test(`PUT booking id that is text`, async ({ request }) => {
let putBody = {
bookingid: bookingId,
roomid: roomId,
firstname: firstname,
lastname: lastname,
depositpaid: depositpaid,
email: email,
phone: phone,
bookingdates: {
checkin: stringDateByDays(futureCheckinDate, 0),
checkout: stringDateByDays(futureCheckinDate, 1),
},
};
const response = await request.put(`booking/asdf`, {
headers: { cookie: cookies },
data: putBody,
});
expect(response.status()).toBe(404);
const body = await response.json();
expect(isValidDate(body.timestamp)).toBe(true);
expect(body.status).toBe(404);
expect(body.error).toBe("Not Found");
expect(body.path).toBe("/booking/asdf");
});
test("PUT booking id with invalid authentication", async ({ request }) => {
let putBody = {
bookingid: bookingId,
roomid: roomId,
firstname: firstname,
lastname: lastname,
depositpaid: depositpaid,
email: email,
phone: phone,
bookingdates: {
checkin: stringDateByDays(futureCheckinDate, 0),
checkout: stringDateByDays(futureCheckinDate, 1),
},
};
const response = await request.put(`booking/${bookingId}`, {
headers: { cookie: "test" },
data: putBody,
});
expect(response.status()).toBe(403);
const body = await response.text();
expect(body).toBe("");
});
test("PUT booking id without authentication", async ({ request }) => {
let putBody = {
bookingid: bookingId,
roomid: roomId,
firstname: firstname,
lastname: lastname,
depositpaid: depositpaid,
email: email,
phone: phone,
bookingdates: {
checkin: stringDateByDays(futureCheckinDate, 0),
checkout: stringDateByDays(futureCheckinDate, 1),
},
};
const response = await request.put(`booking/${bookingId}`, {
data: putBody,
});
expect(response.status()).toBe(403);
const body = await response.text();
expect(body).toBe("");
});
test("PUT booking id without put body", async ({ request }) => {
const response = await request.put(`booking/${bookingId}`, {
headers: { cookie: cookies },
});
expect(response.status()).toBe(400);
const body = await response.json();
expect(isValidDate(body.timestamp)).toBe(true);
expect(body.status).toBe(400);
expect(body.error).toBe("Bad Request");
expect(body.path).toBe(`/booking/${bookingId}`);
});
});
As always there is always more assertions and checks we can add, but for now my confidence is boosted, and I feel if there are any breaking changes introduced to the booking endpoints, our automation should alert us, so we can go investigate and explore how the larger system will be affected.
Overall we have 19 checks that ran on my local machine in 15.9s using 4 workers
If you have made it this far, give yourself a gold ⭐️, you deserve it!
Thanks for reading! If you found this helpful, reach out and let me know on LinkedIn or consider buying me a cup of coffee. If you want more content delivered to you in your inbox subscribe below.