Test automation speeds up the testing and development life cycle, but we often notice that sometimes automated UI tests fail unpredictably without an obvious bug in the application.
We call this "Test Flakiness."
The main factors that contribute to test flakiness are:
While these factors can make test automation challenging, modern testing frameworks like Playwright offer effective solutions to combat flakiness.
In this article, we'll discuss seven key playwright techniques to reduce test flakiness and maintain the reliability of automated test cases. I'll draw on my experience working with an e-commerce project, giving you practical insight into its application in a real-world scenario.
Note that to understand this article, you must have some experience with Playwright and TypeScript.
With that out of the way, let’s start with the first technique.
Automatic Waiting in Playwright lets you wait for elements to be ready before interacting with them, helping you avoid unnecessary sleep calls and reducing timing issues.
You can use page.waitForSelector() and elementHandle.waitFor() methods for automatic waiting.
await page.waitForSelector('#login-button', {state: 'visible'});
await page.click('#login-button');In this example, Playwright will automatically wait until the login-button is visible on the page. Only then will it perform the click event. This way, you don't need to worry about how long it takes for the login button to appear under different network conditions.
This approach was particularly helpful when I developed an end-to-end test suite for an e-commerce project with dynamic web elements such as item availability, total review count, and product ratings.
Playwright supports multiple browser contexts within one instance. Using different browser contexts helps maintain a consistent state across tests while preventing contamination from previous test runs. You can use browser.newContext() to create fresh contexts for each test as in the example below.
const context = await browser.newContext();
const page = await context.newPage();
await page.goto('<https://example.com>');This approach isolates test cases to ensure that one test case's state doesn't influence others. In my e-commerce project, I used a new context in every test to avoid mixing up the state of the previous test, as the testing process required checking out several products after adding them to the cart. Different browser contexts helped maintain fresh cookies, local storage, and session data.
In modern web applications, dynamic content often changes. Playwright's locator API offers more robust ways to interact with such dynamic elements compared to traditional locator strategies. Here’s an example:
const submitButton = page.locator('button', { hasText: 'Submit' });
await submitButton.waitFor();Using this approach, you don't need to worry about the exact text of the submit button. This will work even if the button text is "Submit Form" because hasText checks for any matches in buttons containing the text "Submit". This approach was key in reducing flakiness in my e-commerce project as e-commerce sites typically contain more dynamic elements than static content.
Even with automatic waiting, intermittent issues such as network failures may still cause test flakiness. A custom retry implementation can mitigate these issues.
For instance,
const retryOpertaion = async(operation, retries = 3) => { // Maximum retry attempts = 3
for(let i = 0; i < retries; i++) {
try {
return await operation();
} catch(e) {
if(i === retries - 1) throw e;
}
}
};
// Usage
await retryOpertaion(async() => {
await page.click(#submit-button);
});This approach retries an operation for a given number of attempts before failing, which can help overcome intermittent issues. As an example, my e-commerce application's product listing page occasionally takes longer to load, causing failures. This implementation solved such sporadic issues stemming from network failures or page loading delays inside our CI/CD pipeline.
A major cause of flaky tests is leftover state from previous tests. Cleaning up state after each test can resolve this issue. You can use Playwright's afterEach hook to clean up the state between tests.
For an example,
afterEach(async() => {
await page.evaluate(() => {
localStorage.clear();
});
});The code above clears the browser's local storage, ensuring each test starts with a clean slate. I also used this method for my e-commerce application. Part of my test process involved using multiple user accounts to log in and interact with the app. By clearing local storage, I was able to remove previous users' cookies, allowing for more reliable tests.
Efficient error handling and debugging strategies can identify and fix flaky tests. Using debug logs, adding screenshots of the steps, and including screen recordings of the test in the Playwright report are some of the efficient debugging strategies you can use to identify the root causes of the flaky tests.
The code snippet below adds a screenshot and screen recording to the Playwright report.
await page.screenshot({ path: 'after-login-screenshot.png' });
await page.video().saveAs('login-test.mp4');This is my favorite debugging approach because when your test suite is integrated with the CI/CD pipeline, it’s the easiest way to troubleshoot the root cause. The screen recording will help you to understand the exact point of flakiness.
Unreliable network requests can affect test results. Playwright lets you intercept and simulate network requests, which can boost the reliability of tests that use external APIs.
You can use page.route() to mock network responses as in the example below.
await page.route('**/api/data', (route)=> {
route.fulfill({
contentType: 'application/json',
body: JSON.stringfy({ key: 'value' })
});
});You can apply this method, particularly when your project involves third-party APIs, allowing you to replace actual backend requests with mock data to test your application's functionality
Even when using all seven techniques in Playwright, following best practices remains important. Below are some best practices you can follow to maintain robust tests.
waitFor()