In today's fast-paced development landscape, delivering reliable and high-quality software is paramount. To achieve this, end-to-end testing plays a crucial role in ensuring that all components of your application work seamlessly together.
For a long time Selenium has been the default option for writing automated browser tests, which was good. But there is a better alternative to it, which is Playwright. Playwright, a powerful open-source automation framework, is designed to simplify end-to-end testing across different browsers and devices. In this blog post, we will explore how to leverage Playwright for end-to-end testing and integrate it into an automated build process using Azure Pipelines.
Understanding Playwright:
Playwright, developed by Microsoft, is an innovative Node JS based automation framework that provides a single API for browser automation across Chromium, WebKit, and Firefox. Its unique architecture allows it to interact with browsers using the latest browser automation technologies, resulting in reliable and fast end-to-end tests.
It is compatible with Cross-platforms like Windows, Linux, MacOS, headless or headed, and can be integrated with major CI/CD servers such as Azure pipelines, GitHub Actions, Jenkins or Circle CI.
It supports multiple programming languages such as TypeScript, JavaScript, Python, .NET, Java, giving more options to QAs writing test scripts.
It supports native emulation of mobile web applications for Google Chrome for Android and mobile Safari for iOS.
How it works?
Playwright uses a WebSocket connection to the browsers to control it directly without any middle layer. The WebSocket connection stays open from the beginning of the test until the end, and all instructions are sent via that one connection. This explains the performance, particularly why the playwright tests are faster.
Contrary to Selenium which uses a webdriver (library that translates your test into JSON which browsers can understand) to send each command as a separate HTTP request to the browser, acting as a middle layer.
Key Features of Playwright Automation:
Auto-waits:
Playwright performs various actionability checks on elements prior to performing requested actions. It also has a rich set of introspection events. It means that the testing framework can focus on other aspects of the script instead of controlling the waiting through code, making the tests resilient, no flaky tests and less error-prone.
Browser Contexts:
Playwright creates a browser context for each test. Browser context is equivalent to a brand-new browser profile. This delivers full test isolation with zero overhead.
Parallel Browser Testing:
Playwright also supports the execution of simultaneous tests (aka parallel testing) through Browser Context and can run parallel tests with multiple browsers. This scales up testing and comes in handy when multiple web pages must be tested simultaneously.
Network Control:
Test scripts can be used to customize the conditions under which an application is tested by automatically simulating file uploads and downloads, handling various authentication methods, intercepting network requests, and mocking out request responses. It allows for more accurate testing, ensuring that software functions correctly in multiple scenarios.
Device Emulation:
Playwright enables you to emulate various mobile and desktop devices, allowing you to test your web applications on different screen sizes and resolutions. This feature ensures that your application looks and behaves as expected on a wide range of devices.
Built-in report:
Playwright Test comes with a few built-in reporters for different needs and ability to provide custom reporters.
Powerful Tooling:
Codegen. Generate tests by recording your actions. Save them into any language.
Playwright inspector. Inspect page, generate selectors, step through the test execution, see click points, explore execution logs.
Trace Viewer. Capture all the information to investigate the test failure. Playwright trace contains test execution screencast, live DOM snapshots, action explorer, test source, and many more.
Setting up Playwright:
Pre-Requisites:
- Install Visual Studio Code: Download and Install Visual Studio Code(VSCode).
- Install NodeJS: Download and Install Node JS
- Azure DevOps Account. Sign up here to get one
Step1: Create a new directory(eg: learn-playwright) in VS Code
Step 2: Open VS Code and click on File> Open Folder > Choose the newly created folder(learn-playwright))
Step 3: Click on Terminal Menu > New Terminal
Step 4: Enter the below command to start the playwright installation
npm init playwright@latest --yes -- --quiet --browser=chromium --browser=firefox --browser=webkit –gha
The above command does the following operation:
- Creates Package.json
- Installs npm library
- Downloads and installs web browsers for Chromium, Firefox and Webkit.
- Sets up basic Files and Folders
- tests folder: This folder contains actual test scripts. By default, an example.spec.ts file will be created inside this folder.
- .gitignore: This file helps if you are using git repository
- package.json and package-lock.json: This file helps to track dependencies, create a shortcut for running tests, etc.
- playwright.config.ts: This is the global configuration file for the Playwright, which you can configure with available options.
Create the first playwright test
Step 5: Navigate inside the tests folder and create a test spec file ex: demo.spec.ts
Let’s start a test case with the below scenario
Scenario:
- Navigate to https://www.saucedemo.com/
- Enter wrong credentials
- Verify Error Message is displayed or not.
The demo.spec.ts is our Playwright test script as follows
import { test, expect, Page } from '@playwright/test';
test.beforeEach(async ({ page }) => {
await page.goto('https://www.saucedemo.com/');
});
test.describe('Demo Test', () => {
test('Verify Login Error Message', async ({ page }) => {
await page.waitForSelector('#user-name',{state:'visible'});
await page.locator('[data-test="username"]').type('example1@example.com');
await page.locator('[data-test="password"]').type('examplepassword');
await page.locator('[data-test="login-button"]').click();
const errorMessage = await page.locator('[data-test="error"]').textContent();
console.log("Login Error Message: "+errorMessage);
expect(errorMessage).toBe('Epic sadface: Username and password do not match any user in this service');
page.close();
});
});
Execute Playwright Test Script
Step 6: Click on Green Run icon next to the test to execute it
or execute the test script using the below command from the VS Code terminal
npx playwright test demo.spec.ts –headed
demo.spec.ts: test file; if you want to run all the spec files inside the tests folder, do not mention any spec file name
-headed: Playwright runs in headless mode by default, so we specify -headed to run headed mode.
You will see the below output.
The test script gets executed in 3 different browsers because of the configuration set up in the playwright.config.ts file. By default, it is configured to execute the test on all the browsers, but you can modify it accordingly.
Step 7: View the report
To view the generated HTML report, execute the below command
npx playwright show-report
Before configuring Azure DevOps pipeline, let us modify the package.json file to include the scripts which we will be using it in pipeline task to reference the scripts.
Integrating Playwright with Azure Pipelines:
Step 8: Login to Azure DevOps by navigating to https://dev.azure.com/ and create a new project and click create
Step 9: Once project is created, go to Repos and follow the second steps(highlighted in the snapshot below) to push an existing repository from command line. Run these commands in terminal of VS Code.
Refresh the browser, you will see the repository is updated with your local code.
Step 10: Navigate to pipelines and create your first pipeline and click on ‘use the classic editor’ to create a pipeline without YAML.
Step 11: Select your source ‘Azure Repos Git’ and select repository and click Continue
Step 12: Click on Empty Job
Step 13: Click on ‘+’ icon near to Agent job 1 and search for Node. Select Node.js tool installer and Click on Add
Step 14: Playwright supports from Node v14 and above. Hence, let’s configure the task to 16.x as given below.
Step 15: Add another task for Command line. Click on ‘+’ icon near to Agent job 1 and search for Command line. Select Command line and Click on Add.
Display name: Install playwright and Dependencies
script: npm install && npx playwright install
Click on Advanced –> Click on the little (i) icon and select the Link. This will enable the working directory for the task.
Step 16: Add another task for npm. Click on ‘+’ icon near to Agent job 1 and search for npm. Select npm and Click on Add.
Select Custom for Command. Add 'run tests' in Commands & arguments. [We are referencing the scripts mentioned in the package.json file ->"tests": "playwright test --config=playwright.config.ts --project=chromium"
Select the working directory as configured in previous step.
Step 17: Once all the configurations are done, now it’s time for us to Save & Queue the build
Conclusion:
Automating end-to-end testing is essential for delivering high-quality software. With Playwright's multi-browser support and Azure Pipelines' powerful CI/CD capabilities, you can easily automate your end-to-end testing process. The result is faster feedback, increased confidence in your codebase, and a smoother development workflow for your team. Embrace the power of Playwright and Azure Pipelines to take your testing and deployment process to the next level. Happy testing!