Published on

Cypress vs Playwright – Clash of the Titans: Writing first tests (3)

Authors

In this series of articles, I am going to dive deep into the capabilities of two popular e2e testing frameworks: Cypress and Playwright. The comparison will cover technical specifications, documentation, setup, features, CI/CD integration, etc. The goal is to uncover the main differences and pros/cons of each framework and potentially help anyone who wants to make a decision between these e2e testing frameworks.

Application under test

The application that I’m going to use in this article is a Real World App (https://github.com/cypress-io/cypress-realworld-app). This is the simple payment application created by the Cypress team for demonstration and practice purposes. The frontend is built with React, while the backend is built with Express with a lowdb database which enables convenient data manipulation. The application is very useful for testing/POC purposes since it provides frontend, API and data that is fully under my control.

To get more details about the app and the configuration of the Real World App, feel free to visit the GitHub readme. Otherwise, you get the application up and running in three simple steps:

  1. Clone the repository: git clone git@github.com:cypress-io/cypress-realworld-app.git
  2. Install dependencies: yarn install
  3. Run the app: yarn dev

By default, the app (frontend) will run on port 3000 and the API/backend will run on port 3001.

Code repository

Tests from this article can be found in the following repository: https://github.com/azeljkovic/cypress-vs-playwright Here, you can also find dependencies, test data and selectors for given tests.

Cypress tests

Let’s start with the most straightforward login test, and then expand it further:

describe('simple test', () => {
  it('login via UI', () => {
    cy.visit('http://localhost:3000/signin')
    cy.get('[data-test="signin-username"]').type('Katharina_Bernier')
    cy.get('[data-test="signin-password"]').type('s3cret')
    cy.get('[data-test="signin-submit"]').click()
  })
})

At first look, Mocha BDD syntax can be recognized. Cypress bundles several libraries that improve the development experience, and two of them are Mocha and Chai. This gives a familiar syntax and a variety of available assertions. Cypress commands are issued inside the it() block, as a member of a global cy object. Commands are usually understandable, and self-descriptive:

  • visit() – visit URL
  • get() – find the DOM element to interact with
  • type() – type some text
  • click() – click on the element

Now let’s refactor this test a little and add some assertions:

import loginSelectors from '../fixtures/selectors/login.json'
import mainSelectors from '../fixtures/selectors/main.json'
describe('simple test', () => {
  beforeEach(() => {
    cy.visit('/signin')
  })
  it('login via UI', () => {
    cy.get(loginSelectors.username).type(Cypress.env('username'))
    cy.get(loginSelectors.password).type(Cypress.env('password'), { log: false })
    cy.get(loginSelectors.submit).click()
    // assert
    cy.url().should('equal', 'http://localhost:3000/')
    cy.get(mainSelectors.usernameLabel).should('have.text', `@${Cypress.env('username')}`)
  })
})

Let’s see what was improved here:

  • selectors are moved to dedicated JSON files and imported into the test
  • visit command is moved to a beforeEach hook (Mocha syntax here as well)
  • URL is removed from the visit command since it was moved to cypress.config.js. Cypress does not require the whole URL in the visit command, you can set the baseURL in cypress.config.js and only put the path in the test
  • username and password are moved to cypress.env.json and they are accessed as Cypress environment variables
  • for complete data protection, password logging is turned off, so it doesn’t appear in reports/screenshots/videos
  • once we are logged in, the test is checking the username label in the main window. This way, the test verifies that the main window is loaded and the particular user is logged-in.

Playwright tests

Again, let’s start with the most straightforward login test:

import { test } from '@playwright/test'
test('login via UI', async ({ page }) => {
  await page.goto('http://localhost:3000/signin')
  await page.fill('#username', 'Katharina_Bernier')
  await page.fill('#password', 's3cret')
  await page.click('[data-test="signin-submit"]')
  // wait for 1 second to see what's happening in headed mode
  await page.waitForTimeout(1000)
})

In Playwright, tests are written as async arrow functions that take a page parameter. The page is an instance of a Page class that contains events and methods for interaction with a page, which, in this case, presents a single browser tab. The basic commands that we are using in this example are:

  • goto() – open the URL page
  • fill() – fills the text field
  • click() – clicks on the element
  • waitForTimeout() – wait for a given number of milliseconds before proceeding

After some improvements, the code looks like this:

import { test, expect } from '@playwright/test'
import loginSelectors from '../fixtures/selectors/login.json'
import mainSelectors from '../fixtures/selectors/main.json'
test.beforeEach(async ({ page }) => {
  await page.goto('/signin')
})
test('login via UI', async ({ page }) => {
  await page.fill(loginSelectors.username, process.env.USERNAME) // env file is defined in playwright.config.js
  await page.fill(loginSelectors.password, process.env.PASSWORD)
  await page.click(loginSelectors.submit)
  // assert
  await expect(page).toHaveURL('http://localhost:3000/')
  await expect(page.locator(mainSelectors.usernameLabel)).toHaveText(`@${process.env.USERNAME}`)
  // wait for 1 second to see what's happening in headed mode
  await page.waitForTimeout(1000)
})

Let’s see what was changed:

  • selectors are moved to dedicated JSON files and imported into the test
  • visit command is moved to a beforeEach hook
  • URL is removed from the visit command since it was moved to playwright.config.js. Playwright does not require the whole URL in the visit command, you can set the baseURL in playwright.config.js and only put the path in the test
  • username and password are moved to a dedicated .env file, and are being accessed via dotenv library
  • once we are logged in, the test is checking the username label in the main window. This way, the test verifies that the main window is loaded and the particular user is logged-in.