Testing on blog.iankulin.com

End to end testing - Cypress basics

Mon, 12 May 2025 00:00:00 +0000

When you’ve made a change to your web-app, do you run it then click around the new bits to check it works? Good start, but instead of doing that yourself, do it in a faster, more comprehensive and automated way with an end-to-end (E2E) testing setup using Cypress. Here’s how.

E2E

End to End testing is testing your app as a user might - by clicking links, entering data, looking at the screen and checking everything is okay, but it’s scripted like a unit test and the results are checked with assertions. Like unit testing this allows you to build up a collection of comprehensive tests that easily detect for unexpected behaviours - not just in the results of functions in your app, but in the user experience of the app.

In the case of Cypress, this works by running your app in an instrumented browser. The tests are written in JavaScript and might ask things like “Click the ‘Home’ link” and have an assertion similar to “check the home page loaded”. Let’s see how that will look.

How it looks

In the app I’m working on, if you view an individual customer (say at “http://127.0.0.1:3002/customers/1”) there’s a “Home” link at the top which takes you to the list of customers (at “http://127.0.0.1:3002/customers”).

Here’s the test code:

describe('Page Navigation', () => {
 it('should navigate to the customers list when clicking the Home link', () => {
 // visit the customer details page
 cy.visit('http://127.0.0.1:3002/customers/1');
 
 // find and click the Home link
 cy.get('a').contains('Home').click();
 
 // verify we navigated to the customers list page
 cy.url().should('eq', 'http://127.0.0.1:3002/customers');
 });
});

If you’ve been writing unit tests before, this format will be familiar, but let’s look at the steps:

cy.visit('http://127.0.0.1:3002/customers/1');

You guessed it - we’re telling Cypress to visit that page.

cy.get('a').contains('Home').click();

I’m not sure if Cypress uses JQuery, or just a JQuery like syntax, either way, what we’re doing here is selecting the ‘<a …>’ tag. Of course our page probably contains several anchor tags, so we’re refining this search to the anchor tag that contains ‘Home’. Note that there’s an implied assertion here. If there is no link on the page containing ‘Home’, this test will fail with an error saying something like “Expected to find content: ‘Home’ within the element: but never did.”

Finally the click() at the end of the statement tells Cypress to click this link.

cy.url().should('eq', 'http://127.0.0.1:3002/customers');

Before we look at this statement, consider that we haven’t told Cypress to wait for a bit for the results of our click() to process - one of the benefits of Cypress is it just figures that out magically.

This statement is an assertion - the URL should equal (eq) the URL we’ve provided.

So that gives us a quick overview of a simple test. Naturally Cypress has a heap more operators and assertion types to help us test our application - basically everything you could think of as user-facing testing. Let’s look at a simple demo app then work through the tests we might try for this.

The App

This app is a simple demo I wrote for an earlier blog post about using the Express router. We have Customers and Orders, a single customer can have zero-many orders. The opening page is a list of all customers. Clicking on a customer shows the details for that customer, including a list of their orders. Clicking on an order shows the detail for that order, including a link the customer it belongs to.

The Customer and Order detail views have delete links, and a deletion of a customer should cascade to delete that customer’s orders.

Installing Cypress

Installing Cypress is straightforward. The install steps from the docs are here, but really it’s just starting your Node project (so you’ve got a package.json) then npm install cypress --save-dev to add it as a dev dependency. It’s a big download so expect it to take a bit. It includes lodash, some AWS stuff, tldts, day.js, a heap of vue stuff - just, it’s a lot of big dependencies. Also since Cypress itself does some cool stuff linking into the browser - that functionality requires some code.

The Tests

Actually - the code in our very simple demo above covers about 70% of the testing I do, and the pattern of:

Select a user element
Click it or add some text
Select another user element to check that worked

comes up again and again. So I’m going to try not to repeat myself too much. Most of what’s new in the following tests will be extra selectors, and assertions. We won’t cover all of them, but rather a smattering to get started with.

 // test for customers list page
 describe("Customers Page", () => {
 it("should have the home page redirect to customers page", () => {
 cy.visit("http://localhost:3002");
 cy.url().should("include", "/customers");
 cy.get("h1").contains("Customers");
 });

 it("should display a list of customers", () => {
 cy.visit("http://localhost:3002/customers");
 cy.get("li").should("have.length.at.least", 5);
 cy.get("li").eq(0).contains("Alice");
 });

 it("should have working links to customer details", () => {
 cy.visit("http://localhost:3002/customers");
 // click the first customer (Alice)
 cy.get("a").contains("Alice Johnson").click();
 cy.url().should("include", "/customers/");
 cy.get("h2").contains("Alice Johnson");
 });
 });

.should()

There’s a massive list of should() assertions, and they depend a bit on what you’ve chained on to. In the first example we looked at we used "eq" for equals, in the example directly above we’ve used "include" for a partial match, and "have.length.at.least" for what it says on the box.

Another handy thing might be testing for "not.exist". In my example app if I want to test deleting a customer, I can check they exist in the customers list, click delete, then check that they no longer exist in the list:

 it("should delete a customer when delete link is clicked", () => {
 // first check the customer exists
 cy.visit("http://localhost:3002/customers");
 cy.get("a").contains("Hannah Abbott").should("exist");

 // visit the customer page and delete
 cy.visit("http://localhost:3002/customers/8");
 cy.get("a").contains("Delete customer").click();

 // verify the customer is deleted
 cy.url().should("include", "/customers");
 cy.get("a").contains("Hannah Abbott").should("not.exist");
 });

.get()

We’ve already seen selecting an anchor tag with get(“a”) - this will work for any HTML tag, but of course you’ll frequently need more specificity than that. As described in the docs, most of the JQuery selectors will also work with get.

// Select by element type
cy.get('button')

// Select by class
cy.get('.my-class')

// Select by ID
cy.get('#my-id')

// Combining selectors
cy.get('button.primary#submit')

// Select by attribute
cy.get('[data-test="submit-button"]')

Those first four are straightforward, but you might not know about attributes.

Attributes

As part of the HTML specification, tags can have attributes. You’ve been using them all along. For example. this button:

<button id="submit" class="btn primary" type="submit">Submit</button>

has attributes for:

id
class
type

These all have particular meanings for HTML, CSS and JavaScript, but actually we can make up our own. For example we could say:

<button type="submit" data-test="submit-button">Submit</button>

There’s no specification for ‘data-test’, it’s just a convention, we could just have easily said:

<button type="submit" data-green-zebra="submit-button">Submit</button>

Note that I’ve kept the data- prefix - that is part of the HTML5 specification. We could probably make up anything and it would work, but maybe it would conflict with something in a future HTML version, so best stick to “data-”.

Using attributes for specifying the element we want is highly recommended. Although the element you want to click might currently be the third in a

inside the

- it’s easy to imagine it being moved in a future update to your app. Once we’ve written an E2E test, we want it to continue to work across future app development, and using a ‘data-test’ attribute supports this.

.invoke() and then()

Sometimes the exact test you need might not be available, or you need to do some operation as part of your testing that requires a bit more processing. In that case, you can chain the invoke() method. This allows you to call any jQuery method on an element you’ve selected with Cypress, letting you extract specific properties or manipulate the element in ways that aren’t covered by Cypress’s built-in assertions.

Once you’ve got it, you can use then() to run an arrow function against it to do something. The pseudo codes looks a bit like this:

cy.get(selector)
 .invoke(jQueryMethod) // Extract what you need
 .then((result) => { // Process it with your own logic
 // Custom processing
 });

Let’s look at an example. Imagine the HTML of our page looks like this:

<span class="price">$24.99</span>

And we want to check that the price was greater than $20 - perhaps we are supposed to have added tax or something. Our test could look like this:

cy.get('.price')
 .invoke('text')
 .then((priceText) => {
 const priceValue = parseFloat(priceText.replace('$', ''));
 expect(priceValue).to.be.greaterThan(20.0);
 });

This selects this span based on it’s class, then saves the text ‘$24.99’ to priceText, extracts the value to a JavaScript number, and asserts it to be greater than 20.

In my demo app, I use this to check the cascading delete - when we delete the customer, the orders for that customer should also be deleted. Rather than hard code the order number we can use invoke/then to extract it from the text which looks like this:

8 - 2025-03-08 - $200

Then, after deleting the customer (and orders) we navigate to the orders page to make sure that order number does not exist there any more.

 describe("Cascading Deletions", () => {
 it("should delete owned orders when a customer is deleted", () => {
 // first make a note of an order for a specific customer
 cy.visit("http://localhost:3002/customers/4");
 cy.get("h2").contains("Diana Prince");

 // note an order ID that belongs to this customer
 cy.get("ul li a")
 .first()
 .invoke("text")
 .then((orderText) => {
 // extract the full order text to use for matching later
 const orderTextFull = orderText.trim();
 // extract just the order ID number
 const orderId = orderText.split(" ")[0].trim();

 // delete the customer
 cy.get("a").contains("Delete customer").click();

 // verify customer is deleted
 cy.url().should("include", "/customers");
 cy.get("a").contains("Diana Prince").should("not.exist");

 // check that the order is also deleted 
 cy.visit("http://localhost:3002/orders");
 // make sure we're matching the exact order (not just a substring)
 cy.get(`a[href="/orders/${orderId}"]`).should("not.exist");
 });
 });
 });

There is a lot, lot more to Cypress to this, but with what we’ve covered here it’s possible to write a comprehensive suite of tests that will test all of the functionality in this demo app in about 200 lines.

Using Cypress

So that’s all the code, but how does it look to test like this? For me this is one of the things that makes end-to-end testing cool. I love seeing it’s click away in the browser at super speed as my tests turn green.

First of all I start my app - so however you normally do this. For me, it’s dropping to the terminal in VSCode and running it in Node. Something like node index.js

Once that’s going we can start Cypress. Since the first terminal is running my node app, we need to spawn another terminal to run Cypress in. This is a simple matter in VSCode - just hit that + button I’ve circled in the screen shot below. You can swap between the different terminals you have open by clicking on them in the list underneath that + button.

We start Cypress in the new terminal with npx cypress open but the magic does not happen in the terminal, this thing pops up:

We’re doing E2E testing, so select that.

I’m in a Chrome mood today, so next we see this:

I’ve only got one test file - home.cy.js, so I click that. The tests are listed down the left side of the browser, and my app in an iFrame to the right. As the tests are running, I can see the app flicking through each step. In a couple of seconds the seventeen tests that comprise many page manipulations and assertions are finished and I can see the results.

If we click on a test, the details for it open up, and a screenshot of the application state at the time of that test is displayed.

The most common debugging problem I’ve run into is when I didn’t write the selection correctly (and didn’t use a data- attribute). These are easily checked in this view by hovering over the one we’re interested in - the element that Cypress used in this step will be highlighted.

So, how does a failed test look. I can create that in this test suit just by running the tests again. It won’t be able to delete orders or customers it deleted in the earlier run.

So, at the start of this ‘delete order’ test, I’m checking if the order exists, and it doesn’t (because we didn’t reset after deleting it last time). We can see from the error message that Cypress waited 4 seconds in case it was a timing issue. It’s displayed the test case where the failure has occurred. This along with the before and after snapshots of the app around each test make locating problems a breeze.

Resets

The pattern above (where you run a test twice and it fails the second time because the first execution changed the state) is common. To avoid this, we need some system of resetting the state. Cypress has a mocha like ‘beforeEach’ ability. You most always need this for logging things in:

describe('My app tests', () => {
 beforeEach(() => {
 // This code runs before each test in this block
 cy.visit('/login'); // for example
 cy.get('input[name=username]').type('user');
 cy.get('input[name=password]').type('password');
 cy.get('button[type=submit]').click();
 });

 it('should show the dashboard after login', () => {
 cy.url().should('include', '/dashboard');
 cy.contains('Welcome').should('be.visible');
 });

 it('should navigate to settings', () => {
 cy.get('nav').contains('Settings').click();
 cy.url().should('include', '/settings');
 });
});

But for apps that need things like a fresh database before testing, it’s a bit trickier. In the past I’ve sometimes created some sort of /test-reset endpoint which feels like an unreasonable security risk. The proper answer is to shell out with a Cypress task. That way we can do things like copy in test data, or spin up a whole test environment in a container. These are meaty topics for another post - but really, our tests should be re-run-able, and run-able in any order so we might come back to that.

Test

Any testing is better than none, and if you use these sorts of tools that make it easier you’ll find you’ll add to them, especially when errors crop up. If I sit down to add end-to-end tests to an existing app, I nearly always find things I want to change to make it better. Use end to end testing.

code

Node.js built in test runner

Mon, 17 Mar 2025 00:00:00 +0000

For the longest time, I’ve been using Mocha (test runner) and Chai (assertion library) for my JS testing. They are reliable old friends.

One of the effects of the existence of Bun and Deno has been to spur Node onto adding some new features, so after appearing as an experimental feature in 18, the Node test runner dropped in Node 20.

I’m not sure if the familiar unit test layout of Mocha and Node is inherited from Jest, or comes from older testing frameworks of which JUnit and NUnit were the first ones I’d ever used. Before that I just used to write tests as lumps of assertions in regular code - which worked but wasn’t as pleasant to use as a proper unit test setup. Regardless, the system of bundling a few tests together and having them all run and spit out green ticks is not a new one.

If you are coming from Mocha, there are very few changes to your practice to make. I didn’t read the docs to check that I had to begin my test files with ’test’ or to put them into a ’test’ directory. But I discovered that dragging them into a ’test-dont’ directory didn’t stop them from running.

Testing

As well as what ever you’re testing, you need to import a couple of things from node:

import { isEven } from "../index.js";
import { describe, it } from "node:test";
import assert from "node:assert";

Then we can write our tests, grouping them in a describe:

describe("isEven", () => {
 it("returns true for even numbers", (t) => {
 assert.strictEqual(isEven(4), true);
 assert.strictEqual(isEven(0), true);
 assert.strictEqual(isEven(-2), true);
 });

 it("returns false for odd numbers", (t) => {
 assert.strictEqual(isEven(7), false);
 assert.strictEqual(isEven(-3), false);
 });

 it("returns false for most floating point numbers", (t) => {
 assert.strictEqual(isEven(3.5), false);
 // sadly, this is true because JavaScript
 // assert.strictEqual(isEven(4.0000000000000000000001), false);
 });

 it("returns false for non-numbers", (t) => {
 assert.strictEqual(isEven("a"), false);
 assert.strictEqual(isEven(null), false);
 assert.strictEqual(isEven(undefined), false);
 });
});

Then to run them, just node --test and we’ll get a nice summary

Of course there are a heap of other assertions, as well as stuff to set-up and tear-down and to mock things such as times. What I’ve shown here is very much a getting started, but it also deals with about 80% of my testing needs.

I’m not entirely sure how I feel about moving to a built in test runner. The small convenience I gain has to be weighed up against a small amount of lock in to a run time. I haven’t yet been tempted to Deno or Bun, but I love that they exist and are spurring on innovation. Possibly I’ll continue with portable testing tools for big projects, but for little ones use the built in.

Testing Node.js apps - Mocha, Chai, and Supertest

Mon, 01 Jan 2024 00:00:00 +0000

Bruno is a great open source Postman/Insomnia replacement, and I’ve been using it for basic tests of my node servers using the built in asserts and loving it. This is pretty great, and I gather it’s also possible to go beyond this and write tests in JS in Bruno. I believe it also has the hooks needed to build it into your CI/CD systems.

Any large project is probably going to benefit from a more comprehensive suit of testing tools, and while I’ll still be using Bruno, my serious tests will be managed with these other tools.

I admit I’ve probably put this off a bit longer than I should have - I didn’t really want to install four dependencies and learn four different things just to test my endpoints. It turns out that using the tools together is seamless, and setting it all up was trivial.

Speaking of trivial, here’s my brilliant Node app. It has two endpoints, both of which do a bit of maths.

const express = require('express');const app = express();const port = 3000;app.use(express.json());// endpoint that takes two numbers and returns their sumapp.post('/sum', (req, res) => { const { a, b } = req.body; res.json({ sum: a + b });});// endpoint that takes two numbers and multiplies themapp.post('/multiply', (req, res) => { const { a, b } = req.body; res.json({ product: a * b });});// start the serverapp.listen(port, () => { console.log(`Maths server is running at http://localhost:${port}`);});

Setting up your project for testing

Install the tools

npm install --save-dev mocha chai supertest

The --save-dev bit installs them as a development dependencies - they will go in your package.json and everyone who clones the repo will be working with the same version. Additionally, they won’t needlessly be installed when deployed to production.

Export the app

The testing system needs to be able to control the app a little bit - start it, stop it, and hook into it. To do that, we’ll complicate our app.listen code a bit so that we’ve also got a server variable, then we’ll export the app and server so out test files can import them. It will end up looking something like this:

let server;if (process.env.NODE_ENV !== 'test') { server = app.listen(3000, () => console.log(`Maths server is running at http://localhost:${port}`));}module.exports = { app, server };

This stops the server being started here if we’re in test mode, but exports the bits the test framework needs to manage things.

Create the test files

Our JS test code is all going in a test/ directory in our project, and they will all be named <something>.test.js I usually use the file name of the file I’m testing. So today I’m writing tests for app.js my tests will be in apps.test.js

Each test file will need to pull in our tools (supertest and chai) and the server and app variables.

That’s followed by one or more test suites; each test suite contains one or more test cases. This might be easier to explain if we look at a real file:

const supertest = require('supertest');const chai = require('chai');const { app, server } = require('../app');const expect = chai.expect;describe('POST / add', () => { it('should return the correct sum', () => { return supertest(app) .post('/sum') .send({ a: 5, b: 5 }) .expect(200) .then(res => { expect(res.body.sum).to.equal(10); }); }); it('should return the correct sum with negative numbers', () => { return supertest(app) .post('/sum') .send({ a: -5, b: -5 }) .expect(200) .then(res => { expect(res.body.sum).to.equal(-10); }); });});describe('POST / multiply', () => { it('should return the correct product', () => { return supertest(app) .post('/multiply') .send({ a: 5, b: 5 }) .expect(200) .then(res => { expect(res.body.product).to.equal(25); }); });});server.close();

This file contains two test suites - ‘POST/add’ and ‘POST/multiply’. POST/add contains two test cases (each begins with it<statement of what the test subject should do>).

There’s no end to the tests you can write. I normally do basic functionality as I’m writing code, and I also add in tests for anything that emerges as a bug. If you get into the rhythm of bug -> write failing test -> fix bug -> test passes you can have your day punctuated by little doses of dopamine. I often write a timed test - that an endpoint should respond in 10ms. These don’t help you when you are developing, but sure will later. You should also check that all of the wrong inputs users will eventually try have been handled. If an API expects a number, check for errors being thrown for strings, for negative numbers, for huge numbers, for decimals, for booleans, for objects etc etc.

Another thing I will do is use a code coverage tool to check my test covers all the branches and error conditions. I plan to talk about that another day. First I need to show you how to run the tests.

Add test script

If we had installed mocha globally, we could just call it from the command line with something like:

mocha ./test/app.test.js

But we didn’t do that, so we need npm to start it up for us. I know this seems like another time wasting step, but it’s one of those do it once, benefit from it thousands of times things.

In the package.json file, we can add a section called scripts. If you started you project with npm init you may already have this section, if not, just add it in. It’s common to have a run and a test script, and I often have one or two others. Here’s the sort of thing you want.

{ "name": "test-demo", "version": "0.1.0", "description": "Simple Maths API", "main": "app.js", "scripts": { "start": "node app.js", "test": "mocha './test/*.test.js'" }, "dependencies": { "express": "^4.18.2" }, "devDependencies": { "chai": "^4.3.10", "mocha": "^10.2.0", "nyc": "^15.1.0", "supertest": "^6.3.3" }}

npm does the magic to make the correct version of the library available when this script is run. The end effect of these is that you can type npm test at the command line, and mocha will run your tests. Let’s try it would our tests.

That’s what we like to see, passing tests. I’ll make one fail by telling it to expect 5x5=26.

And that’s it, you’re all set up to write tests against your node apps.

What do the different bits do?

There’s a lot of moving parts here, lets tease those out a little.

mocha - this is the test framework. As we’ve discussed, it’s the command line tool that runs the tests and produces the output.

supertest - manages the connections between the test runner/framework and the code being tested. When I’m pressing a button in Bruno, it’s actually hitting localhost:3000 to exercise the server which I’ve previously started. supertest is doing magic to make that connection without going through the network layers.

chai - it provides the assert()s, expect()s and should()s that we use in the test cases. You could, in theory make do with the assert() library built into node - especially for our toy demo app - but it’s no where near as nice, and in particular chai has a massive set of plugins that both extend it’s use generally, but also into working at a detailed level with other vendor packages.

Bruno asserts

Sat, 11 Nov 2023 00:00:00 +0000

I mentioned Bruno the other day. Although it’s still very much under development, it is shaping up as a great Postman/Insomnia replacement.

One of the aspects I’ve been using today is asserts. As part of a request, you can add some asserts - so when you’re hitting an endpoint it will check what status should it be returning, or given the data you’re passing in, what should be in the response body.

When I’d asked ChatGPT to to review the mdserver code, it had suggested that I should be sanitising URL inputs better to prevent users transversing out of the ‘public’ file directory to other places in the file system. I thought Express had already taken care of this for me, but wanted to check. I had ChatGPT generate a bunch of pass and fail URL examples, then just created asserts for each one in Bruno.

Once that’s done, you can just right click on the collection and have it run all of those.

An extra benefit of Bruno is that all these requests are stored as JSON-like, version-controllable text. I store them in my project and commit them along with the rest of my code.

Testing, testing

Sun, 09 Oct 2022 00:00:00 +0000

I have unit testing in my goals, and if I’m going to throw this space trimming macOS utility up on the web, now might be a good time to figure out how to add unit tests to a project, how to write them, and how to run them. XCode is well set up for this, so it’s really no drama to do.

Although I haven’t worried with any unit testing up to this point in my iOS/Swift learning, in my previous programming work I did a lot of work with the large calculations involved in translating GPS coordinates and robotic positioning models where errors would be bad - so I’ve written a lot of tests over the years. I’ve also definitely felt the confidence you can dramatically refactor code with when you know the code has a robust test suite. I’m a big fan.

Adding Tests to Existing Project

In Xcode, with your project open, you need to add the testing target. File | New | Target... then scroll down to find the Unit Testing Bundle. When you add that, you’ll see a new folder in the Project Navigator, and a new source file - both with the name <app name>Tests.

If you click into the <app name>Test.swift file, you’ll see a couple of methods for setting up the testing environment and cleaning up after it. They are called before and after each test - you’d use them if you needed to set something up like some files in a directory or similar and wanted to ensure it was not affected by the tests run before the current test. Don’t worry about them for the time being.

Writing Tests

Before we can write any tests, we need to import any modules with the code we want to test. In my case, I want to test the function stripSpaces() which is in the ContentView of my app called CodeTrimmer. So in the top of the CodeTrimmerTests.swift file, under the other import, I add the import:

import XCTest
@testable import CodeTrimmer

Then we can write our tests. The test functions must start with the magic prefix test so that Xcode treats them as such. The usual approach in the test is to call a function with some specific input, then test the output is what is expected with a special test version of assert():

Here’s a couple. My function stripSpaces() deletes a number of spaces from each line of a block of code such that the block becomes left aligned. if the XCTAsserts() pass, the test passes and I get a green tick. To run the tests I just click in the breakpoint gutter next to the function declaration.

 func testStripSpaces01() throws {
 let testString =
 """
 single line string no spaces
 """
 let resultString = stripSpaces(testString)
 XCTAssertTrue(resultString == testString, testString)
 }

 func testStripSpaces02() throws {
 let testString =
 """
 single line 4 spaces
 """
 let resultString = stripSpaces(testString)
 XCTAssertFalse(resultString == testString, testString)
 XCTAssertTrue(testString.dropFirst(4) == resultString, testString)
 }