Webdev on blog.iankulin.com

End to end testing - Cypress basics

Mon, 12 May 2025 00:00:00 +0000

When you’ve made a change to your web-app, do you run it then click around the new bits to check it works? Good start, but instead of doing that yourself, do it in a faster, more comprehensive and automated way with an end-to-end (E2E) testing setup using Cypress. Here’s how.

E2E

End to End testing is testing your app as a user might - by clicking links, entering data, looking at the screen and checking everything is okay, but it’s scripted like a unit test and the results are checked with assertions. Like unit testing this allows you to build up a collection of comprehensive tests that easily detect for unexpected behaviours - not just in the results of functions in your app, but in the user experience of the app.

In the case of Cypress, this works by running your app in an instrumented browser. The tests are written in JavaScript and might ask things like “Click the ‘Home’ link” and have an assertion similar to “check the home page loaded”. Let’s see how that will look.

How it looks

In the app I’m working on, if you view an individual customer (say at “http://127.0.0.1:3002/customers/1”) there’s a “Home” link at the top which takes you to the list of customers (at “http://127.0.0.1:3002/customers”).

Here’s the test code:

describe('Page Navigation', () => {
 it('should navigate to the customers list when clicking the Home link', () => {
 // visit the customer details page
 cy.visit('http://127.0.0.1:3002/customers/1');
 
 // find and click the Home link
 cy.get('a').contains('Home').click();
 
 // verify we navigated to the customers list page
 cy.url().should('eq', 'http://127.0.0.1:3002/customers');
 });
});

If you’ve been writing unit tests before, this format will be familiar, but let’s look at the steps:

cy.visit('http://127.0.0.1:3002/customers/1');

You guessed it - we’re telling Cypress to visit that page.

cy.get('a').contains('Home').click();

I’m not sure if Cypress uses JQuery, or just a JQuery like syntax, either way, what we’re doing here is selecting the ‘<a …>’ tag. Of course our page probably contains several anchor tags, so we’re refining this search to the anchor tag that contains ‘Home’. Note that there’s an implied assertion here. If there is no link on the page containing ‘Home’, this test will fail with an error saying something like “Expected to find content: ‘Home’ within the element: but never did.”

Finally the click() at the end of the statement tells Cypress to click this link.

cy.url().should('eq', 'http://127.0.0.1:3002/customers');

Before we look at this statement, consider that we haven’t told Cypress to wait for a bit for the results of our click() to process - one of the benefits of Cypress is it just figures that out magically.

This statement is an assertion - the URL should equal (eq) the URL we’ve provided.

So that gives us a quick overview of a simple test. Naturally Cypress has a heap more operators and assertion types to help us test our application - basically everything you could think of as user-facing testing. Let’s look at a simple demo app then work through the tests we might try for this.

The App

This app is a simple demo I wrote for an earlier blog post about using the Express router. We have Customers and Orders, a single customer can have zero-many orders. The opening page is a list of all customers. Clicking on a customer shows the details for that customer, including a list of their orders. Clicking on an order shows the detail for that order, including a link the customer it belongs to.

The Customer and Order detail views have delete links, and a deletion of a customer should cascade to delete that customer’s orders.

Installing Cypress

Installing Cypress is straightforward. The install steps from the docs are here, but really it’s just starting your Node project (so you’ve got a package.json) then npm install cypress --save-dev to add it as a dev dependency. It’s a big download so expect it to take a bit. It includes lodash, some AWS stuff, tldts, day.js, a heap of vue stuff - just, it’s a lot of big dependencies. Also since Cypress itself does some cool stuff linking into the browser - that functionality requires some code.

The Tests

Actually - the code in our very simple demo above covers about 70% of the testing I do, and the pattern of:

Select a user element
Click it or add some text
Select another user element to check that worked

comes up again and again. So I’m going to try not to repeat myself too much. Most of what’s new in the following tests will be extra selectors, and assertions. We won’t cover all of them, but rather a smattering to get started with.

 // test for customers list page
 describe("Customers Page", () => {
 it("should have the home page redirect to customers page", () => {
 cy.visit("http://localhost:3002");
 cy.url().should("include", "/customers");
 cy.get("h1").contains("Customers");
 });

 it("should display a list of customers", () => {
 cy.visit("http://localhost:3002/customers");
 cy.get("li").should("have.length.at.least", 5);
 cy.get("li").eq(0).contains("Alice");
 });

 it("should have working links to customer details", () => {
 cy.visit("http://localhost:3002/customers");
 // click the first customer (Alice)
 cy.get("a").contains("Alice Johnson").click();
 cy.url().should("include", "/customers/");
 cy.get("h2").contains("Alice Johnson");
 });
 });

.should()

There’s a massive list of should() assertions, and they depend a bit on what you’ve chained on to. In the first example we looked at we used "eq" for equals, in the example directly above we’ve used "include" for a partial match, and "have.length.at.least" for what it says on the box.

Another handy thing might be testing for "not.exist". In my example app if I want to test deleting a customer, I can check they exist in the customers list, click delete, then check that they no longer exist in the list:

 it("should delete a customer when delete link is clicked", () => {
 // first check the customer exists
 cy.visit("http://localhost:3002/customers");
 cy.get("a").contains("Hannah Abbott").should("exist");

 // visit the customer page and delete
 cy.visit("http://localhost:3002/customers/8");
 cy.get("a").contains("Delete customer").click();

 // verify the customer is deleted
 cy.url().should("include", "/customers");
 cy.get("a").contains("Hannah Abbott").should("not.exist");
 });

.get()

We’ve already seen selecting an anchor tag with get(“a”) - this will work for any HTML tag, but of course you’ll frequently need more specificity than that. As described in the docs, most of the JQuery selectors will also work with get.

// Select by element type
cy.get('button')

// Select by class
cy.get('.my-class')

// Select by ID
cy.get('#my-id')

// Combining selectors
cy.get('button.primary#submit')

// Select by attribute
cy.get('[data-test="submit-button"]')

Those first four are straightforward, but you might not know about attributes.

Attributes

As part of the HTML specification, tags can have attributes. You’ve been using them all along. For example. this button:

<button id="submit" class="btn primary" type="submit">Submit</button>

has attributes for:

id
class
type

These all have particular meanings for HTML, CSS and JavaScript, but actually we can make up our own. For example we could say:

<button type="submit" data-test="submit-button">Submit</button>

There’s no specification for ‘data-test’, it’s just a convention, we could just have easily said:

<button type="submit" data-green-zebra="submit-button">Submit</button>

Note that I’ve kept the data- prefix - that is part of the HTML5 specification. We could probably make up anything and it would work, but maybe it would conflict with something in a future HTML version, so best stick to “data-”.

Using attributes for specifying the element we want is highly recommended. Although the element you want to click might currently be the third in a

inside the

- it’s easy to imagine it being moved in a future update to your app. Once we’ve written an E2E test, we want it to continue to work across future app development, and using a ‘data-test’ attribute supports this.

.invoke() and then()

Sometimes the exact test you need might not be available, or you need to do some operation as part of your testing that requires a bit more processing. In that case, you can chain the invoke() method. This allows you to call any jQuery method on an element you’ve selected with Cypress, letting you extract specific properties or manipulate the element in ways that aren’t covered by Cypress’s built-in assertions.

Once you’ve got it, you can use then() to run an arrow function against it to do something. The pseudo codes looks a bit like this:

cy.get(selector)
 .invoke(jQueryMethod) // Extract what you need
 .then((result) => { // Process it with your own logic
 // Custom processing
 });

Let’s look at an example. Imagine the HTML of our page looks like this:

<span class="price">$24.99</span>

And we want to check that the price was greater than $20 - perhaps we are supposed to have added tax or something. Our test could look like this:

cy.get('.price')
 .invoke('text')
 .then((priceText) => {
 const priceValue = parseFloat(priceText.replace('$', ''));
 expect(priceValue).to.be.greaterThan(20.0);
 });

This selects this span based on it’s class, then saves the text ‘$24.99’ to priceText, extracts the value to a JavaScript number, and asserts it to be greater than 20.

In my demo app, I use this to check the cascading delete - when we delete the customer, the orders for that customer should also be deleted. Rather than hard code the order number we can use invoke/then to extract it from the text which looks like this:

8 - 2025-03-08 - $200

Then, after deleting the customer (and orders) we navigate to the orders page to make sure that order number does not exist there any more.

 describe("Cascading Deletions", () => {
 it("should delete owned orders when a customer is deleted", () => {
 // first make a note of an order for a specific customer
 cy.visit("http://localhost:3002/customers/4");
 cy.get("h2").contains("Diana Prince");

 // note an order ID that belongs to this customer
 cy.get("ul li a")
 .first()
 .invoke("text")
 .then((orderText) => {
 // extract the full order text to use for matching later
 const orderTextFull = orderText.trim();
 // extract just the order ID number
 const orderId = orderText.split(" ")[0].trim();

 // delete the customer
 cy.get("a").contains("Delete customer").click();

 // verify customer is deleted
 cy.url().should("include", "/customers");
 cy.get("a").contains("Diana Prince").should("not.exist");

 // check that the order is also deleted 
 cy.visit("http://localhost:3002/orders");
 // make sure we're matching the exact order (not just a substring)
 cy.get(`a[href="/orders/${orderId}"]`).should("not.exist");
 });
 });
 });

There is a lot, lot more to Cypress to this, but with what we’ve covered here it’s possible to write a comprehensive suite of tests that will test all of the functionality in this demo app in about 200 lines.

Using Cypress

So that’s all the code, but how does it look to test like this? For me this is one of the things that makes end-to-end testing cool. I love seeing it’s click away in the browser at super speed as my tests turn green.

First of all I start my app - so however you normally do this. For me, it’s dropping to the terminal in VSCode and running it in Node. Something like node index.js

Once that’s going we can start Cypress. Since the first terminal is running my node app, we need to spawn another terminal to run Cypress in. This is a simple matter in VSCode - just hit that + button I’ve circled in the screen shot below. You can swap between the different terminals you have open by clicking on them in the list underneath that + button.

We start Cypress in the new terminal with npx cypress open but the magic does not happen in the terminal, this thing pops up:

We’re doing E2E testing, so select that.

I’m in a Chrome mood today, so next we see this:

I’ve only got one test file - home.cy.js, so I click that. The tests are listed down the left side of the browser, and my app in an iFrame to the right. As the tests are running, I can see the app flicking through each step. In a couple of seconds the seventeen tests that comprise many page manipulations and assertions are finished and I can see the results.

If we click on a test, the details for it open up, and a screenshot of the application state at the time of that test is displayed.

The most common debugging problem I’ve run into is when I didn’t write the selection correctly (and didn’t use a data- attribute). These are easily checked in this view by hovering over the one we’re interested in - the element that Cypress used in this step will be highlighted.

So, how does a failed test look. I can create that in this test suit just by running the tests again. It won’t be able to delete orders or customers it deleted in the earlier run.

So, at the start of this ‘delete order’ test, I’m checking if the order exists, and it doesn’t (because we didn’t reset after deleting it last time). We can see from the error message that Cypress waited 4 seconds in case it was a timing issue. It’s displayed the test case where the failure has occurred. This along with the before and after snapshots of the app around each test make locating problems a breeze.

Resets

The pattern above (where you run a test twice and it fails the second time because the first execution changed the state) is common. To avoid this, we need some system of resetting the state. Cypress has a mocha like ‘beforeEach’ ability. You most always need this for logging things in:

describe('My app tests', () => {
 beforeEach(() => {
 // This code runs before each test in this block
 cy.visit('/login'); // for example
 cy.get('input[name=username]').type('user');
 cy.get('input[name=password]').type('password');
 cy.get('button[type=submit]').click();
 });

 it('should show the dashboard after login', () => {
 cy.url().should('include', '/dashboard');
 cy.contains('Welcome').should('be.visible');
 });

 it('should navigate to settings', () => {
 cy.get('nav').contains('Settings').click();
 cy.url().should('include', '/settings');
 });
});

But for apps that need things like a fresh database before testing, it’s a bit trickier. In the past I’ve sometimes created some sort of /test-reset endpoint which feels like an unreasonable security risk. The proper answer is to shell out with a Cypress task. That way we can do things like copy in test data, or spin up a whole test environment in a container. These are meaty topics for another post - but really, our tests should be re-run-able, and run-able in any order so we might come back to that.

Test

Any testing is better than none, and if you use these sorts of tools that make it easier you’ll find you’ll add to them, especially when errors crop up. If I sit down to add end-to-end tests to an existing app, I nearly always find things I want to change to make it better. Use end to end testing.

code

User Sessions & Cookies in Node

Fri, 09 Feb 2024 00:00:00 +0000

When you are learning app development, you can create all sorts of apps that work for you, but for any serious app, it’s going to need to authenticate users and persist sessions across visits. So much so, that as a professional developer, you’ll probably build that out first - it becomes a sort of boiler plate you always drop in.

In this post, focusing on the server side, using node, express, and particularly express-session, I’ll try and build up from nothing to a reasonable usable user login system explaining the increasing complexity and reasons for it. To follow along you’ll need basic familiarity with node and express.

The problem we’re addressing

For most web applications, we need to persist state per user. For example, if you go to a drawing app and start a drawing, you want it to be there when you come back to the app. Additionally, you don’t want to come back to someone else’s half-drawn app, or, have them drawing over your picture. What we really want is something like this:

User 1 sees their picture of the star, and User 2 sees their picture of a heart.

Since HTTP is stateless - a request to /picture from one user is indistinguishable from another users request to /picture - so we need to add something to allow the server to distinguish between the two. The something would be a bit of state that the server passes back to the user, then the user sends it in with their actions so the server can identify them.

There are a few of ways to do this. The first (which is not the subject of this post) is to store that in the URL. For example, when a user in the above app requests to create a picture, we could generate a GUID for them, then redirect them to a URL based on that - perhaps /picture/cbe34f. Thereafter, all their requests could include that GUID. This can be a useful way of managing session state and has some affordances that the other method does not, but it’s not the most common.

Another system that was extensively used in the early days of the web was to embed a hidden input with the GUID in the HTML returned to the user. When the user submitted the form later, the GUID was available

The most common method is for the server to create a bit of state (our GUID), send it to the user and have the user’s browser store it, and return it with every request. You will know this bit of state as a cookie.

Code example

Enough theory, lets look at some code. If you google ‘simple node express session’ you’ll find this, or something almost identical. Instead of the state we’re trying to persist being a picture of a heart or a star, we’re trying to remember how many times each user has visited our web site. Note these are separate counts for each user - a new visitor will start at zero, then count up each time they come back or refresh their page.

We’re using a couple of packages, so to run this example, you’ll first need to install them with npm i express express-session

const express = require('express');
const session = require('express-session');

const app = express();

app.use(session({
 secret: 'your-secret-key', 
 resave: false,
 saveUninitialized: true
}));

app.get('/', (req, res) => {
 // Access session data
 if (req.session.views) {
 req.session.views++;
 } else {
 req.session.views = 1;
 }

 res.send(`Views: ${req.session.views}`);
});

app.listen(3000, () => {
 console.log('Server is running on port 3000');
});

If we load this page from http://localhost:3000 it says “Views: 1”. Reloading the page it will say “Views: 2”. If I open a different browser and visit the server, we’ll be back to “Views: 1”. So what’s the magic that’s happening here?

On the first request from a browser it hasn’t seen before, express-session is generating a session ID, and sending that back to the browser (along with ‘Views: 1’) and asking the browser to store it as a cookie. Thereafter, every request to this website http://localhost:3000 will include the cookie containing the session ID. The number of views is being stored on the server in memory. Express-session does the work of looking up the number of views for a particular session ID returned in a cookie so we have the luxury of just grabbing that from rec.session.views

Thanks to the magic of browser development tools, we can have a look in the request header from the browser and see the cookie with it’s session ID contents:

It’s also possible to go and find the cookie your browser has stored. Again, this is easiest in the developer tools - look under ‘Storage’:

Currently, we’re only storing the links between session_ids and the number of views in memory in the server, so if the server restarts, we’ll lose them, and everyone will go back to ‘Views: 1’.

Persisting across server restarts

If we want our view counts to not be reset to zero each time the server restarts, we’ll need to save them somehow. express-session doesn’t let us access its internal array of sessions, but it does provide a mechanism for that access with the concept of stores. Usually we’d keep the sessions in a database, but as files is a simpler solution for a blog post. There is a package called session-file-store that will slot into expression-session that will just use the file system to persist the session-view count key pairs for us. After installing it with npm i session-file-store, we just declare a const for it, and pass it in when initialising the session middleware.

const express = require("express");
const session = require("express-session");
const FileStore = require("session-file-store")(session);

const app = express();

app.use(
 session({
 secret: "your-secret-key", // This should be a secret, used to sign the session ID cookie
 resave: false,
 saveUninitialized: true,
 store: new FileStore,
 name: "session_cookie",
 })
);

app.get("/", (req, res) => {
 // Access session data
 if (req.session.views) {
 req.session.views++;
 } else {
 req.session.views = 1;
 }

 res.send(`Views: ${req.session.views}`);
});

app.listen(3000, () => {
 console.log("Server is running on port 3000");
});

Now view counts for each browser are persisted even when the server is re-started. If we look inside one of the files we should see a view_count field:

The name of each file is the session id, but these don’t match the session id’s you see in the browser cookies - presumably because they’ve been encrypted by the secret we used when establishing the express-session.

So this is a better experience - our users see their own view counts increasing on each refresh, and the view counts don’t get lost when you take the server down for maintenance. But what happens if the user wants to pick up their phone and check their view count. It’s a different browser without the cookie containing the session id, so they’ll get ‘View: 1’ again.

Or, what if your partner sits down at your laptop to check their view count, expecting it to be ‘1’ because they’ve never visited this page, and they see your view count instead?

Users

Our current system is really based around browser instances, but we want it to be about people. So a better system, and one that you’ll be used to is the concept of logging in as a user. This can solve both the problems we described above - users will be able to log in from any browser and see only their own data.

This is going to take things up a substantial step in complexity because now instead of two bits of data (the cookie with the session_id, and the session store linking the session_id and the view count) there is going to be:

The cookie with the session_id
The session store linking the session_id and the user
A new store we will have to manage that links the user and the view_count

Also, we’ll need:

Some mechanism to create a user
Some mechanism to log in
And log out
If there’s no user logged in, redirect to the log in page

Log out

Logging the user out is reasonably simple - we just delete the session information. In a real app we’d have a ’log out’ button of some sort, but for simplicity here, I’m just going to add a ’log out’ route.

app.get("/logout", (req, res) => { req.session.destroy((err) => { if (err) { return console.log(err); } res.clearCookie("session_cookie"); res.redirect("/"); });});

clearCookie() tells the browser to delete the cookie - which just saves us from console messages later when the browser sends it, but express-session can’t find the matching session.

If you add this to the code from earlier and run it, view will count up as before, but when you visit the /logout endpoint views will be set back to 1.

Users & view counts

In a real application we’d be keeping this in a database, but again for simplicity of this demo, I’m just going to keep an array of objects and persist them as a text file. The array will be user_views, and somewhere at the top of our code we’ll add:

let user_views = [];// if the user_views.json file exists, read it and parse it to user_views arrayif (fs.existsSync("./user_views.json")) { user_views = JSON.parse(fs.readFileSync("./user_views.json"));}

Creating a user

We’ll use a route parameter to create a new user (like /create/jane or /create/robert where the second part is accessible in req.params). That user will be set as the session user, then we’ll add it to our array and save the array to disk.

app.get("/create/:user", (req, res) => { const user = req.params.user; const views = 0; req.session.user = user; user_views.push({ user, views }); fs.writeFile("./user_views.json", JSON.stringify(user_views), (err) => { if (err) { res.send(err); return; } }); res.send(`User ${user} created & logged in`);});

Logging a user in

If the user is in our array, we’ll set the session user to it, otherwise redirect to the create endpoint:

app.get("/login/:user", (req, res) => { // see if this user is in the user_views array if (user_views.find((u) => u.user === req.params.user)) { req.session.user = req.params.user; res.send(`User ${req.params.user} logged in`); return; } else { res.redirect(`/create/${req.params.user}`); }});

Showing the view count

The default route that shows our view count has a few jobs to do:

Check we’re logged in, if not, tell the user to log in
If we are logged in, fetch the view count for that user
Increment the view count
Show it to the user
Re-save the user_views data since we’ve mutated it

app.get("/", (req, res) => { if (req.session.user) { const user_view = user_views.find((u) => u.user === req.session.user); user_view.views++; res.send(`User "${req.session.user}" has ${user_view.views} views`); writeUserViewFile(); } else { res.send("Please log in"); }});

So that’s our system for associating the view counts with a user done. Here’s the whole thing where we’re up to (I’ve done a little refactoring).

const express = require("express");
const session = require("express-session");
const FileStore = require("session-file-store")(session);
const fs = require("fs");
const app = express();

const user_view_file = "./user_views.json";
const cookie_name = "session_cookie";

let user_views = [];
// if the user_views.json file exists, read it and parse it to user_views array
if (fs.existsSync(user_view_file)) {
 user_views = JSON.parse(fs.readFileSync(user_view_file));
}

function writeUserViewFile() {
 fs.writeFile(user_view_file, JSON.stringify(user_views), (err) => {
 if (err) {
 console.log(err);
 return;
 }
 });
}

function findUser(user) {
 return user_views.find((u) => u.user === user);
}

app.use(
 session({
 secret: "your-secret-keyz", // This should be a secret, used to sign the session ID cookie
 resave: false,
 saveUninitialized: true,
 store: new FileStore(),
 name: cookie_name,
 })
);

app.get("/", (req, res) => {
 if (req.session.user) {
 const user_view = findUser(req.session.user);
 user_view.views++;
 res.send(`User "${req.session.user}" has ${user_view.views} views`);
 writeUserViewFile();
 } else {
 res.send("Please log in");
 }
});

app.get("/logout", (req, res) => {
 req.session.destroy((err) => {
 if (err) {
 return console.log(err);
 }
 res.clearCookie(cookie_name);
 res.redirect("/");
 });
});

app.get("/create/:user", (req, res) => {
 const user = req.params.user;
 const views = 0;
 req.session.user = user;
 user_views.push({ user, views });
 writeUserViewFile();
 res.send(`User ${user} created & logged in`);
});

app.get("/login/:user", (req, res) => {
 // see if this user is in the user_views array
 if (findUser(req.params.user)) {
 req.session.user = req.params.user;
 res.send(`User ${req.params.user} logged in`);
 return;
 } else {
 res.redirect(`/create/${req.params.user}`);
 }
});

app.listen(3000, () => {
 console.log("Server is running on port 3000");
});

Authentication

It won’t have escaped your attention that our view counting app isn’t at all secure. At the moment, any one can log in as ‘jane’ and see her view count. The common way to address this is to also require a password.

I will eventually have to start serving some actual HTML, but for this next step I’ll stick to just using the routes. So our user interface will be this:

Logging in /login/<user>/<password>
- Check the user exists, and that this is the correct password
Creating a new user /create/<user>/<password>
- Save the new user and password to the file.

We’ll do create first:

app.get("/create/:user/:password", (req, res) => { const user = req.params.user; const password = req.params.password; const views = 0; req.session.user = user; user_views.push({ user, password, views }); writeUserViewFile(); res.send(`User ${user} created & logged in`);});

We should probably first check there’s not an existing user with the same name to avoid having two users and the second user never being able to log in.

app.get("/create/:user/:password", (req, res) => { if (findUser(req.params.user)) { res.send(`User ${req.params.user} already exists`); return; } const user = req.params.user; const password = req.params.password; const views = 0; req.session.user = user; user_views.push({ user, password, views }); writeUserViewFile(); res.send(`User ${user} created & logged in`);});

and logging in:

app.get("/login/:user/:password", (req, res) => { const user = findUser(req.params.user); if (user && user.password === req.params.password) { req.session.user = req.params.user; res.send(`User ${req.params.user} logged in`); return; } else { res.send(`Incorrect username or password`); }});

Although this is an improvement, clearly, it would a stretch to call this improved security. Here’s a few things that are jumping out at me, and I’m sure there’s some being missed:

We’re transmitting plaintext passwords over http
We have plaintext passwords in a URL - any intermediate networking that’s recording access logs will the logging our passwords
We’re storing plaintext passwords on our server in the user_views.json file. So when our server is breached, our users’ passwords will get sold on the dark web and they’ll be hacked if they’ve reused name/password combos.
Passwords are limited to characters that reliably work in URLs
Our passwords and user names may be open to injection attacks

So it seems like there is four jobs to do:

Encrypt the passwords
Don’t use URLs for passing this data
Sanitise our inputs
Enforce HTTPS

Passwords

Although I described this as ’encrypting’ that’s not exactly what we’re going to do. The current state of the art for this is to hash and salt passwords before storing them.

hash - turn the password into some gooblygook such that that the hashed version always comes out the same if you put the same password into it. Preferably the hash is always the same length regardless of the length of the input password.

salt - mix some random characters in to the the hashed password so that the combination of hashing and salting the password comes out different every time, even though you are putting in the same password as input to the process.

How this is going to work is that once we get the users password, we’ll hash and salt it, then save the result of that in our array (and eventually file). If someone has access to that file, it’s practically impossible for them to reverse engineer the password - even if they have a known password somewhere else in the file to work from. Then when a user attempts to log in, we’ll take the password they gave us and hash it, and we’ll ‘un-salt’ the password from the file and compare those two hashed versions of the passwords. If they are the same, then we know the passwords were also the same, and we can log this user in.

This is a fun area of computational science, so it’s somewhat tempting to write our own hash and salting functions. This is widely regarded as a Bad Idea. As long as you don’t run into supply chain problems, a proper library, written by cryptographic experts, subject to public scrutiny, that gets updates when vulnerabilities are discovered is always going to be more secure. Almost universally, the answer for JS developers is bcrypt. We’ll install that with npm i bcrypt

Here’s our create and login endpoints using bcrypt with the changes highlighted.

app.get("/create/:user/:password", (req, res) => { if (findUser(req.params.user)) { res.send(`User ${req.params.user} already exists`); return; } const user = req.params.user; const hash = bcrypt.hashSync(req.params.password, saltRounds); const views = 0; req.session.user = user;  user_views.push({ user, hash, views });  writeUserViewFile();  res.send(`User ${user} created & logged in`); });});app.get("/login/:user/:password", (req, res) => { const user = findUser(req.params.user); if (user && bcrypt.compareSync(req.params.password, user.hash)) { req.session.user = req.params.user; res.send(`User ${req.params.user} logged in`); return; } else { res.send(`Incorrect username or password`); }});

Forms

To move the passwords out of the URLs, we’ll present a simple forms to the user for creating and logging in. The log in page is the basic user name / password form you’ve built before. This is served from the app.get("/login") route.

And here’s the endpoint it posts to:

app.post("/login", (req, res) => { const user = findUser(req.body.username); if (user && bcrypt.compareSync(req.body.password, user.hash)) { req.session.user = req.body.username; req.session.save((err) => { if (err) { res.send('Cookie saving error, <a href="/login">try again</a>`'); } else { res.redirect("/"); } }); } else { res.send(`Incorrect username or password, <a href="/login">try again</a>`); }});

Normally express-session will deal with saving the session without us having to worry about, but I decided that a successful login should be followed to a re-direct to the view count page.

Since express-session normally does its saving at the end of a http response, and if we’re redirecting, that response hasn’t happened. There’s a bit more about that here (search for session save).

Apart from that, the changes are really just grabbing the user name and passwords from the form post body instead of the URL.

The /register form is the same as the log in, but with a second password field and some client side scripting to check the two passwords are the same. Processing the new user is very similar to the previous create route.

app.post("/register", (req, res) => { if (findUser(req.body.username)) { res.send(`User ${req.body.username} already exists`); return; } const user = req.body.username; const hash = bcrypt.hashSync(req.body.password, saltRounds); const views = 0; req.session.user = user; user_views.push({ user, hash, views }); writeUserViewFile(); req.session.save((err) => { if (err) { res.send('Cookie saving error, <a href="/login">try again</a>`'); } else { res.redirect("/"); } });});

Sanitising inputs

Cautious developers do not trust any input from users. There are numerous libraries to deal with cleaning it up which I’d recommend you consider, but for our case here let’s think through what the possibilities are:

password - the password is going to be hashed and salted before it is stored, and never used to build HTML. The only risk I can think of would be if a very long password might cause some sort of trouble - so we can just truncate it. Note we don’t even need to tell the user - if we truncate it when they register, and when they log in, they’ll still match.

user name - is stored in a json file, and is output to the user. In the future that output is likely to be HTML.

Here’s our users_view.json

[ { "user": "user1", "hash": "$2b$10$8Zf9LZWH78mWnSKjxKQxXe9TlPoqe7L3SOABcPHIUQ5Pq3jIbVQVm", "views": 16 }, { "user": "user2", "hash": "$2b$10$f/QAQ7we6Hh/hTx35LjfGeYtCY8aRG3ZqbJZqhEZRDXUqxkKCgPhq", "views": 14 }, { "user": "user3", "hash": "$2b$10$KBZe6orYpjG3JeIGhnLDCuyYQXxfVZYBjovGf3XIdU8rr6kXlNrLC", "views": 1 }]

The obvious attack here would be injection. For example a hacker might register with the user name:

user4", "role": "admin

That’s a smart try at privileged escalation. Even if we had an ‘admin’ role, it wouldn’t actually work though since any double quotes will be escaped by JSON.stringify(), but to be cautious, we can eliminate the possibility by just deleting any double quotes out.

Another injection possibility would be with HTML. Perhaps we will display the logged in user later inside a

, something like:

<div>User: user1</div>

Our hacker might try registering this as their user name as:

user1</div><script>alert('you've been hacked')</script></div>

It’s not great to allow anyone on the internet to run code in our visitors’ browsers.

You should really use a library designed by someone who knows what they are doing, but I just wanted to do enough here to prompt you to think about. For that demo purpose, I’m going to replace all " < and > with _

// sanitise a string by replacing all " < and > with _ (underscore) // and truncating it at 20 charactersfunction sanitise(str) { return str.replace(/["<>]/g, "_").slice(0, 20);}

This is all a bit doge. If we are going to have rules like this (and the other rules we should have about min lengths) we should be implementing them in the browser and on the backend, and there should be helpful prompting to the users to enable them to understand and correct their inputs. As I mentioned earlier, this is just to get you to think about it.

HTTPS

In the list of four security improvements we wanted to make, the last one was to enforce HTTPS. There’s two places to do this. One is that when we initialise our session at the top of the app, we can tell it we want ‘secure’ cookies. This setting means that cookies will not be sent over plain HTTP, but only the end-to-end encrypted HTTPS. Currently our cookies contain a user name:

{ "cookie": { "originalMaxAge": null, "expires": null, "httpOnly": true, "path": "/" }, "__lastAccess": 1706315491081, "user": "user1"}

Even though a user name isn’t a lot of information, it could still be critical. If there was rumors about your company being acquired and a hacker leaked that j_bezos had been looking at your view counts, it could have implications. To turn on secure cookies:

app.use( session({ secret: sessionSecret, resave: false, saveUninitialized: true, store: new FileStore(), name: cookie_name, cookie: { secure: true, httpOnly: true, }, }));

Note that your infrastructure needs to support HTTPS for this to be useful - if the cookies are not sent, this particular app is rendered useless.

But we want to enforce HTTPS anyway, because a bigger problem is that if we don’t someone could intercept the login form data and collect plaintext usernames and passwords.

I generally do this by running all web services behind NGINX as a proxy. Then the NGINX configs can be set to redirect all HTTP requests to HTTPS, and valid requests can be passed off to your app. Here’s the sort of thing you might have in a config.

server { listen 80; server_name viewcount.example.com; return 301 https://$host$request_uri;}server { listen 443 ssl; server_name viewcount.example.com; # SSL certificate configuration ssl_certificate /path/to/your/certificate.crt; ssl_certificate_key /path/to/your/private-key.key; # Additional SSL configuration, such as preferred protocols and ciphers, can be added here location / { # Your application configuration goes here # Proxy pass to your Node.js or other application server # Example for proxying to a Node.js server running on localhost:3000 proxy_pass http://localhost:3000; proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection 'upgrade'; proxy_set_header Host $host; proxy_cache_bypass $http_upgrade; }}

There are other security actions that can be taken at the NGINX level - things like rate limiting, blocking particular IP ranges, and access logging - all of which can be handy for protecting your endpoint from bad actors.

Other security

I’d normally have the cookie secret in a .env file that was being .gitignored. A good hacker hobby is to scan public code repos for API keys and other secrets.
Helmet.js is a sort of magic bullet for enforcing some security around request headers.
Probably a stack of other things - ask your senior dev.

Here’s our app with the changes we’ve discussed:

const express = require("express");
const session = require("express-session");
const FileStore = require("session-file-store")(session);
const fs = require("fs");
const app = express();
const bcrypt = require("bcrypt");
const saltRounds = 10;

const user_view_file = "./user_views.json";
const cookie_name = "session_cookie";

app.use(express.urlencoded({ extended: false }));

let user_views = [];
// if the user_views.json file exists, read it and parse it to user_views array
if (fs.existsSync(user_view_file)) {
 user_views = JSON.parse(fs.readFileSync(user_view_file));
}

function writeUserViewFile() {
 fs.writeFile(user_view_file, JSON.stringify(user_views), (err) => {
 if (err) {
 console.log(err);
 return;
 }
 });
}

function findUser(user) {
 return user_views.find((u) => u.user === user);
}

// sanitise a string by replacing all " < and > with _ (underscore)
// and truncating it at 20 characters
function sanitise(str) {
 return str.replace(/["<>]/g, "_").slice(0, 20);
}

app.use(
 session({
 secret: "your-secret-keyz", // This should be a secret, used to sign the session ID cookie
 resave: false,
 saveUninitialized: true,
 store: new FileStore(),
 name: cookie_name,
 })
);

app.get("/", (req, res) => {
 if (req.session.user) {
 const user_view = findUser(req.session.user);
 if (!user_view) {
 res.send(`Error finding user, <a href="/login">try again</a>`);
 return;
 }
 user_view.views++;
 res.send(
 `User "${req.session.user}" has ${user_view.views} views, <a href="/">reload</a> or <a href="/logout">logout</a>`
 );
 writeUserViewFile();
 } else {
 res.redirect("/login");
 }
});

app.get("/logout", (req, res) => {
 req.session.destroy((err) => {
 if (err) {
 return console.log(err);
 }
 res.clearCookie(cookie_name);
 res.redirect("/");
 });
});

app.post("/register", (req, res) => {
 const user = sanitise(req.body.username);
 if (findUser(user)) {
 res.send(`User ${req.body.username} already exists`);
 return;
 }
 const hash = bcrypt.hashSync(req.body.password, saltRounds);
 const views = 0;
 req.session.user = user;
 user_views.push({ user, hash, views });
 writeUserViewFile();
 req.session.save((err) => {
 if (err) {
 res.send('Cookie saving error, <a href="/login">try again</a>`');
 } else {
 res.redirect("/");
 }
 });
});

app.post("/login", (req, res) => {
 const username = sanitise(req.body.username);
 const user = findUser(username);
 if (user && bcrypt.compareSync(req.body.password, user.hash)) {
 req.session.user = username;
 req.session.save((err) => {
 if (err) {
 res.send('Cookie saving error, <a href="/login">try again</a>`');
 } else {
 res.redirect("/");
 }
 });
 } else {
 res.send(`Incorrect username or password, <a href="/login">try again</a>`);
 }
});

app.get("/login", (req, res) => {
 res.status(200).sendFile(__dirname + "/login.html");
});

app.get("/register", (req, res) => {
 res.status(200).sendFile(__dirname + "/register.html");
});

app.listen(3000, () => {
 console.log("Server is running on port 3000");
});

Conclusion

Nearly every web app we write is going to need a user auth and session management solution. In this very long post we’ve looked at a way to develop that from scratch in Express/Node. In the process our code base went from about 10 lines to 130. Now that it’s done however, the only extra code to ensure users are only accessing the routes they should will be a line at the entry point of each route.

Since building out session management is such a common and onerous task, and one that can have serious consequences if not done correctly, you might be wondering if there’s libraries to do some of this, and other, lifting for us. There is, and I plan to look at some in the future.

React Expense Tracker App

Mon, 22 Jan 2024 00:00:00 +0000

I’m focused on React frontend skills these holidays, and working through Mosh’s React 18 course. The exercise today (which I think I nailed, although I spent more than the recommended hour on) was a small app to track expenses. Like most of Mosh’s exercises it was great because it exercised all the understandings up to that point - so it’s a good starting React project. It used Zod for the form validation which is completely new to me, but looks great.

The Specification

This is an app for tracking expenses. Expenses can be in one of three categories, they also must have a description and an amount. There’s a form for entering a new expense with a submit button. The form fields are validated before the expense item is added.

Below the form for adding a new expense item is a list of expenses. Each one shows its description, amount and category. There is also a button to delete this expense. At the bottom of the list is a total for the expenses shown. The expenses shown in the list can be filtered by category with a drop down.

Design Decisions

The first decision in an React app is “What are the components going to be?”. Clearly the form at the top is a component (I called mine AddForm). The bottom section could be two - the filter and the list, but my style is to start with less components then seperate them out if they are getting complex so I considered the filtered list a single component called ExpenseList. In Mosh’s solution, he did have these as separate components, arguing that the filter is likely to become more complex in future which is a sound argument, but too much premature optimisation for me.

Code

With that decision made, we have enough to write the App.tsx. I also decided to persist the array of Expenses to local storage to make it a nicer demo app. So with that, the app component code looks like this:

import { useState, useEffect } from "react";import AddForm from "./AddForm";import { Expense } from "./types.ts";import ExpenseList from "./ExpenseList";import { v4 as uuidv4 } from "uuid";function App() { const [expenses, setExpenses] = useState<Expense[]>(() => { // Try to load expenses from local storage const savedExpenses = localStorage.getItem('mosh-expense'); if (savedExpenses) { return JSON.parse(savedExpenses); } else { // If there are no saved expenses, load the sample expenses return loadSampleExpenses(); } }); useEffect(() => { // Save expenses to local storage whenever they change localStorage.setItem('mosh-expense', JSON.stringify(expenses)); }, [expenses]); return ( <> <AddForm expenses={expenses} setExpenses={setExpenses} /> <hr /> <ExpenseList expenses={expenses} setExpenses={setExpenses} /> </> );}export default App;

Because I’ve got my big boy TypeScript pants on, there’s a types.ts file with the types defined since they are common across a number of components:

export interface Expense { id: string; description: string; amount: number; category: string;}// needed so we can validate the form without the id then// add it laterexport interface FormExpense { description: string; amount: number; category: string;}export interface ExpenseProps { expenses: Expense[]; setExpenses: (expenses: Expense[]) => void;}

The reason for the existence of FormExpense without the id field is that the interaction between Zod and React got very complicated if the id field existed - Zod was determined to validate it. If I left it out of the Zod schema it caused problems because the type and the schema didn’t match, if I made it optional in the schema, it created type mismatch problems that were solvable with typecasting calisthenics but it was ugly and difficult to follow. In the end, I went with these two types and just added the id after validation.

I also made a decision in coding the two types (FormExpense and Expense) like this. I’ve made a small footgun for a future developer in that they might add a new Field to one of them, and neglect to add it to the other. I could have avoided that by extending one, like this:

export interface FormExpense { description: string; amount: number; category: string;}export interface Expense extends FormExpense { id: string;}

That solves that problem, but to my mind is not as clear - conceptually, Expense is the main thing here, and really FormExpense is a derivative of it - just with the id removed. The TypeScript language designers could have helped me out here with some syntax, but I forgive you Anders because of the good living I once made out of Delphi. In my alternative timeline TypeScript it could look like this:

export interface Expense { description: string; amount: number; category: string;  id: string;}export interface FormExpense reduces Expense { id: %removed;}

So, I don’t love the duplication in the existing type definition, but it seemed like the lessor of two evils, the definitions are right next to each other, and we are working in TypeScript so the future developer will discover their mistake at the time they’ll make it.

There’s also a subtle design decision in the very simple ExpenseProps worth thinking about. Firstly, I like having the props here with the Expense definition, but mostly how simple they are. There’s no addExpense(), updateExpense(), or deleteExpense(). That’s because, to the extent they are needed, they are dealt with inside each component.

Components are already tightly bound to their data types, so we’re not making that worse by having this code with the compnent, but it would be perfectly valid to argue that all the data manipulation for expenses should be in one place. That argument would be won for me the second I needed to duplicate any of it - for example if two different components needed to delete an expense. But as the app stands now, this is neater, so that’s what I’ve gone with.

Expenses List

The mechanism for the filter is that we have a local function that returns the filtered list based on the selected category which is a state variable of the ExpensesList component. That selectedCategory is updated from the onChange of the drop-down selector.

The JSX just builds a table by .mapping the filtered expenses. I don’t love the table code - it looks clunky. When I came back to look at it there were still some refactoring opportunities in it. Let’s have a look:

<table className="table-bordered"> <thead> <tr> <th>Description</th> <th>Amount</th> <th>Category</th> <th></th> </tr> </thead> <tbody> {filteredExpenses.map((expense) => ( <tr key={expense.id}> <td className="centred-td">{expense.description}</td> <td style={{ textAlign: "right" }}> $ {expense.amount.toLocaleString("en-US", { minimumFractionDigits: 2, maximumFractionDigits: 2, })} </td> <td className="centred-td">{expense.category}</td> <td className="centred-td"> <button className="btn btn-outline-danger" onClick={() => handleDelete(expense.id)} > Delete </button> </td> </tr> ))} </tbody> <tfoot> <tr> <td></td> <td style={{ textAlign: "right", fontWeight: "bold" }}> $ {filteredExpenses .reduce((total, expense) => { return total + expense.amount; }, 0) .toLocaleString("en-US", { minimumFractionDigits: 2, maximumFractionDigits: 2, })} </td> <td colSpan={2}></td> </tr> </tfoot></table>

The first thing I’m noticing is the code to format the amounts to US currency. That’s in there twice - once for each expense, and once for the total. We could write a function for that, but since we’re in React-land, let’s make it a component. Ideally we’d extract the user’s local currency from the browser settings somehow, but I don’t think that’s possible. In a real app, I guess we’d let them set it in settings. For the moment, they’ll just have to live in dollar land.

interface TDCurrencyProps { children: number; fontWeight?: string; }function TDCurrency({children, fontWeight = "normal"}:TDCurrencyProps) { return ( <td style={{ textAlign: "right", fontWeight: fontWeight }}> $ {children.toLocaleString("en-US", { minimumFractionDigits: 2, maximumFractionDigits: 2, })} </td> );}export default TDCurrency;

Now we’re thinking React-ivly! The table looks better already. Also, since now the alignment for the number column is in its own component, the centred styling can be applied to all the <td>s so I can remove the class I had added for that:

<table className="table-bordered"> <thead> <tr> <th>Description</th> <th>Amount</th> <th>Category</th> <th></th> </tr> </thead> <tbody> {filteredExpenses.map((expense) => ( <tr key={expense.id}> <td>{expense.description}</td> <TDCurrency>{expense.amount}</TDCurrency> <td>{expense.category}</td> <td> <button className="btn btn-outline-danger" onClick={() => handleDelete(expense.id)} > Delete </button> </td> </tr> ))} </tbody> <tfoot> <tr> <td></td> <TDCurrency fontWeight="bold">{filteredTotal}</TDCurrency> <td colSpan={2}></td> </tr> </tfoot></table>

Much nicer. Would it be crazy to component-ise that delete button as well? Probably, but while we’re on a roll, and this is starting to feel like fun.

I really feel I’m starting to get the benefits of React that we’re paying for with the tooling complexity and larger bundle size.

interface TDDeleteButtonProps { onClick: (id: string) => void; id: string;}function TDDeleteButton({ onClick, id}: TDDeleteButtonProps) { return ( <td> <button className="btn btn-outline-danger" onClick={() => onClick(id)} > Delete </button> </td> );};export default TDDeleteButton;

I’m much happier with the table now:

<table className="table-bordered"> <thead> <tr> <th>Description</th> <th>Amount</th> <th>Category</th> <th></th> </tr> </thead> <tbody> {filteredExpenses.map((expense) => ( <tr key={expense.id}> <td>{expense.description}</td> <TDCurrency>{expense.amount}</TDCurrency> <td>{expense.category}</td> <TDDeleteButton onClick={handleDelete} id={expense.id}/> </tr> ))} </tbody> <tfoot> <tr> <td></td> <TDCurrency fontWeight="bold">{filteredTotal}</TDCurrency> <td colSpan={2}></td> </tr> </tfoot></table>

The selector at the top of the expense list, is, well, okay.

<div className="mb-3 category-selector"> <label htmlFor="category" className="form-label"> Category:{" "} </label> <select id="category" name="category" className="form-control drop-down" onChange={handleCategoryChange} > <option value="All">All</option> <option value="Groceries">Groceries</option> <option value="Utilities">Utilities</option> <option value="Entertainment">Entertainment</option> </select></div>

The specification only specified these three categories - Groceries, Utilities, and Entertainment. and I have them hard-coded here, and in the add form. In Mosh’s version he’s made them a enum which is definitely nicer and future-proof-ier for a very small increase in complexity.

Mosh

I really enjoy Mosh’s videos and courses. He generally puts the first hour of his courses on Youtube, so you can try before you buy. He has a knack for anticipating the questions that occur to you as he’s explaining something so you feel a sense of continually being slightly challenged, but never overwhelmed. Hard recommend if you are learning programming.

CSS for React Components

Fri, 12 Jan 2024 00:00:00 +0000

Subscribe to my UX design course 😉

If you think back to HTML as being a document with headings and paragraphs and other semantic bits, it made a lot of sense to have the styles (expressed as CSS) separate to the document. This allows us to change the styles without touching the document - perhaps the user wanted a dark theme, needed the text bigger for accessibility, or perhaps the document was being consumed in some other way - for example a screen reader - so the styles were superfluous.

In tension to this idea, is the idea that all the code related to a single thing should be encapsulated in one place. This is why we invented object orientated programming - we are creating such huge software systems that for a human to be able to maintain them, they need broken down into chunks that can be fully held in mind while we are working on them. Also, when we are talking about a modern single page application, we’ve come a long way from thinking of a web ‘page’ as being a document to be passively consumed.

Since the point of React is to create reusable ‘components’ where the JS and HTML are written together, it’s reasonable to wonder if perhaps the CSS shouldn’t be in there too.

Let’s look at a few different approaches to managing styles for our components in React.

Old Style Global

Most times when I’m writing vanilla JS, if I’m not leveraging off Pico or Bootstrap I have a single site-wide styles.css file. Obviously this is going to be an option for a React app too. We’d just link it from the index.html in the root of our folder. Job’s a goodun.

The most common downside of this approach, that I come across, is that I change the rest of the HTML through the various iterations, but never clean up the CSS. So I get left with bits of CSS that I’m not sure if they are being used somewhere - so I’m not brave enough to delete them because my UX testing is not good enough, so those fragments just end up sitting around forever.

Old Style local

The first CSS we ever wrote was just stuffed in the HTML in style tags, and of course that’s still an option. We can use variables to help with the readability, and end up with something like this:

function Card(props: CardProps) { const cardStyles: React.CSSProperties = { width: '120px', boxShadow: '0 4px 8px 0 rgba(0, 0, 0, 0.2)', textAlign: 'center', backgroundColor: 'cornsilk', margin: '10px', padding: '10px', transform: 'scale(1)', transition: 'transform 0.3s', }; const pictureStyles: React.CSSProperties = { width: '100px', height: '150px', margin: 'auto', borderRadius: '50%', border: '2px solid #000', }; const nameStyles: React.CSSProperties = { color: 'black', fontSize: '18px', }; return ( <div style={cardStyles}> <img style={pictureStyles} src={props.image} alt={props.name} width={100} /> <p style={nameStyles}>{props.name}</p> </div> );}

Initially, this seemed like a good solution, but there’s a few bumps involved. The first is that, or course, this is not real CSS. You can see from the type that TypeScript forced me to use (React.CSSProperties) that these are React types that will get turned into CSS at some distant time in the future. Because of this, there are some oddities - my muscle memory wants to type the CSS property names like box-shadow, but for this they need to be camel case.

The next issue of ‘it’s not actually CSS’ was when I wanted to do my hover effect for the cards. In CSS this is just:

.card:hover { transform: scale(1.05); background-color: lightgoldenrodyellow;}

If I want to do that with inline styles, I need to capture the MouseEnter and MouseLeave events for the card, then swap the inline styles in and out in code. Ain’t nobody got the time for that.

CSS & Component Libraries

A common and appealing answer to inline styles is to use CSS libraries such as BootStrap, TailWind etc. As well as allowing you to add complex styles into JSX with short memorable tags, they often enforce a design aesthetic without the developer having to put much thought into it. Since I have minimal design skills, that’s an appealing option.

Taking this a step further are component are React component libraries such as chakra, Material UI, and Ant. With these systems, you get styled components (that can be modified) that follow a unified design - you’re not really thinking of CSS but at a high level of abstraction.

component.css

I guess the default (since it’s generated like this in the create-app step) way of managing CSS closer to our components is to have a .CSS file for each component. This overcomes the ‘cleaning up’ problem described above. If I’m deleting the NavBar component from this project, I can safely eliminate the NavBar.css file at the same time.

The component CSS file is just imported at the top of the component file.

If you build an app with the CSS in component.css files like this, all the CSS is combined into a single CSS file by the bundler. This raises the possibility of naming conflicts leading the hard to track down problems later. To avoid this I tend to use the component name as a class name and use that to tightly scope the CSS to avoid it bleeding out into other elements.

Modules

A bit fancier approach is to use CSS modules. This is somewhat similar to the component CSS files described above. Our CSS files have to be renamed to end in .module.css, then in the code where you normally insert the class names, you use the class names from the styles library:

import styles from "./Card.module.css";// props for cardinterface CardProps { name: string; image: string;}function Card(props: CardProps) { return ( <div className={styles.card}> <img className={styles.picture} src={props.image} alt={props.name} width={100} /> <p className={styles.name}>{props.name}</p> </div> );}export default Card;

Those class names match the CSS:

.card { width: 120px; box-shadow: 0 4px 8px 0 rgba(0, 0, 0, 0.2); text-align: center; background-color: cornsilk; margin: 10px; padding: 10px; /* tilt and enlarge on mouseover */ transform: scale(1); transition: transform 0.3s;}.card:hover { transform: scale(1.05); background-color: lightgoldenrodyellow;}.name { font-size: 18px; color: black;}.picture { width: 100px; height: 150px; margin: auto; border-radius: 50%; border: 2px solid black;}

The advantage of using modules, is that you now do not have worry about name clashes. If I build the app and have a look in the generated CSS file, you can see how that works:

._card_j2a8o_2 { width: 120px; box-shadow: 0 4px 8px #0003; text-align: center; background-color: #fff8dc; margin: 10px; padding: 10px; transform: scale(1); transition: transform 0.3s;}

All the generated CSS has unique class names created for it! Presumably these match the ones being used in the JSX, and this explains why you need to use the styles. names in your components.

Styled Components

There are several libraries that tackle the issue of what to do about CSS in React. One of the more popular ones is Styled Components. How this works is that you define a base component with some styles in it (with a weird backticky syntax), then build your own components from the styled one. It makes more sense when you see it than my written explanation.

import styled from "styled-components";const StyledDiv = styled.div` width: 120px; box-shadow: 0 4px 8px 0 rgba(0, 0, 0, 0.2); text-align: center; background-color: cornsilk; margin: 10px; padding: 10px; transform: scale(1); transition: transform 0.3s; &:hover { transform: scale(1.05); background-color: lightgoldenrodyellow; }`;const StyledImage = styled.img` width: 100px; height: 150px; margin: auto; border-radius: 50%; border: 2px solid black;`;const StyledP = styled.p` font-size: 18px; color: black;`;// props for cardinterface CardProps { name: string; image: string;}function Card(props: CardProps) { return ( <StyledDiv className="card"> <StyledImage className="picture" src={props.image} alt={props.name} width={100} /> <StyledP className="name">{props.name}</StyledP> </StyledDiv> );}export default Card;

If you don’t mind more dependencies, this is a great solution - I like the clarity of the philosophy of creating styled versions of regular elements as React elements, then using them as the building blocks of our new component. All the CSS attributes you’ve learned are still there.

The only thing I needed to look up was how to deal with my hover. In the CSS this had been:

.card { width: 120px; box-shadow: 0 4px 8px 0 rgba(0, 0, 0, 0.2); text-align: center; background-color: cornsilk; margin: 10px; padding: 10px; transform: scale(1); transition: transform 0.3s;}.card:hover { transform: scale(1.05); background-color: lightgoldenrodyellow;}

Which had to be translated into this with the & syntax:

const StyledDiv = styled.div` width: 120px; box-shadow: 0 4px 8px 0 rgba(0, 0, 0, 0.2); text-align: center; background-color: cornsilk; margin: 10px; padding: 10px; transform: scale(1); transition: transform 0.3s; &:hover { transform: scale(1.05); background-color: lightgoldenrodyellow; }`;

So - lots of options for dealing with CSS in React. I haven’t written much, so I’m not sure what my personal preference is. styled-components is definitely the most elegant of the approaches I’ve looked at here, but I’m a dependency calorie counter so my natural inclination is look elsewhere. The CSS file for each component seems like the next best system - I like that there’s no special hooks in the JSX besides the class names I would have used in ordinary HTML, but then you have the risk of name clashes. They can be avoided with the modules - but I am sort of used to dealing with that anyway.

React - a To Do Example

Mon, 08 Jan 2024 00:00:00 +0000

Since I’m on a roll making different versions of the To Do app, this might be a good time to talk about React. React is one of the giants of front end libraries. It’s based on a few big ideas - and to work effectively in React you need to wrap your head around these.

Overview

Components - when you are developing in React, the starting point of your build is to decompose the user interface in to logical pieces. These components (comprising a mixture of HTML and Javascript) will be the building blocks of your app. In a good composable architecture components are reusable, and that is true for React (there are several sources of components you can pull in). For example, if you created some sort of special slider for your app, it is possible to reuse that quite easily.

Declarative - this was one of the big barriers when I was learning SwiftUI. The UI is just described. It’s not a big step when you’re coming from HTML - all that is is a description of the user interface. The next step of this is that React deals with using the state of your data model to update the UI. This means that these state-UI connections are made very explicit (which I like) and protected. For example if there’s a counter on a web page, you can’t change the HTML of the page to increment the counter, and in fact, you can’t directly change the counter. These things are wrapped up so React can manage them, and you have to play by React’s rules.

Virtual DOM - since each component knows what state it depends upon, and changing that state causes the component to be redrawn, it quickly gets expensive with parts of the page being reloaded. To improve performance, and reduce often unneeded flashes of content loading, React keeps a copy of the DOM and makes changes to it rather than the real DOM. Since this is instrumented, React can easily see what changes really need to be percolated to the browser DOM and can manage how that happens in the most efficient way. Sometimes I think I know what I want to happen for something to be efficient but in React, it’s often not in your control, and you just need to trust the system.

Tooling

We’re leaving my comfort zone of straightforward development environments, along with the major benefit of working in Javascript. The complexity of the React system requires a build step to produce the production artifacts. Whether you use the standard Create-React, or Vite (as I did for this project) you get a system for bundling the code, mapping it (since for debugging you need a way to translate the source line that’s running back to the human readable source), and a development server to run the app while you’re working on it.

These things inevitably add to complexity and errors, and are the reason that big projects start to need tools like development containers to remove a pain point. Like anything, you get better with experience, but especially at the start there’s a developer cost involved. React is incredibly popular, so most people’s calculus on this is that the extra complexity in project management is made up for with improved developer experience.

In any case, I got started by typing npm create vite@latest . and then choosing React and Javascript. This created a starter project, and spun it up in the development server. In the package.json file it gives you, there is a build command that can be run with npm run build to create the output files. I had issues with the development server that serves up these frontend files - since I also needed to run a real server (the unchanged REST API server for Todo Items data from the earlier project) on a different port. Then it complained - thinking it was part of a cross-site scripting hack. To overcome it, I just built the production code for each round of testing - with such a little project this wasn’t a hardship. But when I did have an error to track, it pointed me to a line number that turned out to be about 20 pages of minified code.

Components

Just a little reminder of how this app looks so we can think about how we’re going to break it down. There is a list of todo items, each one has a button to delete it (which we do when it has been completed) then at the bottom there’s a little form to add a new item (by clicking the button or hitting enter) to the bottom of the list.

I ended up with three components - the App (every React app has one), the TodoList, and the AddTodoForm. Note that I could easily have had a forth component - the TodoItem. This is a bit of matter of taste - I probably would have if I wanted to do something fancier like editing in place - but for the current UX the cost of extracting out another component wasn’t worth the benefit.

Anatomy of a component

I claimed earlier that a component was it’s HTML and Javascript wrapped up together which, while a massive simpliciation, is a good place to start thinking about it. Every component is just a function that returns a bunch of (templated) HTML. We’ll start off by developing our AddTodoForm. At it’s simplest, it could be something like this:

function AddTodoForm() { return ( <form> <input type="text" name="todo_item" id="todo" required /> <button type="submit">Add</button> </form> );}export default AddTodoForm;

But this little form component can’t really talk to the world, where as we need it to add a todo item. First, let’s track any changes to the text field.

import { useState } from 'react';function AddTodoForm(props) { const [value, setValue] = useState(''); return ( <form onSubmit={handleSubmit}> <input type="text" name="todo_item" id="todo" value={value} onChange={e => setValue(e.target.value)} required /> <button type="submit">Add</button> </form> );}export default AddTodoForm;

This is an example of the very explicit management of state I was talking about earlier.

const [value, setValue] = useState('');

useState() is a React hook for managing state. This line gives us a getter (value) and setter (setValue) for this variable, and set’s its initial state to ‘’. If the value changes the component will be redrawn. React will know that the value has changed as this is built into the setValue() function where we never need see or worry about it. If you foolishly decided to side-step React and assign directly to value, I guess there’d be a runtime error, or even worse, no error and the management of the DOM state wuld fall into some type of chaos.

What we’re doing with this value (which I’m now realising is very badly named) is using it to collect the text input from out form. It’s constantly updated as the user types.

onChange={e => setValue(e.target.value)}

So that’s our value sorted, but of course we need to add it to our data model. This model is being managed at the App level so there’s a bit of juggling needed to get it out to there. This ‘plumbing’ cost is the downside of these types or framework, but it’s not really complex and quickly becomes routine.

We’ll start at the top, here’s the relevant code from App.jsx - our App component.

 const [todos, setTodos] = useState([]); const addTodo = (newTodo) => { fetch('http://localhost:3000/todos', { method: 'POST', headers: { 'Content-Type': 'application/json', }, body: JSON.stringify(newTodo), }) .then(response => response.json()) .then(data => { // Update the todos state with the new todo setTodos([...todos, data]); }); };

This useState hook should be starting to look familiar. We read the todos from todos, and write to them with setTodos()

addTodo does the actual work of saving it to the database (via our REST API) then creates a new todos array by adding our new one to the end. But we need to pass this down into our AddTodoForm. Here’s the main part of the App that returns out HTML, in this case that’s the list of todos and our form:

 return ( <main> <h1>To do</h1> <TodoList todos={todos} onDeleteTodo={deleteTodo}/> <AddTodoForm onAddTodo={addTodo} /> </main> )

You can see here that the todos array and a function deleteTodo() are passed into the the TodoList component, and that our addTodo() function is passed to the AddTodoForm.

In React, things like this that are passed into a component are passed as a single object variable called ‘props’ - short for properties. It seems crazy to me to be bundling them like this in a language in 2023 rather than passing them explicitly as separate variables. This lack or clarity about what’s being passed into a component is doubtless one of the reasons TypeScript is such a common combo with React. It’s certainly the first time I’ve felt the need of it.

There is a lighter partial solution to adding types to props so that the linter can call out any issues - this is the PropTypes library - once it’s installed, we import it with import PropTypes from 'prop-types'; then we can add a definition for the props at the bottom of the file. For our AddTodoForm, this would look like this:

AddTodoForm.propTypes = { onAddTodo: PropTypes.func.isRequired,};

Now the linter will prevent us from using anything other than props.onAddToDo, and it will flag if it’s being used as anything other than a function.

Anyway, the props are passed in and we can extract the function from it to use.

import { useState } from 'react';function AddTodoForm(props) { const [value, setValue] = useState(''); const handleSubmit = (event) => { event.preventDefault(); if (!value) return; props.onAddTodo({ todo_item: value }); setValue(''); }; return ( <form onSubmit={handleSubmit}> <input type="text" name="todo_item" id="todo" value={value} onChange={e => setValue(e.target.value)} required /> <button type="submit">Add</button> </form> );}export default AddTodoForm;

The arrangement of elements in this file will start to become familiar - there’s often some state at the top with the useState() hook, then a few handler functions, then our final return of the ‘HTML’.

Our TodoList is a bit simpler in that it doesn’t have any handlers, but a better illustration of using PropTypes since it has access to the global state of the todos.

import PropTypes from 'prop-types';function TodoList(props) { return ( <ul> {props.todos.map(todo => ( <li key={todo.id}> {todo.todo_item} <button onClick={() => props.onDeleteTodo(todo.id)}> Done </button> </li> ))} </ul> )}TodoList.propTypes = { todos: PropTypes.array.isRequired, onDeleteTodo: PropTypes.func.isRequired};export default TodoList;

Conclusion

You would not sensibly reach for React for a project this size - the complexity of the tooling, and the fact that we’re now shipping 150K of Javascript to do something we were nicely achieving in 2K of vanilla, or 15K with htmx makes me deeply uncomfortable. Nevertheless, hundreds of thousands of developers can’t be wrong - React’s component model is a powerful one for building modern single page applications, especially when it allows you to pull in components from public or corporate collections.

I plan on doing some more React - partly because its just such big part of the webdev world, and partly because I’d like to get some experience with TypeScript, so I’m fishing around for a medium size project to play with both of these technologies.

(source)

htmx - A To Do Example

Fri, 05 Jan 2024 00:00:00 +0000

HTMX is an interesting project to me, and I’ve used it a bit in my large collection of 70% completed side projects, but haven’t really discussed it here. The plan for this post is to talk briefly about what it is exactly, then convert a simple ‘conventional’ (HTML/CSS/Javascript) app to htmx and think about some the differences.

htmx

You could (I recommend you do) read the book about the concepts behind htmx. Carson Gross (the man behind htmx) calls it a book, but its quite the treatise, it could fairly be called a manifesto.

The book points out that the ‘hyper’ bit of hypertext markup language, is currently limited to a couple of tags - <a>, and <form>.

The anchor tag sends a request to the server that says something like ‘fetch the HTML from this URL and completely replace the current view with it’
The form tag with a post method says to the server ‘do something with this data’, or with the get method, ‘get me the thing I’m describing’.

So the first paradigm shift in htmx is ‘why don’t we give all html tags the superpower to make server calls?’

Of course, we can do this in JavaScript - call any endpoint with put, patch, get etc, then add listeners for the code to many parts of the DOM, but Carson has imagined (and then implemented) an alternative timeline where HTML kept being developed to have this capability without the developer having to leave their beloved HTML.

The other ‘big thing’ is what the server returns, and what we do with it. We’re used to just getting data (JSON these days, but XML in the days when senior devs had big beards) then the front-end JavaScript needing to know about the data and it’s format and the intent so it can present it, and the affordances (options for the user to do things with it) available. These things (the presented data and the affordances) combine to represent the application state.

This is an old concept from the birth of REST with a terrible acronym HATEOAS - “Hypermedia as the engine of application state”. Just passing some JSON (which is usually regarded as a REST practice) breaks this constraint. Instead, we’d pass back (in practical terms) HTML that contains the data and it’s affordances (the books sometimes calls this the hypermedia representation of the data). In my Todo app, this might be the to do items, in a list, with the button to mark them as done.

A key benefit of HATEOAS at it’s inception was that the server in this relationship could change business logic without any change to the client end. That argument can still be made to some extent, but a more important benefit of HATEOAS for htmx is that it means so little processing needs done in the client, we don’t need a programming language beyond what we’ve got in html/x. This is the second big thing in htmx:

Returning application state (data plus affordances) in HTML from the server means we don’t need an extra programming language to process it.

In practice

So what does this all mean in practice?

In htmx, HTML tags get attributes if they need to talk to the server. The attributes say what endpoint they are hitting, with what method, and where to put the returned HTML.
The server responds with chunks of HTML.
The client slots that in where it’s supposed to go.
You can write SPA type applications where a part of the page can be updated without a full refresh, without any Javascript*

This last point explains some of the keen interest of htmx from the non-JavaScript language people. If you’re a Python, Go or Ruby developer with low love for JS, this is an easy sell.

Traditional Version

I want to show you a demo htmx app, but first let’s look at the Javascript version. It is the Todo app from day one of that coding Udemy you never finished. There’s a list of items to do, shown sequentially on the screen. Each one has a button to mark it as done, and at the bottom, a spot to enter a new one.

My ‘basic’ Javascript version is based around a Node/Express server. Express serves the .html, .css & .js statically, then runs an API for creating, reading and deleting the Todo items as JSON.

<!-- index.html for simple todo app that uses node endpoints to process json-->
<!DOCTYPE html>
<html lang="en">
<head>
 <meta charset="UTF-8">
 <meta name="viewport" content="width=device-width, initial-scale=1.0">
 <title>Todos</title>
 <link rel="stylesheet" href="./styles.css">
</head>
<body>

<!-- Main section-->
 <main>
 <h1>To do</h1>
 <ul id="todos_list"></ul>
 <!-- the list of todo items from the database gets inserted here-->

 <!-- form to add a todo -->
 <form action="/todos" method="POST">
 <input type="text" name="todo" id="todo" required>
 <button type="submit">Add</button>
 </form>
 </main>

 <script src="index.js"></script>
</body>
</html>

Nothing fancy there. The Todo items will go in the

get /todos

post

delete

// Simple Express ToDo app using SQLite3

const express = require('express');
const app = express();
const port = 3000;

const sqlite3 = require('sqlite3').verbose();
const db = new sqlite3.Database('db/todos.sqlite');

app.use(express.json());
app.use(express.static('public'));

app.get('/todos', (req, res) => {
 db.all('SELECT * FROM todos', (err, rows) => {
 if (err) {
 res.status(500).json({ error: err.message });
 }
 res.status(200).json(rows);
 });
});

app.post('/todos', (req, res) => {
 if (!req.body.todo_item) {
 return res.status(400).json({ error: 'Missing todo_item field in request body' });
 }
 db.run('INSERT INTO todos (todo_item) VALUES (?)', req.body.todo_item, function(err) {
 if (err) {
 return res.status(500).json({ error: err.message });
 }
 // well behaved APIs return the newly created resource
 db.get('SELECT * FROM todos WHERE id = ?', this.lastID, (err, row) => {
 if (err) {
 return res.status(500).json({ error: err.message });
 }
 res.status(201).json(row);
 });
 });
});

app.delete('/todos/:id', (req, res) => {
 db.run('DELETE FROM todos WHERE id = ?', req.params.id, (err) => {
 if (err) {
 res.status(500).json({ error: err.message });
 }
 res.status(204).end();
 });
});

// if it doesn't exist, create the table
db.run('CREATE TABLE IF NOT EXISTS todos (id INTEGER PRIMARY KEY AUTOINCREMENT, todo_item TEXT)');

// close the database gracefully on exit
process.on('SIGINT', () => {
 db.close((err) => {
 if (err) {
 return console.error(err.message);
 }
 console.log('Database closed.');
 });
 process.exit(0);
 });

// start the server on port 3000 
app.listen(port, () => console.log(`Todo app listening on http://localhost:${port}`));

All the work of translating the data into a representation is being done in the Javascript.

// index.js - code for simple todo app
// json data served from node endpoint

function createTodoItem(todoItemText, id) {
 const li = document.createElement('li');
 const button = document.createElement('button');
 button.innerHTML = 'Done';
 button.addEventListener('click', () => handleDelete(id, li));

 // Create a text node with the todo text and append it to the li
 const todoTextNode = document.createTextNode(todoItemText);
 li.appendChild(todoTextNode);

 // Then append the delete button
 li.appendChild(button);

 return li;
}

function handleDelete(id, li) {
 fetch('/todos/' + id, {
 method: 'DELETE'
 })
 .then(() => {
 li.remove();
 });
}

// Fetch the todo items to build the list
fetch('/todos')
 .then(res => res.json())
 .then(todos => {
 // Loop through todos and add to list
 todos.forEach(todo => {
 const li = createTodoItem(todo.todo_item, todo.id);
 document.querySelector('#todos_list').appendChild(li);
 });
 });

// handler for adding a todo item
document.querySelector('form').addEventListener('submit', (e) => {
 e.preventDefault();
 const todo_item = document.querySelector('#todo').value;
 fetch('/todos', {
 method: 'POST',
 headers: {
 'Content-Type': 'application/json'
 },
 body: JSON.stringify({ todo_item })
 })
 .then(res => res.json())
 .then(data => {
 const li = createTodoItem(todo_item, data.id);
 document.querySelector('#todos_list').appendChild(li);
 document.querySelector('#todo').value = '';
 });
});

Again, there’s no startling innovation. When it loads, the API is called to get the list of todo items which are turned into list items with ‘Done’ buttons and appended to the

htmx Version

index.html

<!DOCTYPE html><html lang="en"><head> <meta charset="UTF-8"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> https://unpkg.com/htmx.org/dist/htmx.min.js <title>Todos</title> <link rel="stylesheet" href="./styles.css"></head><body><!-- Main section--> <main> <h1>To do</h1> <ul id="todos_list" hx-get="/todos" hx-trigger="load"></ul> <!-- the list of todo items from the database gets inserted here--> <!-- form to add a todo --> <form hx-post="/todos" hx-target="#todos_list" hx-swap="beforeend" hx-on::after-request="this.reset()"> <input type="text" name="todo_item" id="todo" required> <button type="submit">Add</button> </form> </main>                  </body></html>

The script tag at the top pulls in html (which is actually just 14K of gzipped Javascript) from a CDN. You can alternatively download it and serve it statically with your other assets. It’s worth noting, that’s all the tooling involved - no build tools etc.

<ul id="todos_list" hx-get="/todos" hx-trigger="load"></ul>

This is the unordered list I want the todo items to go into. When the page is loaded (hx-trigger) we’ll hit the app.get("/todos") endpoint (hx-get). I don’t need to specify where the returned html goes to - by default it’s the innerHTML of the calling tag.

<form hx-post="/todos" hx-target="#todos_list" hx-swap="beforeend" hx-on::after-request="this.reset()">

This is the form for adding a new todo item. It’s going to hit the app.post("/todos") endpoint (hx-post), and the returned HTML (which will be the the new list item to add to the list) needs to go onto the unordered list we talked about earlier (hx-target). The hx-swap=“beforeend” part means the returned list item will be inserted just before the end of the

After the user has hit return or the ‘Add’ button to save their todo item, I don’t want the text they just entered to be sitting there, so a tiny Javascript snippet needs to be run. There are a heap of html hooks for these sorts of jobs (hx-on::afterrequest).

The final change is that we’ve removed the script tag at the bottom that referred to our own Javascript code. None of that is needed now - the application state is delivered as complete HTML by the server.

server.js

Now our node/express server. I’ll dump the whole file here, then talk about each part.

// Simple Express ToDo app using SQLite3 & HTMX

const express = require('express');
const app = express();
const port = 3000;

const sqlite3 = require('sqlite3').verbose();
const db = new sqlite3.Database('db/todos.sqlite');

app.use(express.static('public'));
app.use(express.urlencoded({ extended: true }));

function htmlForTodoItem(uid, item_text) {
 let html = `<li>${item_text}`;
 html += `<button hx-delete="todos/${uid}" hx-target="closest li" `;
 html += 'hx-swap="outerHTML">Done</button></li>';
 return html;
}

app.get('/todos', (req, res) => {
 db.all('SELECT * FROM todos', (err, rows) => {
 if (err) {
 console.log(err.message);
 return res.status(500).send('<li>database error</li>');
 }
 // loop through the rows and create a list item for each
 let list = '';
 rows.forEach(row => {
 list += htmlForTodoItem(row.id, row.todo_item);
 });
 res.status(200).send(list);
 });
});

app.post('/todos', (req, res) => {
 if (!req.body.todo_item) {
 return res.status(400).send('Missing todo_item field in request body');
 }
 db.run('INSERT INTO todos (todo_item) VALUES (?)', req.body.todo_item, function (err) {
 if (err) {
 console.log(err.message);
 return res.status(500).send('<li>database error</li>');
 }
 // return just this item for HTMX to insert at the list end
 res.status(200).send(htmlForTodoItem(this.lastID, req.body.todo_item));
 });
});

app.delete('/todos/:id', (req, res) => {
 db.run('DELETE FROM todos WHERE id = ?', req.params.id, (err) => {
 if (err) {
 console.log(err.message);
 return res.status(500).send('<li>database error</li>');
 }
 res.status(200).end();
 });
});

// if it doesn't exist, create the table
db.run('CREATE TABLE IF NOT EXISTS todos (id INTEGER PRIMARY KEY AUTOINCREMENT, todo_item TEXT)');

// close the database gracefully on exit
process.on('SIGINT', () => {
 db.close((err) => {
 if (err) {
 return console.error(err.message);
 }
 console.log('Database closed.');
 });
 process.exit(0);
});

// start the server on port 3000 
app.listen(port, () => console.log(`Todo app listening on http://localhost:${port}`));

The first change in the top of the file is that we’ve removed the middleware for JSON request bodies and switched to URLEncoded which is what htmx will be sending us by default. Then we dive into this function which builds the HTML for each of the Todo items encapsulated in an

with it’s ‘Done’ button to delete it.
```
function htmlForTodoItem(uid, item_text) { let html = `<li>${item_text}`; html += `<button hx-delete="todos/${uid}" hx-target="closest li" `; html += 'hx-swap="outerHTML">Done</button></li>'; return html;}
```
It’s my habit to name these little fragments like this - htmlForXXXX - and group them at the top of the file. I used to use EJS templates, and that’s a valid approach, but the functions seem less complicated somehow.

The remaining code is our endpoints:
- app.get('/todos' - called when the page loads. Calls the htmlForTodoItem() for each todo item in the database, the returns all of that to be inserted into the
  - app.post('/todos' - for adding a single new todo item. It saves it in the database, then returns the new item as an
  - to be inserted at the end of the list.
  - app.delete('/todos/:id’ - this is the route called by the ‘Done’ button on each todo item. It deletes the item from the database and pointedly returns nothing with that res.status(200).end(); - this is important, because the way the
  - is being deleted from the page is that it’s being replaced with what is returned - ie nothing.
  These match up to the original versions, with the only significant difference being that instead of returning raw JSON, the HTML is being returned.
  
  Reflections
  
  As far as I can see, these two apps are identical to the user. The htmx version is going to be a bit larger over the wire with that initial pull down of 14K, but we’re only saving 1.6K of Javascript by eliminating our index.js. That 14K is a big deal in a tiny app like this, but probably not for any serious app.
  
  In regard to the developer experience; Is the htmx version easier to live with and maintain? For me, I think the answer is yes - I’d rather think at the level of ‘add this html to the end of the list’ rather than ‘document query selector appendtochild’ then programmatically build a list item from the JSON i’m interpreting , so it’s a useful abstraction. I acknowledge this is going to be a highly subjective thing.
  
  The killer usecase for htmx is going to be for people who want a bunch of cool modern stuff in their web apps, but who don’t want to write frontend Javascript. So for Go, Rust, PHP, Ruby etc people it’s probably a no-brainer. This is probably also my situation, I’m a strong server-side Javascript programmer, but have little interest in learning all the cool stuff around the DOM.
  
  htmx might appeal to developers with a small-web flavour to their dev-politics. If you like semantic html, accessibility, reducing bandwidth and power consumption of your apps, and being a good guardian of your users’ data on the servers you control, you’ll probably like htmx. If you like AWS lambda functions, Angular, Next, Vercel and outsourcing your auth to Octa, htmx may not be your thing.
  
  The type of app is going to be a major consideration when deciding if htmx is an appropriate choice. If you are writing the next Google Sheets, htmx is not going to be able to do that, you need the raw power of JS. If your bread and butter is commercial CRUD apps and you want to make them quicker, avoid page flashes, and have modern UI magic such as search results that update as you type, then htmx is going to be your jam.
  
  It’s not an unrealistic dream that the functionality in htmx becomes part of the HTML specification. The book sets out some good arguments for it, and the htmx implementation of that shows how possible and appealing it is. Whether it does or doesn’t, I expect to be using a lot in the future.

Adding Front Matter To mdserver

Fri, 24 Nov 2023 00:00:00 +0000

The very first issue I opened on mdserver - my server project that serves HTML from markdown files - was that the title of the page (which shows in the browser tab, and is used for browser bookmarks) needed to be set inside the markdown file, rather than generated from the file name. I didn’t invent this idea - I’ve seen this sort of metadata in the top of Jekyll and Hugo markdown. Here’s an example from the Jekyll website:

---
layout: post
title: Blogging Like a Hacker
---

You can’t really see in this example, but the format is YAML. Although I might be interested in using it for other things (such as selecting a template) later, for now, all I need is a title. The process would be that the server would extract the title from the front matter, then inject that into the template HTML so the page had a proper title.

I’m using the Showdown library to do the conversion from markdown. Here’s a short demo of how that works:

const showdown = require('showdown');
const converter = new showdown.Converter();

const markdown = `
# heading
Some random Text
* list item
* another`

const rawHtml = converter.makeHtml(markdown);

console.log(rawHtml);

This would output:

<h1 id="heading">heading</h1>
<p>Some random Text</p>
<ul>
<li>list item</li>
<li>another</li>
</ul>

Showdown is an 827K dependency, so I figured it might already deal with front matter, or would at least have some sort of extension hooks so I could write something to scrape the title out. In fact it has both.

To enable front matter, you just have to set a flag in the converter, then there’s a .getMetadata() method on the converter to get an object of all the metadata. Let’s flesh out my demo code a bit to show this, I’ll highlight the changes.

const showdown = require('showdown');
const converter = new showdown.Converter({metadata: true});

const markdown = `
---
title: Test Title
---
# heading
Some random Text
* list item
* another`

const rawHtml = converter.makeHtml(markdown);

//console.log(rawHtml);
console.log(converter.getMetadata().title);

This simply outputs Test Title.

If you’re wondering if that YAML pollutes the HTML output at all, it does not. The HTML from this second example is exactly the same as the first example above without the YAML.

Webdev on blog.iankulin.com

End to end testing - Cypress basics

E2E

How it looks

The App

Installing Cypress

The Tests

.should()

.get()

Attributes

User Sessions & Cookies in Node

The problem we’re addressing

Code example

Persisting across server restarts

Users

Log out

Users & view counts

Creating a user

Logging a user in

Showing the view count

Authentication

Passwords

Forms

Sanitising inputs

HTTPS

Other security

Conclusion

React Expense Tracker App

The Specification

Design Decisions

Code

Expenses List

Mosh

CSS for React Components

Old Style Global

Old Style local

CSS & Component Libraries

component.css

Modules

Styled Components

React - a To Do Example

Overview

Tooling

Components

Anatomy of a component

Conclusion

htmx - A To Do Example

htmx

In practice

Traditional Version

htmx Version

index.html

server.js

Reflections

Adding Front Matter To mdserver