Security on blog.iankulin.com

Share files securely with Enclosed

Mon, 27 Jan 2025 00:00:00 +0000

My accountant works for one of those giant firms, and it bugs me that I’m emailing him password protected zip files of my accounts rather than to a secure upload facility at his firm. I can fix this with the power of self hosting, by running my own secure file dropping app on a VPS.

There’s a number of applications that do this sort of thing - allow you to upload a file, get a link in return which you can then share to people to download the file. For this to be more secure than emailing, the file needs to be encrypted on the server, and we want to be able to set a password, impose limits on downloads, and limit how long the link lives for. I’ve previously looked at Sharry which adds the ability for unauthenticated users to upload files to you securely, but for this slightly simpler job, I chose Enclosed by Corentin Thomasset.

The docs provide a simple compose file to get going docker. Mine is slightly more complex because it’s proxy-ed with Nginx Proxy Manager, so it needs to share it’s network.

services:
 enclosed:
 container_name: enclosed
 image: corentinth/enclosed
 restart: unless-stopped
 networks:
 - nginx-proxy-manager_default

networks:
 nginx-proxy-manager_default:
 external: true

Authentication

What’s not well explained in the docs is how to set up authenticated login. By default, if you throw this up on a VPS, the entire world can use it to share their files. What I’d like is that I log in to share a file, but the person I send the link to can download the file without logging in. This is easy to do, we just need to add a couple of environment variables to our compose file. I always like to keep my secrets in an .env file since I source control all my home-lab and VPS setups, and I don’t want the secrets in source control.

Here’s a sample .env file. This just goes in the same directory as our docker-compose.yml

AUTHENTICATION_USERS=example@example.com:$$2a$$10$$n4StEr5Tcat7jItq
PUBLIC_IS_AUTHENTICATION_REQUIRED=true

The AUTHENTICATION_USERS string is just the username and a bcrypt salted/hashed password. You don’t need to do anything hard to create this, the project kindly provides a tool for it.

The tool includes an option for escaping the ‘$’ character correctly for docker compose files (hence the double $ in the string above.

To use this .env file, we pull in the values in the docker-compose thus:

services:
 enclosed:
 container_name: enclosed
 image: corentinth/enclosed
 environment:
 - AUTHENTICATION_USERS=${AUTHENTICATION_USERS}
 - PUBLIC_IS_AUTHENTICATION_REQUIRED=${PUBLIC_IS_AUTHENTICATION_REQUIRED}
 restart: unless-stopped
 networks:
 - nginx-proxy-manager_default

networks:
 nginx-proxy-manager_default:
 external: true

Once that’s running, the user name and password will required to upload files or write notes. The interface is clean and self-explanatory:

npm ERR! Exit handler never called!

Mon, 21 Oct 2024 00:00:00 +0000

I quite like GitHub scanning all my code and sending me security advisories. Here’s today’s:

With these, and my dependabot alerts, fixing them is usually just a matter of pulling down the project, running an npm update, building any artifacts, then pushing it back up. But today, not so:

package-lock.json

It’s probably worth revisiting what the package-lock.json does. It contains all the versions of any packages you’ve imported, and their dependencies. The idea is that this will make the build reproducible. We don’t commit the node_modules folder (that actually contains all that package code), but npm can reproduce it exactly by using the version information in the package-lock.json file. Here’s a snippet where you can see all those versions:

 "node_modules/body-parser": {
 "version": "1.20.2",
 "resolved": "https://registry.npmjs.org/body-parser/-/body-parser-1.20.2.tgz",
 "integrity": "sha512-ml9pReCu3M61kGlqoTm2umSXTlRTuGTx0bfYj+uIUKKYycG5NtSbeetV3faSU6R7ajOPw0g/J1PvK4qNy7s5bA==",
 "dependencies": {
 "bytes": "3.1.2",
 "content-type": "~1.0.5",
 "debug": "2.6.9",
 "depd": "2.0.0",
 "destroy": "1.2.0",
 "http-errors": "2.0.0",
 "iconv-lite": "0.4.24",
 "on-finished": "2.4.1",
 "qs": "6.11.0",
 "raw-body": "2.5.2",
 "type-is": "~1.6.18",
 "unpipe": "1.0.0"
 },
 "engines": {
 "node": ">= 0.8",
 "npm": "1.2.8000 || >= 1.4.16"
 }
 },

For me, I don’t really care that I’m using “iconv-lite” version 0.4.24, but if I’m working on a project with someone else, it might be important that we’re using the same version so we’re not chasing our tails trying to sort out a bug.

npm update

There are some rules about how the versions of packages are entered in package.json; when we run npm update, it uses those rules to look in the npm registry to find the most recent version of all the packages it’s allowed. Then it updates them in package-lock.json, and downloads the code into the node_modules directory.

This is potentially a substantial change to your app, so you’d definitely want to be running your testing process again afterwards.

The Error

npm ERR! Exit handler never called!

npm ERR! This is an error with npm itself. Please report this error at:
npm ERR! <https://github.com/npm/cli/issues>

This sounds quite serious, but before you head off to report it, try this:

npm install --no-package-lock

This just runs the update ignoring the package-lock.json file - as if you’d just deleted it. If that works, it was a problem with the package-lock.json file, which in this context of just wanting all the latest versions we don’t care about. We do want to rebuild the package-lock.json file though, so go ahead and delete it and run npm install to create a nice new one.

Now your project will have a couple of version changes in those package files. You’ll need to redo all your testing and rebuild any Docker images etc, and then you’re all up to date and secure again!

SSH login notification

Mon, 13 May 2024 00:00:00 +0000

My VPS’s are usually locked down so just ports 80 & 443 (for web server) and 22 (for ssh) are open. That’s great for reducing the attack surface, but having ssh open is a potentially disastrous vulnerability. For this reason I often close that at the cloud firewall level as well, but it has to be open when I’m making changes or running the weekly ansible update/cleanup playbooks.

To make things a bit safer, I run Fail2Ban on the ssh logs, and also have notifications turned on via Ntfy. Ntfy is so useful I make an annual donation to support it’s development and help with Phil’s server costs. I recommend you do to. In fact, my setup for getting a notification on my watch everytime someone ssh’s into one of my VPS’s is just copied directly from Phil’s examples.

Changes

If you’re not logged in as root (that should be turned off) you’ll need to run all these as sudo.

Edit /etc/pam.d/sshd to add these lines to the bottom:

# at the end of the file
session optional pam_exec.so /usr/bin/ntfy-ssh-login.sh

Then create the file /usr/bin/ntfy-ssh-login.sh

#!/bin/bash
if [ "${PAM_TYPE}" = "open_session" ]; then
 curl \
 -H prio:high \
 -H tags:warning \
 -d "SSH login: ${PAM_USER} from ${PAM_RHOST}" \
 ntfy.sh/your-unique-notification-string
fi

but replace your-unique-notification-string with whatever string you’re monitoring with the app. Note that if you’re using the shared (rather than self hosted) service, these are public. If I’d used mine here, you’d be able to use it to spam my phone with alerts. For this reason, many people use GUIDs.

We need to make this executable:

chmod +x /usr/bin/ntfy-ssh-login.sh

And restart the ssh daemon

sudo systemctl restart sshd

Then log out, and ssh back in, and you should get the notification.

Due Diligence on a Docker Image

Mon, 08 Apr 2024 00:00:00 +0000

Photo by Brett Jordan on Unsplash

I need a survey tool, and a quick search turned up LimeSurvey, there’s a ‘community edition’ so naturally I plan to self-host it. I scrolled down to the ‘installation’ section of the manual which has a big list of PHP dependencies.

Ain’t nobody got the time for that in 2024, I scroll further looking for the docker-compose but there isn’t one. Huh. No official Docker image.

Making my own docker image will be a little bit more work than just the pain of installing it manually and writing the Ansible playbook, but almost certainly someone else will have done that for us, so pop over to Docker Hub to have a look.

We’re going to want the ’trusted content’ right?

Who do you trust?

At some stage when you run software, you are going to need to decide who you trust for this application. You might say “no, as an Open Source advocate, I’m going to read through the code and compile it myself” - and that’s going to be a legit approach in some circumstances. But of course you’re still choosing to trust all the libraries it uses, the compiler, the operating system, and the computer chipset (or worse - your hosting provider).

If you want to go hardcore secure, you’ll eventually find yourself constructing your own [fab](https://en.wikipedia.org/wiki/Semiconductor_fabrication_plant). This is the concept of the Security-Convenience trade-off. More security generally equals less convenience and vice versa.

To make a sensible decision about this when choosing software I think about the threat profile, and have some rough metrics about what makes me more comfortable or less comfortable about a software artifact.

The reason this is a front-of-mind issue for me is that software packaging is an easy stage for bad actors to insert their code. We’ve seen a bit of that over the past month in the Canonical Snap store. Nefarious bitcoin stealers were packaging their software as legit, users were downloading them and installing them and one cryptobro lost a heap of money. It’s easy to imagine lots of related exploits - for example just altering the original code and packaging it. In the case of LimeSurvey (which is used by many researchers at leading universities perhaps a bad actor might want to be able to run bots from inside verifiable uni IP addresses.

Trust Factors

What are the things that might reduce our anxiety about the software bill of materials we’re dealing with? These are just mine - a complete non-expert:

Trusted organisation - If the Ubuntu team have signed off on something, I’m going to trust it more that some other group I never heard of till today.
Popular - it’s not just it’s reassuring that other people have decided it’s trustworthy, it’s also a comforting idea that if there is an exploit, it’s more likely to be discovered, and to hurt someone else first. If there’s a zero day exploit in MacOS I’ll hear about it on Mastodon and be able to get some advice about a work-around or fix.
Updated - a project under current development is more likely to be applying updates to any compromised libraries, and there’s probably a community of some sort where security concerns are considered.
Verifiable - One of the big strengths of Open Source. Even if I don’t have the skills or time to read the code, there’s a good chance that other people are. And in the case of a container image, I actually do sort of have the skills to read the dockerfile and know what’s gone into it.

Choosing an Image

Let’s apply some of these rules-of-thumb to my LimeSurvey container needs.

When Docker Hub says something is “trusted content” they’ve already done some of my work for me. And when I click through to eucm/limesurvey it says that the developer (eucm) is a “Sponsored OSS” which means a human at Docker thinks this developer group are legit. These are all encouraging factors.

What’s less encouraging is “Pulls 635” which does not seem a lot, and “Updated over 3 years ago”. Also I’ve got no reason to think this is an official image - eucm doesn’t seem to have any connection to the LimeSurvey developers. Let’s look at the other images.

There’s more further down but all with about 100K pulls or less. Of these four, we’ve already eliminated the top one, and we can probably eliminate the bottom one based on it’s age - although it’s always interesting to see an image with so many pulls and no updates since that sometimes happens when a project changes hands or gets relisted under a different owner.

Either of the other two (acspri/limesurvey & martialblog/limesurvey) are worth investigating - there’s nothing much to tell them apart in this view since they both have about the same number of stars (the number next to the pulls number). I’ll start at the top.

It’s promising to have a good readme, including a docker-compose example, but there’s no github link, and I really want a peek at the dockerfile. If I click through to the developer, they seem to be the Australia Consortium for Social and Political Research Inc - which it makes sense why they would be interested in a survey tool. Additionally, they seem to have an official link to the project. They do have an active GitHub account, but it doesn’t include a repository for this which seems, odd.

However, there is this repo adamzammit / limesurvey-docker which appears to get pushed to it’s own dockerhub as well as the ACSPRI one. There is a note on the adamzamit dockerhub saying this used to be the acspri but has been moved here - that would be more convincing if the note was on the acspri dockerhub. The github looks legit, and there was nothing suspicious (to my inexpert view) in the dockerfile.

The martialblog/limesurvey one also looks good - recently updated, lots of pulls, a comprehensive readme, and a github link right at the top. The only criteria it doesn’t meet is “trusted organisation” but the link to an active github goes part of the way. I’d be happy to use this container for the purpose I’ve got in mind.

Quick && Dirty auth with nginx & Node

Fri, 23 Feb 2024 00:00:00 +0000

One of the basic requirements for any serious web app is a proper users/roles/authentication system - but if you’re just throwing up a utility of some kind on a public IP for testing, and you don’t want it to be abused, then this could be an option. There’s a few components:

Your app. In this demo it’s going to be Node, but it could be Go or whatever your server-side poison is. The app is listening for connections on a non-web port (ie not on 80 or 443), I’m going to use the traditional 3000.
A firewall. That port (in my example 3000) must not be accessible from the internet. It has to be blocked by a firewall.
A web server (I’m using nginx) that enforces basic auth.

I briefly discussed web server basic auth earlier - it’s a system built into the web server that requires a log in for a route, and authenticates it against the credentials in a password file (usually named .htpasswrd) and only serves the content if authenticated.

We’re going to complicate that a bit by then inserting the authenticated user name into a header, so that we can access it in our node app. The web server does this as it passes the incoming request to our app in a process called proxy-ing.

Prerequisites

You’re going to need a server, separate to the machine you’re using. I’m going to use an LXC container on one of my Proxmox servers, but perhaps you’re on windows and have a WSL to play with, or you’ve perhaps you’ve spun up a baby server on Hetzner, Linode or Digital Ocean. What ever floats your boat. You need to be able to set it up and ssh into it to follow along.

All my examples are assuming Debian, so that or a Debian based distro like Ubuntu is going to be simplest, but if you’re on something with a different package management system, you’re probably able to translate things to that.

Install nginx

To install nginx, we just

sudo apt install nginx

Now if we open the server ip address, we should see the nginx test page:

If you’re wondering where this page comes from, it’s /var/www/html/index.nginx-debian.html. There’s a default nginx site config at /etc/nginx/sites-available/default that points to it. We’ll be playing in there later.

Installing Node

sudo apt install nodejssudo apt install npm

This is going to install the version of node and npm that are provided by Debian or the Debian related distro you’re using, so they won’t be the latest and greatest, but they will be stable and bug patched to whatever level your distro maintainers think they should be. You could check with node -v and npm -v if you were interested, but we’re not using any bleeding edge features here, so whatever you’ve got it should be fine. For reverence, I have node v18.19.0, and npm 9.2.0

The App

We’re going to create a very basic node/Express server app to run on our server. I’m going to remote in with VS Code because that’s how I roll this week, but do this however you want. Nano is fine, or maybe you’re a vim person. Perhaps for these examples we’ll assume you’re a sane person near the start of their dev journey and use nano. ssh to the server, then:

mkdir appcd appnpm initnpm install expressnano app.js

Then, our app code in app.js

const express = require('express');const app = express();const port = 3000;app.get('/', (req, res) => { res.send('Hello World');});app.listen(port, () => { console.log(`Server is listening at http://localhost:${port}`);});

If we’ve done everything right, once you’ve saved that (ctl-O, ctl-X) if we run node app.js we’ll get the message Server is listening at http://localhost:3000 and visiting the IP address of our server with :3000 on the end should get this result:

The Firewall

Firewalls are their own big thing that I should write about another time. Suffice to say we’re going to make it so outside traffic can’t access our app on port 3000 (so we can force them to go through nginx where we authenticate them).

sudo apt-get install netfilter-persistentsudo iptables -A INPUT -p tcp --dport 3000 -j DROPsudo netfilter-persistent savesudo netfilter-persistent reload

Now if you start the app again with node app.js and visit :3000 in the browser, it should eventually just time out because the request is never making it to our app.

Proxy Pass

So now that raw access from the network to our app is blocked off, we want to configure nginx to pass any requests to our app. There’s a number of good reasons why you should put a web server in front of you apps, but today we’re doing it so we can authenticate the users. We’ll get to that, but for the moment, we need to edit /etc/nginx/sites-available/default

Scroll down till you see the location / { block. Delete out the contents and replace it with

proxy_pass http://localhost:3000;

Then we’ll check the configuration is okay, and restart the nginx server.

sudo nginx -tsudo service nginx restart

Now if our app is running (node app.js) you should be able to go to the server address (without the :3000) and see the app working again.

Credentials

Now we need to create a file with our credentials, so nginx can have something to check against. The first web server that I ever used that did this was Apache, and that format has carried forward to be used by nginx. I’m mentioning this to explain why I’m about to tell you to install some Apache tools.

sudo apt install apache2-utilssudo htpasswd -c /etc/nginx/.htpasswd user1

This second command is creating (that’s the -c flag) a text file called .htpasswd in the /etc/nginx directory. It doesn’t matter that much what it’s called or where it is - we’re going to specify that later in the nginx conf, but I like to put it somewhere I’d probably guess later.

user1 is just what I’ve called this user - it could of course be just about anything. htpasswd will ask you to enter a password for this user, and confirm it.

If you’re curious about how that looks in the file, you can just cat it out. You won’t see the plaintext password, it’s been hashed into gooblygook.

If you want to add more users, go ahead; it’s the same command without the -c

sudo htpasswd /etc/nginx/.htpasswd ian

Next, we need to tell nginx to use this. We need to go back to the same spot in the /etc/nginx/sites-available/default where we added the proxy pass statement. Just above the proxy statement, add:

auth_basic "Protected app";auth_basic_user_file /etc/nginx/.htpasswd;

“Protected app” is the explanation that should pop up in the modal, and the other directive just tells nginx where to look for the credentials.

I’m pretty sure nginx processes these in order, so put the auth_basic directives before the proxy_pass.

Once that’s saved, we’ll check the configuration and restart nginx to load it.

ian@ct372-authplay:~$ sudo nginx -tnginx: the configuration file /etc/nginx/nginx.conf syntax is oknginx: configuration file /etc/nginx/nginx.conf test is successfulian@ct372-authplay:~$ sudo service nginx restart

If we go back to the page, it should pop up and ask for the credentials. If you input your credentials it will direct you to the “hello world” message from our app.

Accessing the user in node

That’s all great, but how do we access the authenticated user in our app so we know what content to serve? Nginx knows the username, but our node app does not. To fix that, nginx needs to put it in the header passed to the app. To do this, we need to edit the nginx conf file again to add:

proxy_set_header X-Username $remote_user;

This takes the user name (in remote_user) and inserts it to the request header.

After making this change, we need to restart nginx to pick up the config change again - sudo service nginx restart

Back in our node app, we need to recover the username from the request header.

app.get('/', (req, res) => { const username = req.get('X-Username'); res.send('Hello '+username);});

In the example above I’ve extracted the username in the route - often in my apps I do that in middleware and use it to set some request variables with allowed roles and so on.

Limitations

This is not a sophisticated system, here are some shortcomings:

The most dangerous thing (although I guess this applies to any auth) is that if you’re not securing the web traffic with SSL, the password is transmitted in plaintext across the internet.
There’s no simple way to logout or change the user.
I entered wrong credentials about twenty times as fast as I could and it never stopped me trying, so a brute force is possible. There are ways of addressing this that I haven’t covered here.

All in all, this is a handy tool that doesn’t require a lot of libraries or setup. It is very simple and doesn’t provide any fancy functionality like password resets, but sometimes it’s all you need.

Beginning Node App Security

Fri, 16 Feb 2024 00:00:00 +0000

Since I’m using Tailscale to painlessly manage all my networking on the homeserver here and my remotes, I’ve had the luxury of being a bit casual about the security of my internal apps and self hosted dev tools. I’m currently iterating on a web app that requires public access, and is therefore up on a VPS and exposed to all the evils of the open internet.

I am in no way a security expert, but here’s a few of the (reasonably simple) steps I’ve taken to secure my node app.

Put it behind Nginx

I could just change the port number of the Node app to listen to port 80 and connect it directly to the world, but Nginx is battle hardened for outward facing tasks so that seems safer. Additionally, it opens up a lot of functionality such as putting my app on a subdomain and some other security options we’ll come to. Putting Nginx in front of your app like this is called ‘reverse proxying’.

	server_name sub.example.com;	location / { 		proxy_pass http://localhost:3000;			proxy_http_version 1.1;		proxy_set_header Upgrade $http_upgrade;		proxy_set_header Connection 'upgrade';		proxy_set_header Host $host;		proxy_cache_bypass $http_upgrade;	}

Basic Auth

One of the Nginx options is the ability to turn on ‘basic auth’. This can be enabled for a route, subdomain, or a whole domain. It forces a user to authenticate before being able to access resources from that area. It’s basic in the sense that the password is manually set on the server in a file that the nginx conf points to. Ideally your app will have comprehensive auth built in, but (especially when you are still developing it) basic auth is a quick and easy way to prevent all of the internet from accessing your app.

	server_name sub.example.com;	location / {		auth_basic "Secure app";	 auth_basic_user_file /etc/nginx/.htpasswd;	}

HTTPS

Enforcing SSL connections is much easier than it used to be (thank you Let’s Encrypt and Certbot) and it keeps all the data being sent between your app and the user’s browser encrypted - including the username and password you are using for your auth.

Logging

Ensuring that logs are turned on means that you’ve got some chance of detecting problems and possible attacks. In fact, you’ll probably be aghast at the number of bots that start accessing your server as soon as it’s live. For the most part, they are probing for the existence of known vulnerabilities in well known packages - may of the php. When I look up the IP addresses for these, they almost always come from China or Eastern Europe.

Note that logging does involve some future maintenance. Logs need rotated and deleted or we’ll soon be running out of disk space.

Fail2ban

Manually checking the logs is not effective, we need to automate this a bit. The things I’m looking for in my system is brute force attempts at breaking the basic auth, and the same with SSH.

Fail2Ban can automate this (and many other things, but I’m just using these two) by scanning the logs for failed attempts. There are various settings to determine the thresholds - say check for 5 failed attempts in 10 minutes then ban the IP address for 30 minutes. Once the threshold is met, Fail2Ban updates iptables (the internal firewall) to block them.

ian@novel-ironic:~$ sudo fail2ban-client status sshdStatus for the jail: sshd|- Filter| |- Currently failed:	1| |- Total failed:	2960| `- File list:	/var/log/auth.log`- Actions |- Currently banned:	6 |- Total banned:	483 `- Banned IP list:	218.92.0.27 61.177.172.136 61.177.172.140 218.92.0.113 218.92.0.31 218.92.0.76ian@novel-ironic:~$ sudo fail2ban-client status nginx-http-authStatus for the jail: nginx-http-auth|- Filter| |- Currently failed:	1| |- Total failed:	6| `- File list:	/var/log/nginx/error.log`- Actions |- Currently banned:	0 |- Total banned:	1 `- Banned IP list:	ian@novel-ironic:~$

In the output above, you can see there are 6 ip addresses currently blocked for trying to crack SSH, and there’s been 483 banned in the couple of days since I turned it on - so this is a very common attack vector. The basic auth one just has a single ban (from when I tested it). I’m not sure why I like looking at the list of bans so much, but I do!

Cloud Firewall

Many VPS providers will have a cloud firewall (although they may call it something else). We can use this to lock down all the ports we are not using to massively reduce the attack surface for this machine. One of the very appealing things about this firewall which is external to the VPS is that it’s accessed via the VPS provider web interface - so it’s not possible to lock yourself out if you make a mistake - as opposed to when you’re SSHd in and fiddling with iptables.

Since this VPS is just running web apps, I just have ports 80, 443 and 22 open.

By default, Ubuntu does not allow root login by password, but once I’ve added a new user and added them to the sudo group, I turn it off entirely. Most of those SSH attempts that failed would have been trying common user names including root, so may as well take it out of the possibilities.

SSH keys

Apart from being more convenient, well managed SSH keys are much safer than using passwords. So my new user copies up their keys and I set that user to no login with password as well.

Updates

One of the wonderful things about open source and the modern web, is that as new vulnerabilities are being discovered, they are being patched. We only get them if we run updates though. It’s possible (and recommended) to use automatic updates, but I have a weekend ansible routine to do them that I like to look at the output of to check everything’s healthy.

Monitoring

I use a very simple monitoring system for all the VM’s and containers in my Tailnet - just checking the root disk space and available memory. This is exposed as an http endpoint, then checked by Uptime Kuma. That’s better than nothing, but for a production system not really enough. This is one of the areas I’ll revisit in the future.

User Sessions & Cookies in Node

Fri, 09 Feb 2024 00:00:00 +0000

When you are learning app development, you can create all sorts of apps that work for you, but for any serious app, it’s going to need to authenticate users and persist sessions across visits. So much so, that as a professional developer, you’ll probably build that out first - it becomes a sort of boiler plate you always drop in.

In this post, focusing on the server side, using node, express, and particularly express-session, I’ll try and build up from nothing to a reasonable usable user login system explaining the increasing complexity and reasons for it. To follow along you’ll need basic familiarity with node and express.

The problem we’re addressing

For most web applications, we need to persist state per user. For example, if you go to a drawing app and start a drawing, you want it to be there when you come back to the app. Additionally, you don’t want to come back to someone else’s half-drawn app, or, have them drawing over your picture. What we really want is something like this:

User 1 sees their picture of the star, and User 2 sees their picture of a heart.

Since HTTP is stateless - a request to /picture from one user is indistinguishable from another users request to /picture - so we need to add something to allow the server to distinguish between the two. The something would be a bit of state that the server passes back to the user, then the user sends it in with their actions so the server can identify them.

There are a few of ways to do this. The first (which is not the subject of this post) is to store that in the URL. For example, when a user in the above app requests to create a picture, we could generate a GUID for them, then redirect them to a URL based on that - perhaps /picture/cbe34f. Thereafter, all their requests could include that GUID. This can be a useful way of managing session state and has some affordances that the other method does not, but it’s not the most common.

Another system that was extensively used in the early days of the web was to embed a hidden input with the GUID in the HTML returned to the user. When the user submitted the form later, the GUID was available

The most common method is for the server to create a bit of state (our GUID), send it to the user and have the user’s browser store it, and return it with every request. You will know this bit of state as a cookie.

Code example

Enough theory, lets look at some code. If you google ‘simple node express session’ you’ll find this, or something almost identical. Instead of the state we’re trying to persist being a picture of a heart or a star, we’re trying to remember how many times each user has visited our web site. Note these are separate counts for each user - a new visitor will start at zero, then count up each time they come back or refresh their page.

We’re using a couple of packages, so to run this example, you’ll first need to install them with npm i express express-session

const express = require('express');
const session = require('express-session');

const app = express();

app.use(session({
 secret: 'your-secret-key', 
 resave: false,
 saveUninitialized: true
}));

app.get('/', (req, res) => {
 // Access session data
 if (req.session.views) {
 req.session.views++;
 } else {
 req.session.views = 1;
 }

 res.send(`Views: ${req.session.views}`);
});

app.listen(3000, () => {
 console.log('Server is running on port 3000');
});

If we load this page from http://localhost:3000 it says “Views: 1”. Reloading the page it will say “Views: 2”. If I open a different browser and visit the server, we’ll be back to “Views: 1”. So what’s the magic that’s happening here?

On the first request from a browser it hasn’t seen before, express-session is generating a session ID, and sending that back to the browser (along with ‘Views: 1’) and asking the browser to store it as a cookie. Thereafter, every request to this website http://localhost:3000 will include the cookie containing the session ID. The number of views is being stored on the server in memory. Express-session does the work of looking up the number of views for a particular session ID returned in a cookie so we have the luxury of just grabbing that from rec.session.views

Thanks to the magic of browser development tools, we can have a look in the request header from the browser and see the cookie with it’s session ID contents:

It’s also possible to go and find the cookie your browser has stored. Again, this is easiest in the developer tools - look under ‘Storage’:

Currently, we’re only storing the links between session_ids and the number of views in memory in the server, so if the server restarts, we’ll lose them, and everyone will go back to ‘Views: 1’.

Persisting across server restarts

If we want our view counts to not be reset to zero each time the server restarts, we’ll need to save them somehow. express-session doesn’t let us access its internal array of sessions, but it does provide a mechanism for that access with the concept of stores. Usually we’d keep the sessions in a database, but as files is a simpler solution for a blog post. There is a package called session-file-store that will slot into expression-session that will just use the file system to persist the session-view count key pairs for us. After installing it with npm i session-file-store, we just declare a const for it, and pass it in when initialising the session middleware.

const express = require("express");
const session = require("express-session");
const FileStore = require("session-file-store")(session);

const app = express();

app.use(
 session({
 secret: "your-secret-key", // This should be a secret, used to sign the session ID cookie
 resave: false,
 saveUninitialized: true,
 store: new FileStore,
 name: "session_cookie",
 })
);

app.get("/", (req, res) => {
 // Access session data
 if (req.session.views) {
 req.session.views++;
 } else {
 req.session.views = 1;
 }

 res.send(`Views: ${req.session.views}`);
});

app.listen(3000, () => {
 console.log("Server is running on port 3000");
});

Now view counts for each browser are persisted even when the server is re-started. If we look inside one of the files we should see a view_count field:

The name of each file is the session id, but these don’t match the session id’s you see in the browser cookies - presumably because they’ve been encrypted by the secret we used when establishing the express-session.

So this is a better experience - our users see their own view counts increasing on each refresh, and the view counts don’t get lost when you take the server down for maintenance. But what happens if the user wants to pick up their phone and check their view count. It’s a different browser without the cookie containing the session id, so they’ll get ‘View: 1’ again.

Or, what if your partner sits down at your laptop to check their view count, expecting it to be ‘1’ because they’ve never visited this page, and they see your view count instead?

Users

Our current system is really based around browser instances, but we want it to be about people. So a better system, and one that you’ll be used to is the concept of logging in as a user. This can solve both the problems we described above - users will be able to log in from any browser and see only their own data.

This is going to take things up a substantial step in complexity because now instead of two bits of data (the cookie with the session_id, and the session store linking the session_id and the view count) there is going to be:

The cookie with the session_id
The session store linking the session_id and the user
A new store we will have to manage that links the user and the view_count

Also, we’ll need:

Some mechanism to create a user
Some mechanism to log in
And log out
If there’s no user logged in, redirect to the log in page

Log out

Logging the user out is reasonably simple - we just delete the session information. In a real app we’d have a ’log out’ button of some sort, but for simplicity here, I’m just going to add a ’log out’ route.

app.get("/logout", (req, res) => { req.session.destroy((err) => { if (err) { return console.log(err); } res.clearCookie("session_cookie"); res.redirect("/"); });});

clearCookie() tells the browser to delete the cookie - which just saves us from console messages later when the browser sends it, but express-session can’t find the matching session.

If you add this to the code from earlier and run it, view will count up as before, but when you visit the /logout endpoint views will be set back to 1.

Users & view counts

In a real application we’d be keeping this in a database, but again for simplicity of this demo, I’m just going to keep an array of objects and persist them as a text file. The array will be user_views, and somewhere at the top of our code we’ll add:

let user_views = [];// if the user_views.json file exists, read it and parse it to user_views arrayif (fs.existsSync("./user_views.json")) { user_views = JSON.parse(fs.readFileSync("./user_views.json"));}

Creating a user

We’ll use a route parameter to create a new user (like /create/jane or /create/robert where the second part is accessible in req.params). That user will be set as the session user, then we’ll add it to our array and save the array to disk.

app.get("/create/:user", (req, res) => { const user = req.params.user; const views = 0; req.session.user = user; user_views.push({ user, views }); fs.writeFile("./user_views.json", JSON.stringify(user_views), (err) => { if (err) { res.send(err); return; } }); res.send(`User ${user} created & logged in`);});

Logging a user in

If the user is in our array, we’ll set the session user to it, otherwise redirect to the create endpoint:

app.get("/login/:user", (req, res) => { // see if this user is in the user_views array if (user_views.find((u) => u.user === req.params.user)) { req.session.user = req.params.user; res.send(`User ${req.params.user} logged in`); return; } else { res.redirect(`/create/${req.params.user}`); }});

Showing the view count

The default route that shows our view count has a few jobs to do:

Check we’re logged in, if not, tell the user to log in
If we are logged in, fetch the view count for that user
Increment the view count
Show it to the user
Re-save the user_views data since we’ve mutated it

app.get("/", (req, res) => { if (req.session.user) { const user_view = user_views.find((u) => u.user === req.session.user); user_view.views++; res.send(`User "${req.session.user}" has ${user_view.views} views`); writeUserViewFile(); } else { res.send("Please log in"); }});

So that’s our system for associating the view counts with a user done. Here’s the whole thing where we’re up to (I’ve done a little refactoring).

const express = require("express");
const session = require("express-session");
const FileStore = require("session-file-store")(session);
const fs = require("fs");
const app = express();

const user_view_file = "./user_views.json";
const cookie_name = "session_cookie";

let user_views = [];
// if the user_views.json file exists, read it and parse it to user_views array
if (fs.existsSync(user_view_file)) {
 user_views = JSON.parse(fs.readFileSync(user_view_file));
}

function writeUserViewFile() {
 fs.writeFile(user_view_file, JSON.stringify(user_views), (err) => {
 if (err) {
 console.log(err);
 return;
 }
 });
}

function findUser(user) {
 return user_views.find((u) => u.user === user);
}

app.use(
 session({
 secret: "your-secret-keyz", // This should be a secret, used to sign the session ID cookie
 resave: false,
 saveUninitialized: true,
 store: new FileStore(),
 name: cookie_name,
 })
);

app.get("/", (req, res) => {
 if (req.session.user) {
 const user_view = findUser(req.session.user);
 user_view.views++;
 res.send(`User "${req.session.user}" has ${user_view.views} views`);
 writeUserViewFile();
 } else {
 res.send("Please log in");
 }
});

app.get("/logout", (req, res) => {
 req.session.destroy((err) => {
 if (err) {
 return console.log(err);
 }
 res.clearCookie(cookie_name);
 res.redirect("/");
 });
});

app.get("/create/:user", (req, res) => {
 const user = req.params.user;
 const views = 0;
 req.session.user = user;
 user_views.push({ user, views });
 writeUserViewFile();
 res.send(`User ${user} created & logged in`);
});

app.get("/login/:user", (req, res) => {
 // see if this user is in the user_views array
 if (findUser(req.params.user)) {
 req.session.user = req.params.user;
 res.send(`User ${req.params.user} logged in`);
 return;
 } else {
 res.redirect(`/create/${req.params.user}`);
 }
});

app.listen(3000, () => {
 console.log("Server is running on port 3000");
});

Authentication

It won’t have escaped your attention that our view counting app isn’t at all secure. At the moment, any one can log in as ‘jane’ and see her view count. The common way to address this is to also require a password.

I will eventually have to start serving some actual HTML, but for this next step I’ll stick to just using the routes. So our user interface will be this:

Logging in /login/<user>/<password>
- Check the user exists, and that this is the correct password
Creating a new user /create/<user>/<password>
- Save the new user and password to the file.

We’ll do create first:

app.get("/create/:user/:password", (req, res) => { const user = req.params.user; const password = req.params.password; const views = 0; req.session.user = user; user_views.push({ user, password, views }); writeUserViewFile(); res.send(`User ${user} created & logged in`);});

We should probably first check there’s not an existing user with the same name to avoid having two users and the second user never being able to log in.

app.get("/create/:user/:password", (req, res) => { if (findUser(req.params.user)) { res.send(`User ${req.params.user} already exists`); return; } const user = req.params.user; const password = req.params.password; const views = 0; req.session.user = user; user_views.push({ user, password, views }); writeUserViewFile(); res.send(`User ${user} created & logged in`);});

and logging in:

app.get("/login/:user/:password", (req, res) => { const user = findUser(req.params.user); if (user && user.password === req.params.password) { req.session.user = req.params.user; res.send(`User ${req.params.user} logged in`); return; } else { res.send(`Incorrect username or password`); }});

Although this is an improvement, clearly, it would a stretch to call this improved security. Here’s a few things that are jumping out at me, and I’m sure there’s some being missed:

We’re transmitting plaintext passwords over http
We have plaintext passwords in a URL - any intermediate networking that’s recording access logs will the logging our passwords
We’re storing plaintext passwords on our server in the user_views.json file. So when our server is breached, our users’ passwords will get sold on the dark web and they’ll be hacked if they’ve reused name/password combos.
Passwords are limited to characters that reliably work in URLs
Our passwords and user names may be open to injection attacks

So it seems like there is four jobs to do:

Encrypt the passwords
Don’t use URLs for passing this data
Sanitise our inputs
Enforce HTTPS

Passwords

Although I described this as ’encrypting’ that’s not exactly what we’re going to do. The current state of the art for this is to hash and salt passwords before storing them.

hash - turn the password into some gooblygook such that that the hashed version always comes out the same if you put the same password into it. Preferably the hash is always the same length regardless of the length of the input password.

salt - mix some random characters in to the the hashed password so that the combination of hashing and salting the password comes out different every time, even though you are putting in the same password as input to the process.

How this is going to work is that once we get the users password, we’ll hash and salt it, then save the result of that in our array (and eventually file). If someone has access to that file, it’s practically impossible for them to reverse engineer the password - even if they have a known password somewhere else in the file to work from. Then when a user attempts to log in, we’ll take the password they gave us and hash it, and we’ll ‘un-salt’ the password from the file and compare those two hashed versions of the passwords. If they are the same, then we know the passwords were also the same, and we can log this user in.

This is a fun area of computational science, so it’s somewhat tempting to write our own hash and salting functions. This is widely regarded as a Bad Idea. As long as you don’t run into supply chain problems, a proper library, written by cryptographic experts, subject to public scrutiny, that gets updates when vulnerabilities are discovered is always going to be more secure. Almost universally, the answer for JS developers is bcrypt. We’ll install that with npm i bcrypt

Here’s our create and login endpoints using bcrypt with the changes highlighted.

app.get("/create/:user/:password", (req, res) => { if (findUser(req.params.user)) { res.send(`User ${req.params.user} already exists`); return; } const user = req.params.user; const hash = bcrypt.hashSync(req.params.password, saltRounds); const views = 0; req.session.user = user;  user_views.push({ user, hash, views });  writeUserViewFile();  res.send(`User ${user} created & logged in`); });});app.get("/login/:user/:password", (req, res) => { const user = findUser(req.params.user); if (user && bcrypt.compareSync(req.params.password, user.hash)) { req.session.user = req.params.user; res.send(`User ${req.params.user} logged in`); return; } else { res.send(`Incorrect username or password`); }});

Forms

To move the passwords out of the URLs, we’ll present a simple forms to the user for creating and logging in. The log in page is the basic user name / password form you’ve built before. This is served from the app.get("/login") route.

And here’s the endpoint it posts to:

app.post("/login", (req, res) => { const user = findUser(req.body.username); if (user && bcrypt.compareSync(req.body.password, user.hash)) { req.session.user = req.body.username; req.session.save((err) => { if (err) { res.send('Cookie saving error, <a href="/login">try again</a>`'); } else { res.redirect("/"); } }); } else { res.send(`Incorrect username or password, <a href="/login">try again</a>`); }});

Normally express-session will deal with saving the session without us having to worry about, but I decided that a successful login should be followed to a re-direct to the view count page.

Since express-session normally does its saving at the end of a http response, and if we’re redirecting, that response hasn’t happened. There’s a bit more about that here (search for session save).

Apart from that, the changes are really just grabbing the user name and passwords from the form post body instead of the URL.

The /register form is the same as the log in, but with a second password field and some client side scripting to check the two passwords are the same. Processing the new user is very similar to the previous create route.

app.post("/register", (req, res) => { if (findUser(req.body.username)) { res.send(`User ${req.body.username} already exists`); return; } const user = req.body.username; const hash = bcrypt.hashSync(req.body.password, saltRounds); const views = 0; req.session.user = user; user_views.push({ user, hash, views }); writeUserViewFile(); req.session.save((err) => { if (err) { res.send('Cookie saving error, <a href="/login">try again</a>`'); } else { res.redirect("/"); } });});

Sanitising inputs

Cautious developers do not trust any input from users. There are numerous libraries to deal with cleaning it up which I’d recommend you consider, but for our case here let’s think through what the possibilities are:

password - the password is going to be hashed and salted before it is stored, and never used to build HTML. The only risk I can think of would be if a very long password might cause some sort of trouble - so we can just truncate it. Note we don’t even need to tell the user - if we truncate it when they register, and when they log in, they’ll still match.

user name - is stored in a json file, and is output to the user. In the future that output is likely to be HTML.

Here’s our users_view.json

[ { "user": "user1", "hash": "$2b$10$8Zf9LZWH78mWnSKjxKQxXe9TlPoqe7L3SOABcPHIUQ5Pq3jIbVQVm", "views": 16 }, { "user": "user2", "hash": "$2b$10$f/QAQ7we6Hh/hTx35LjfGeYtCY8aRG3ZqbJZqhEZRDXUqxkKCgPhq", "views": 14 }, { "user": "user3", "hash": "$2b$10$KBZe6orYpjG3JeIGhnLDCuyYQXxfVZYBjovGf3XIdU8rr6kXlNrLC", "views": 1 }]

The obvious attack here would be injection. For example a hacker might register with the user name:

user4", "role": "admin

That’s a smart try at privileged escalation. Even if we had an ‘admin’ role, it wouldn’t actually work though since any double quotes will be escaped by JSON.stringify(), but to be cautious, we can eliminate the possibility by just deleting any double quotes out.

Another injection possibility would be with HTML. Perhaps we will display the logged in user later inside a

, something like:

<div>User: user1</div>

Our hacker might try registering this as their user name as:

user1</div><script>alert('you've been hacked')</script></div>

It’s not great to allow anyone on the internet to run code in our visitors’ browsers.

You should really use a library designed by someone who knows what they are doing, but I just wanted to do enough here to prompt you to think about. For that demo purpose, I’m going to replace all " < and > with _

// sanitise a string by replacing all " < and > with _ (underscore) // and truncating it at 20 charactersfunction sanitise(str) { return str.replace(/["<>]/g, "_").slice(0, 20);}

This is all a bit doge. If we are going to have rules like this (and the other rules we should have about min lengths) we should be implementing them in the browser and on the backend, and there should be helpful prompting to the users to enable them to understand and correct their inputs. As I mentioned earlier, this is just to get you to think about it.

HTTPS

In the list of four security improvements we wanted to make, the last one was to enforce HTTPS. There’s two places to do this. One is that when we initialise our session at the top of the app, we can tell it we want ‘secure’ cookies. This setting means that cookies will not be sent over plain HTTP, but only the end-to-end encrypted HTTPS. Currently our cookies contain a user name:

{ "cookie": { "originalMaxAge": null, "expires": null, "httpOnly": true, "path": "/" }, "__lastAccess": 1706315491081, "user": "user1"}

Even though a user name isn’t a lot of information, it could still be critical. If there was rumors about your company being acquired and a hacker leaked that j_bezos had been looking at your view counts, it could have implications. To turn on secure cookies:

app.use( session({ secret: sessionSecret, resave: false, saveUninitialized: true, store: new FileStore(), name: cookie_name, cookie: { secure: true, httpOnly: true, }, }));

Note that your infrastructure needs to support HTTPS for this to be useful - if the cookies are not sent, this particular app is rendered useless.

But we want to enforce HTTPS anyway, because a bigger problem is that if we don’t someone could intercept the login form data and collect plaintext usernames and passwords.

I generally do this by running all web services behind NGINX as a proxy. Then the NGINX configs can be set to redirect all HTTP requests to HTTPS, and valid requests can be passed off to your app. Here’s the sort of thing you might have in a config.

server { listen 80; server_name viewcount.example.com; return 301 https://$host$request_uri;}server { listen 443 ssl; server_name viewcount.example.com; # SSL certificate configuration ssl_certificate /path/to/your/certificate.crt; ssl_certificate_key /path/to/your/private-key.key; # Additional SSL configuration, such as preferred protocols and ciphers, can be added here location / { # Your application configuration goes here # Proxy pass to your Node.js or other application server # Example for proxying to a Node.js server running on localhost:3000 proxy_pass http://localhost:3000; proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection 'upgrade'; proxy_set_header Host $host; proxy_cache_bypass $http_upgrade; }}

There are other security actions that can be taken at the NGINX level - things like rate limiting, blocking particular IP ranges, and access logging - all of which can be handy for protecting your endpoint from bad actors.

Other security

I’d normally have the cookie secret in a .env file that was being .gitignored. A good hacker hobby is to scan public code repos for API keys and other secrets.
Helmet.js is a sort of magic bullet for enforcing some security around request headers.
Probably a stack of other things - ask your senior dev.

Here’s our app with the changes we’ve discussed:

const express = require("express");
const session = require("express-session");
const FileStore = require("session-file-store")(session);
const fs = require("fs");
const app = express();
const bcrypt = require("bcrypt");
const saltRounds = 10;

const user_view_file = "./user_views.json";
const cookie_name = "session_cookie";

app.use(express.urlencoded({ extended: false }));

let user_views = [];
// if the user_views.json file exists, read it and parse it to user_views array
if (fs.existsSync(user_view_file)) {
 user_views = JSON.parse(fs.readFileSync(user_view_file));
}

function writeUserViewFile() {
 fs.writeFile(user_view_file, JSON.stringify(user_views), (err) => {
 if (err) {
 console.log(err);
 return;
 }
 });
}

function findUser(user) {
 return user_views.find((u) => u.user === user);
}

// sanitise a string by replacing all " < and > with _ (underscore)
// and truncating it at 20 characters
function sanitise(str) {
 return str.replace(/["<>]/g, "_").slice(0, 20);
}

app.use(
 session({
 secret: "your-secret-keyz", // This should be a secret, used to sign the session ID cookie
 resave: false,
 saveUninitialized: true,
 store: new FileStore(),
 name: cookie_name,
 })
);

app.get("/", (req, res) => {
 if (req.session.user) {
 const user_view = findUser(req.session.user);
 if (!user_view) {
 res.send(`Error finding user, <a href="/login">try again</a>`);
 return;
 }
 user_view.views++;
 res.send(
 `User "${req.session.user}" has ${user_view.views} views, <a href="/">reload</a> or <a href="/logout">logout</a>`
 );
 writeUserViewFile();
 } else {
 res.redirect("/login");
 }
});

app.get("/logout", (req, res) => {
 req.session.destroy((err) => {
 if (err) {
 return console.log(err);
 }
 res.clearCookie(cookie_name);
 res.redirect("/");
 });
});

app.post("/register", (req, res) => {
 const user = sanitise(req.body.username);
 if (findUser(user)) {
 res.send(`User ${req.body.username} already exists`);
 return;
 }
 const hash = bcrypt.hashSync(req.body.password, saltRounds);
 const views = 0;
 req.session.user = user;
 user_views.push({ user, hash, views });
 writeUserViewFile();
 req.session.save((err) => {
 if (err) {
 res.send('Cookie saving error, <a href="/login">try again</a>`');
 } else {
 res.redirect("/");
 }
 });
});

app.post("/login", (req, res) => {
 const username = sanitise(req.body.username);
 const user = findUser(username);
 if (user && bcrypt.compareSync(req.body.password, user.hash)) {
 req.session.user = username;
 req.session.save((err) => {
 if (err) {
 res.send('Cookie saving error, <a href="/login">try again</a>`');
 } else {
 res.redirect("/");
 }
 });
 } else {
 res.send(`Incorrect username or password, <a href="/login">try again</a>`);
 }
});

app.get("/login", (req, res) => {
 res.status(200).sendFile(__dirname + "/login.html");
});

app.get("/register", (req, res) => {
 res.status(200).sendFile(__dirname + "/register.html");
});

app.listen(3000, () => {
 console.log("Server is running on port 3000");
});

Conclusion

Nearly every web app we write is going to need a user auth and session management solution. In this very long post we’ve looked at a way to develop that from scratch in Express/Node. In the process our code base went from about 10 lines to 130. Now that it’s done however, the only extra code to ensure users are only accessing the routes they should will be a line at the entry point of each route.

Since building out session management is such a common and onerous task, and one that can have serious consequences if not done correctly, you might be wondering if there’s libraries to do some of this, and other, lifting for us. There is, and I plan to look at some in the future.

Disable SSH root logins

Mon, 18 Sep 2023 00:00:00 +0000

This always makes me laugh:

It’s like half the traffic on the internet is bots trying random passwords on root accounts over ssh. This is on an Ubuntu VPS on BinaryLane that had only been spun up five minutes or so. Looks like about one attempt every 10 seconds.

This is why the number three thing on my new install list is to disable root access via ssh. Here’s my system - possibly just for Ubuntu and related systems:

Add a new user, and put them in the sudo group

add user fred
usermod -aG sudo fred

Then log out, and ssh back in with the user you just created. Now we want to edit the config file for the ssh daemon. Since we’re not logged in as root now, we’ll have to use sudo, so we’ll also find out if that’s working.

sudo nano /etc/ssh/sshd_config

If sudo doesn’t work, either you stuffed up adding the new username to the sudo group, or you don’t have sudo installed. If the problem is the latter, log out, and ssh back in as root and install it with apt install sudo

There is probably a line with PermitRootLogin in it. It may be commented out, or set to yes. But we want it to end up looking like this

PermitRootLogin no

We’ll need to restart the daemon to pick up the config changes.

sudo systemctl restart sshd

Now if you log out, and try to ssh back in as root, it should fail. If it doesn’t, a likely issue is that there’s other configuration files being included. I feel I’ve mentioned before that a common pattern with Linux config files, at least on the Debian based systems I use, is that there’s a main config file that you probably shouldn’t mess with, but it pulls in subsidiary config files, often in a subdirectory called conf.d

In the case of this file, there’s a line up the top saying

Include /etc/ssh/sshd_config.d/*.conf

which does exactly that.

Spy vs Spy

I’d love to know what passwords these bots are trying. I was thinking it wouldn’t be all that hard to write something that would face the password login process and run it on port 22 to see. I asked ChatGPT about this, but goodie-goddie that it is, all I got was a warning about ethics and some ssh security tips.

I’m not the first person to think of this, so I might come back to this idea later. If I was running brute force ssh I guess I’d use one of the common password lists from one of the big leaks, so it might not be that exciting. It would also be interesting to see what the first command they tried to run is as well.

Ansible with Secrets

Sun, 13 Aug 2023 00:00:00 +0000

We wrote a nice little Ansible playbook the other day to install nginx on our web servers and ensure it was running. We were able to store the usernames in the hosts inventory file using the ansible_ssh_user variable. Then, we ran the playbook with the command:

ansible-playbook web_installs.yaml --ask-become-pass

This asked us the password to use with the usernames in the hosts file. Luckily that day, it was the same username/password combo to use for sudo on every server. What happens if that’s not the case? Here’s our new hosts file for today. There’s a cool new sysadmin in town - Jane.

[vm323_deb]
100.108.154.133

[vm323_deb:vars]
ansible_ssh_user=ian

[vm324-deb]
100.77.75.14

[vm324_deb:vars]
ansible_ssh_user=jane

We could still use --ask-become-pass but it only asks us one password, and it’s highly unlikely (we hope) that Jane and Ian have chosen the same password.

If you look at the inventory file above, you can see how the variables work - it’s the same variable name - Ansible swaps the correct value in for each server as it accesses them. There’s many of these variables in addition to ansible_ssh_user, including ansible_ssh_pass, so maybe we can do something like this:

[vm323_deb]
100.108.154.133

[vm323_deb:vars]
ansible_ssh_user=ian
ansible_become_password=mittens96

[vm324_deb]
100.77.75.14

[vm324_deb:vars]
ansible_ssh_user=jane
ansible_become_password=GBLEzrvc8rnUFruVrCwm

If you try that, it will work. However it is a terrible idea to store our ssh passwords in that inventory file in plaintext. They would be available to anyone who gets access to my workstation, AND when I commit my work to git, it’s getting copied somewhere, probably including github. This is such a common problem there’s some business that have come into being to scan for passwords and API keys in people’s repos.

There is a better way. Ansible will allow us to store the secrets in a separate file as variables, and that separate file can be encrypted while it’s on disk, and Ansible will decrypt it to use from memory then clean up after itself. There’s a couple of new (to us) things here - variables, and the encryption. Let’s look at them separately.

Variables

[Ansible variables](https://docs.ansible.com/ansible/latest/playbook_guide/playbooks_variables.html) is a whole subject, we’re just going to look at the minimum we need to solve out problem.

We can create another file in our project, let’s call it plaintext.yaml and store our usernames and passwords in there as key: value pairs.

Then we need to tell our playbook to import that plaintext.yaml file with vars_files.

Then in the inventory file, we can substitute the usernames and passwords with our variables.

Now, when the playbook runs, it will substitute the real values into the host inventory file for us. Let’s check that.

Bingo bongo. But we haven’t actually solved our security problem yet. The ssh passwords are still dangerously stored in plaintext in our file system. What we have done is learned how to have variables in an external file, pull that into the playbook and have the values substituted in. Now we need the next step - keeping those passwords safe.

Ansible Vault

Clearly, every serious use of Ansible is going to have this issue (of needing ssh credentials, but not wanting to give them away to hackers) so of course, there is an elegant solution for it.

What if the file with all the passwords was stored encrypted, then only decrypted for use by our playbook, and it was never saved anywhere as plaintext? That solves the problem.

Ansible has a tool for this called Ansible Vault. We’ll create the yaml file with our variables with that tool, and it will be saved encrypted. When we run the playbook we’ll get it to ask us for the password to decrypt the file. It will do that in memory and run the playbook.

The command to create our file, which we’ll call vault.yaml will be:

ansible-vault create vault.yaml

This will ask us to enter the password we want to use. The strength of this password needs to be good. If it’s crackable, you are giving away root access to your systems. And it’s in a file, not in a login situation where you can time out after three logins. Whoever has the file has the time to brute force password of the the AES 256 encryption.

Once you’ve entered your strong password twice, it will open up the new file in the default editor - probably vim. You may need help to use this editor.

When you’re done, save the file and exit. What does this file look like:

Yep. That should do it. We still need to tell our playbook to use this file instead of the other one.

--- 
- name: nginx for all web servers
 vars_files: ./vault.yaml
 hosts: all
 become: yes
 tasks: 
 - name: nginx installed
 apt:
 name: nginx
 state: latest
 - name: nginx running
 service: 
 name: nginx
 state: started
 enabled: yes

And when we run the playbook, we need to let it know to ask us the vault password.

ansible-playbook web_installs.yaml --ask-vault-pass

If you need to edit the secrets file in the future, the command for that would be

ansible-vault edit vault.yaml

Once again it will ask the password, and once again it will open up in vim.

ssh key login on VPS

Sun, 12 Feb 2023 00:00:00 +0000

Due to potential brute force attacks, it’s a good idea to turn off password access via shh and instead rely on ssh keys. In this post, I’ll run through that process.

Generating your key

On a mac (or actually most *ix systems), your ssh keys live in the .ssh directory inside the users home directory. Since it starts with a period, it’s a ‘hidden’ directory. To see it in Finder press

Shift|Command|.

(that command includes the full stop). Here’s mine:

The keys are in pairs, there are two pairs of keys above: id_ecdsa and id_rsa. Each pair of keys includes a public key and a private key. It’s the public key we want to put on the server. The private key is precious - it should not be anywhere that others can access it. The public key can safely be provided to a server that can use it to securely authenticate the holder of the private key by doing some complicated stuff.

If you don’t already have a key pair, we need to create one. On Mac or Linux that is a straightforward process in a terminal, on Windows I think the recommendation is usually to use PuTTY.

Installing your key on the target system

It’s possible to create a file on the system we want to ssh onto and to paste the public key into in, but from a Mac or Linux machine, we can use the ssh-copy-id command which will do all that for us in an error-free way. It has the format:

ssh-copy-id <username>@<host>

Turning off password access

Although it’s convenient to ssh in without a password (because we’re using keys), the main reason for doing this is to turn off passwords to make brute-force password attacks impotent. For that, we need to turn off passwords for ssh.

Note that I’m running on Unbuntu server 22.x so your mileage may vary. Certainly when I was reading around about this I found many slightly different approaches, but this is what I’ve done and tested so that ssh refuses to accept passwords.

The configuration files for ssh on Ubuntu are at /etc/ssh/

It’s the sshd_config we’re interested in (ssh_config is for the client, we’re wanting to change the ssh server - daemon). You’ll also notice there’s a sshd_config.d directory. The reason for this is that the config file has a line in it at the top that includes all the config files in that directory. This is a common pattern in Unbuntu - the main config file pulls in other config files. When you see that, you really shouldn’t edit the main config file as it’s possible that a future update will change it, you edit, or add to the files in the directory below.

The way it words is that the commands in the files in the sub-directory will have priority over the defaults in the main config file (which is slightly counter-intuitive for me).

The VPS I am using is on binarylane, and they already have a config file in the subdirectory, so I’ll edit that.

With a standard Unbuntu install, there are no files in there, so you’ll need to create one with touch, then add the line PasswordAuthentication no

Many of the articles on the internet also talk about turning off ChallengeResponseAuthentication and UsePAM, but I found with my VPS and local Unbuntu servers, all that was needed was PasswordAuthentication if my intention was to prevent these attacks.

Note that once this config is activated, you won’t be able to log in via ssh with a password, so it would be foolhardy to do it without having set up and tested your login with keys.

This config won’t be active until the ssh daemon is restarted.

sudo systemctl reload ssh

Now if someone attempts to ssh in they’ll see this:

Not that turning off passwords like this is only for ssh. You’ll still be able to log in via the console if you’ve stuffed something up.

Chinese Hackers Want to steal my Hello World container

Mon, 06 Feb 2023 00:00:00 +0000

A smart thing to do after setting up a server on the internet, is to set up SSH keys and then turn passwords off for SSH. The reason for this is that scanning for open port 22 on IP addresses, then brute forcing password files on them is pretty much hacker 101. So if you have passwords turned on, and especially if you have a weak password you are really inviting someone to take over your server as root and add it to their botnet army for liking Putin’s twitter posts or whatever.

When I was writing the post about looking for the sudo attempt ‘report’, you might have noticed some sshd timeouts:

That’s what’s going on there. SSH has a timeout value of about a minute. I’d also guess those kex_exchange_identification messages are suspicious as well. I thought I’d google one of the IP addreses:

Oh, so it’s China, and multiple people are reporting SSH brute force attacks:

sudo Incident Reports - where do they go?

Sat, 04 Feb 2023 00:00:00 +0000

Even though it’s my server, I still have a pang of guilt when this happens.

I always imagine Richard Stallman (or someone with a similar 2000’s database administrator beard) looking at me disappointedly and shaking his head slowly.

It does raise the question though - since it’s my server, shouldn’t I be getting a text message from CERN or something?

Where is this report?

(Relevant xkcd)

Like everything, the answer is ‘it’s logged’. We can use the journalctl command to look at the logs, on this server that’s been running less than 20 hours, there’s already several thousand lines to look through if you just enter journalctl, so I’m going to just send all the high priority logs to a file:

journalctl -p 3 > errors.txt

Then since this just happened, it should be at the end of the file:

tail errors.txt

Jan 28 12:10:40 enrico-rider sshd[5168]: fatal: Timeout before authentication for 110.41.153.190 port 41826
Jan 28 12:11:01 enrico-rider sshd[5170]: fatal: Timeout before authentication for 110.41.153.190 port 41856
Jan 28 12:23:15 enrico-rider sshd[5222]: fatal: Timeout before authentication for 61.177.173.39 port 29421
Jan 28 12:23:26 enrico-rider sshd[5223]: fatal: Timeout before authentication for 61.177.173.39 port 49692
Jan 28 12:23:37 enrico-rider sshd[5226]: fatal: Timeout before authentication for 61.177.173.39 port 10416
Jan 28 12:39:51 enrico-rider sshd[5517]: fatal: Timeout before authentication for 61.177.172.108 port 53867
Jan 28 12:50:06 enrico-rider sshd[5653]: error: kex_exchange_identification: Connection closed by remote host
Jan 28 13:03:53 enrico-rider sshd[5696]: error: kex_exchange_identification: Connection closed by remote host
Jan 28 13:24:58 enrico-rider sshd[5804]: fatal: Timeout before authentication for 61.177.173.39 port 46041
Jan 28 13:40:06 enrico-rider sudo[6077]: ian : user NOT in sudoers ; TTY=pts/0 ; PWD=/home/ian ; USER=root ; COMMAND=/usr/bin/docker ps

There we go, it really has been reported!

How to add a user to the sudoers file

To avoid these terrible reports, it sounds like I need to add myself to the ‘sudoers file’. I won’t be able to do that as myself, so I’ll log back in as root for a bit. The reason I don’t just operate as root all the time is that I quite like the constant reminder that I’m about to do something administratory - so I should have a second thought before I sudo that shell command I just copied out of a stackoverflow answer.

Since the error message says I’m not in the sudoers file, I should just add my name right? Well yes, and no. That is possible, but it’s slightly dangerous - it has a specific format, and if you stuff things up bad things can happen. For this reason there’s a special command to edit it (visudo) which refuses to save it if you make a mistake.

The sudoers file is at /etc/sudoers, if you cat it, it has a heap of commented out stuff, but there’s be a section that looks like this:

# User privilege specification
root	ALL=(ALL:ALL) ALL

# Members of the admin group may gain root privileges
%admin ALL=(ALL) ALL

# Allow members of group sudo to execute any command
%sudo	ALL=(ALL:ALL) ALL

# See sudoers(5) for more information on "@include" directives:

@includedir /etc/sudoers.d

That last line includes any files in the /ect/sudoers.d as part of this one, so if we really did want to add ian to this file, we’d do it there, but still by using the visudo command to do it safely.

But, we don’t need to. The %admin and %sudo lines are granting these permissions to groups, so all we need to do is add ian to the sudo group and those permissions will be granted, safely.

usermod -a -G sudo ian

Success:

ian@enrico-rider:~$ docker ps
Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get "http://%2Fvar%2Frun%2Fdocker.sock/v1.24/containers/json": dial unix /var/run/docker.sock: connect: permission denied
ian@enrico-rider:~$ sudo docker ps
[sudo] password for ian: 
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
520ed656ef12 dockersamples/101-tutorial "nginx -g 'daemon of…" 14 hours ago Up 14 hours 0.0.0.0:80->80/tcp, :::80->80/tcp pedantic_bartik
ian@enrico-rider:~$

Mock Data

Mon, 28 Nov 2022 00:00:00 +0000

One of the things we need during app development is some data to play with. It would be unethical for me to use real student data to test my app, even if I wasn’t sharing screenshots of the development here, so I’ll need to build some mock data. The prospect of making 400 rows of data manually does not sound like a good use of time, so I started to think about generating it in Excel. I’d used an online “random address generator” for an earlier project, so I was contemplating pasting that sort of data into Excel workbooks and randomly selecting from it.

It occurred to me someone probably already solved this problem, and a quick search confirmed there are hundreds of web based test data generating sites. They provide data in all sorts of formats as downloads or via APIs. Somewhat at random, I selected Mockaroo.

In addition to letting you specify the fields, you control the type of data to go in them. They have a heap of options of specific formats such as ISBN’s, airport codes and so on. The whole thing took a couple of minutes to setup.

As I was doing it, it occurred to me that a completely random date of birth didn’t make perfect sense - there really should be a connection between the year group of students and their date of birth. It’s completely immaterial for this project, so I didn’t bother with it, but in fact Mockaroo has ruby based scripting that allows you the flexibility to make one field somehow calculated from others so that would be possible

It was a simple matter to download my data as JSON, but there’s also options for fetching it from a REST API.

I’ll download and host this on my home Raspberry Pi server that I bought for a different project, but it’s possible just to pull it direct from their server.

A small note of caution is that if you sign in to save your schema, it is saved, but it’s also public. That may not matter for your context, but in some situations it could be important. This could be from the data you provide as options, or even just the property names might provide intelligence useful to someone. As IT professionals we need to be mindful of the risks we’re taking with client or user information and to either eliminate or explain them.

Security on blog.iankulin.com

Share files securely with Enclosed

Authentication

npm ERR! Exit handler never called!

package-lock.json

npm update

The Error

SSH login notification

Changes

Due Diligence on a Docker Image

Who do you trust?

Trust Factors

Choosing an Image

Quick && Dirty auth with nginx & Node

Prerequisites

Install nginx

Installing Node

The App

The Firewall

Proxy Pass

Credentials

Accessing the user in node

Limitations

Links

Beginning Node App Security

Put it behind Nginx

Basic Auth

HTTPS

Logging

Fail2ban

Cloud Firewall

No root login

SSH keys

Updates

Monitoring

User Sessions & Cookies in Node

The problem we’re addressing

Code example

Persisting across server restarts

Users

Log out

Users & view counts

Creating a user

Logging a user in

Showing the view count

Authentication

Passwords

Forms

Sanitising inputs

HTTPS

Other security

Conclusion

Disable SSH root logins

Spy vs Spy

Ansible with Secrets

Variables

Ansible Vault

ssh key login on VPS

Generating your key

Installing your key on the target system

Turning off password access

Chinese Hackers Want to steal my Hello World container

sudo Incident Reports - where do they go?

Where is this report?

How to add a user to the sudoers file

Mock Data