A few weeks ago, I was very excited to be able to take a snapshot of a virtual machine, copy it across the network from that Proxmox node, copy it back across the network to a different Proxmox node, start it there, and have it up and running, without it noticing it was actually on different hardware.
Backing up a VM is pretty simple, you just click on the node, choose Backup and click the Backup Now button. The ease, and completeness of backing up a VM is one of the main reasons I’m using Proxmox for my systems.
I’m still loving the Docker “just works” magic, despite their terrible PR skills, but sometimes I start a container, then the docker ps -a shows it exited almost immediately. Clearly I’ve made a mistake, but there’s no stdout error message to tell me what I’ve done wrong, where is it.
Let’s look at an example from today. I’m testing Filebrowser on a dev machine before I deploy it to the remote backup machine I’m assembling. And instead of following the official instructions, I’m following a blog post which has a few more details, but unfortunately also a small error.
I’ve discussed before, that when you first install Proxmox, it grabs an IP address from your DHCP server (this usually runs in your ISP modem if you haven’t created a better setup), but then it stores it as a static ip. This is a sort of compromise that makes sense and works for most circumstances.
As soon as I’ve provisioned a new Proxmox server, I then usually tell the DHCP server, to always serve that address to the MAC address of the new Proxmox server. Since Proxmox does not use the DHCP server on subsequent boots, all that really does is prevent the DHCP server give the same IP address out to another device - which had happened to me prompting the earlier post. The DHCP server had given the address to a wifi lightbulb while the server was off, then when the Proxmox server booted up, the netwrok access was all messed up.
I’m in the process of shuffling disks around as I move towards my 3-2-1 storage arrangements. I thought after my extensive rsync adventures I’d mirrored everything everywhere, but then realised, with a sinking (no pun) feeling, after I’d repurposed a drive out of the 2 drive Synology as a USB caddy drive and wiped it, that I’d forgotten my audio book directory. All my rsync fiddling around had been on the video subdirectory of the media folder, not the whole media directory that included my audiobooks.
The hardware engineering of these corporate world mini-PCs is really nice. I swapped out the RAM today to bump my main machine up to 32GB from 16GB. It was a straightforward task - no screwdrivers, no drama.
To open the machine up, there is a single large screw on the back that can be undone with your fingers - it’s a captive screw, as in it doesn’t fall out - just another nice engineering thought.
I’ve got some extra RAM to drop into the HP 800 G2 mini that I use as my production server. I feel like that’s a low risk change, but since it’s easy to take VM snapshots I shutdown the VM’s and did that, and wanted to just copy them off the local storage.
I’m moving towards having these backups (and the ISOs) on the NAS rather than locally, but have not implemented that. So to get my backups I need to SSH in and find them.
With all those earlier rsync bumps out of the way, I was ready to try my first rsync backup at the command line to sync my movies directory on the NAS to a (NTFS formatted) USB drive plugged into the same NAS. This is to be one of the simplest since there’s no remote server involved, just copying from mount point directory to another - so no drama with remote permissions.
The reason I’ve been figuring out rsync is to setup my backup strategy. Eventually this will partly be managed with scheduled tasks (ie cron jobs) running rsync. I wanted the SSH in and try this out, since I didn’t know some basic things like the mount points of the shares.
Mount points
My first issue was to find the paths to all my data. This turned out not to be a drama. Each of the volumes you create when the NAS is set up are just in the root directory. This includes any USB drives plugged in.
The Synology operating system DSM (I’m on DSM 7.1.1) is Linux, but its highly customised for the purpose of making running a complicated Linux NAS doable for less technical users.
Due to that, some things that are routine in a regular distro, require a few more steps to jump through to get them to work. SSH-ing in to a Synology with keys is one of those things.
Should you?
Before you do start fiddling around, it’s probably worth mentioning that almost all the things you might want to do on the Synology can be accomplished through their web interface, or by installing a ‘package’ from the Package Center. For example, if you need to run a cron job, that’s done through the Control Panel ‘Task Scheduler’. If you need TailScale installed to easily access it over Wireguard, there’s a TailScale package. In general it’s probably easier and safer to do things their way.
I’ve started down the path of improved storage management, including embracing the 3-2-1 mantra. I’ve settled on a RAID6 NAS for local, mirrored to an off-site NAS, and an offline local USB drive.
While I’ve been setting those up, my services have been live, so files have been changing on my main storage, which I’ve then switched to the bigger NAS, and I’ve been trying to keep data in sync by remembering what changes have been made where, and manually replicating them. That’s not sustainable and not the plan.
When I was a young whipper-snapper, working at the “data processing” centre, you could see if one CPU was better than another one by the CPU name/number. No one wanted an 8086 once the 286’s came out. Then a 386 was what you wanted for the latest multitasking support, but only till the 486 was available, then you wanted that for the gargantuan memory addressing.
With that idea firmly in mind, I’ wanted an i5 to be better than an i3, and an i7 better than all of them, but it’s apparently not that simple. I do come across people in forums talking about ‘generations’ of Intel processors - so all this is probably decodable, but I’m not exactly sure how.
Since the HP EliteDesk 800 G1 I’m using as a dev/homelab machine is going to be re-purposed as a media/backup server elsewhere, I’ve grabbed another G2 to use as a second box. The homelab machine serves as a backup device for the production server that runs my self-hosted services, but also is the machine I play with - testing my software, but also trying out any new self-hosted software I’m having a look out or configurarions I’m thinking about for the ‘production’ server.
I went over mounting a Synology NAS share on a Mac or Linux host a while ago. Now I’ve populated a new NAS, and I want to copy my data over to it. I could mount them both to my laptop, and the data flow would look like this:
Since I’m copying 4TB, it will take a few hours, and if I forget what’s going on and close the laptop, or take it outside of my wifi the transfer will die, and I won’t be sure which files are patent. What might be better would be something like this:
I ordered some RAM this week for my production server - it’s quickly becoming clear that memory is the limiting factor when running lots of services and VM’s that don’t get much use - rather than processing power. I’m not really a hardware guy, so figuring out exactly what RAM I need is a slightly fraught process - I won’t be fully confident I’ve ordered the right thing until I install it, boot up, and see my G2 800 come to life maxed out at 32GB.
I ran into a little hiccup today. I’m building out a Jellyfin media server in a little HP G2 Mini PC. The config was going to be a Debian server inside Proxmox (because I love VM snapshots for backups) running Jellyfin in a container. There’ll be an external USB3 hard drive for the media storage.
I was intending to build it all out and test it, then ship it to it’s final home.
I’m not super interested in FreeDOS, but did enjoy this video from Jim Hall since I lived through all this, and was working in IT (well, ‘data processing’ actually) during the introduction of the IBM PC.
My first DOS was 2.11, but spent a lot more time on 3.12, and later 4.01. Windows wasn’t really ready for anyone until 3.1 which is when I dived in there. I seem to remember purchasing a PC with a whole megabyte of RAM in anticipation!
I’ve been really happy with my two bay Synology NAS - a DS216j. The Synology’s seem to have great reputation for just pushing on. Mine is loaded up with two 8TB Seagate Barracudas in RAID 1 leaving me with a one drive failure redundancy.
I guess a more hard-core host-er than me would be building their own array and using Unraid or ZFS or something. I’m pretty comfortable with the Synology off the shelf system; it’s a good match for my (low) level of expertise, and more robust than my previous storage system of a USB external drive.
I’ve spent a few hours over the weekend migrating a media library from an external USB drive to the NAS, and in the process reorganised it, and in many cases bulk changed file names. I’ve also added a heap of metadata.
I’d like to check that I haven’t missed any files, but a side by side listing of each data source won’t do the trick, so I’ll probably end up pulling the data into a spreadsheet, but I’d like to get as close as possible with Linux-fu first.
Unix, and therefore Linux, was built from the ground up as a multi-user system. Thanks to this, great security is baked in, for example every file has permission attributes for it’s owner, the group the owner is a member of, and then everyone. For example, it might be a good idea if I can read, write and execute my own files, but the other members of my group can just read them, and any other user on the system has none of those rights.
