brainbaking/content/post/2024/05/the-backup-and-sync-strateg...

5.9 KiB

title date categories tags
The Backup And Sync Strategy, Revised 2024-05-16T08:30:00+02:00
software
backup
NAS
archiving
self-hosted

After fiddling with various ways to sync notes across multiple vaults and iterating on my local data backup strategy, I figured I needed a new overview on the current design and the tools involved. I tried to keep things as simple and as low-friction as possible, with varied success, as you can see in the following naive attempt to draw the moving components in my notebook:

Let's try to break that down.

The hardware involved:

  • Our local NAS with multiple responsibilities of which backup server and photo server are the relevant ones for this post;
  • A USB HDD plugged into the NAS that for the moment acts as the "external backup site" (yeah I know...);
  • My wife's and my personal laptops;
  • My work laptop that should access the work vault notes repository;
  • My wife's and my own smartphones (hers iPhone mine Android);

The required backup software involved:

The optional software that holds data or makes it easier to work with data:

  • Obsidian as a local note-taking vault, both on macOS and on Android;
  • PhotoPrism as a local photo server;
  • GeniusScan as a way to digitize analog notes;
  • Gitea as a local code repository.

The general strategy is to leverage Restic to periodically backup selected folders to the NAS. I keep for example at most 60 snapshots per machine, which are periodically pruned. restic-nas-backup.sh runs daily on the NAS itself to backup itself (wait what?), while on other machines I use Restic Tray that does the same by pushing it to the NAS through SFTP. Everything's put in a separate backup volume. That's then rsynced to the USB HDD---which obviously should be replaced by something off-site.

Some stuff, such as local Git repositories, end up on the NAS uncompressed in multiple ways, but I don't care: the more copies, the better. In theory, I could exclude that from the Restic strategy, but that sounds like a bad move. Bought music albums---physical or otherwise---are put on the NAS (accessed through Navidrome) and included in the copy-to-USB-HDD.

The mobile strategy works a bit different. For Android1, I have three folder configured in Syncthing: the Obsidian vault, my photos, and a general documents folder for PDF files or other stuff I might want to have transferred. These are intentionally kept separate. Document/digital note syncing on iOS isn't needed for us.

Speaking of iOS, of course Syncthing doesn't work, and Mobiüs Sync, the unofficial offical port, can't sync photos, even in the paid version, which had me reach (and pay) for PhotoSync instead. That works well enough: I've configured it so that when my wife favorites a photo, only that one automatically ends up in our digital photo album in PhotoPrism. PhotoSync can push using lots of protocols, and you can even directly point it to your PhotoPrism WebDAV endpoint.

The work strategy also diverts a bit: I don't want any work-related data in my personal backups, but I do want to sync my work-only notes (Obsidian vault) with my personal vault. At the moment, it's simply a subfolder, and again Syncthing does the hard work here. I want to avoid having personal notes on my work laptop (which is my client's, not mine). See Syncing Notes Across Multiple Vaults.


Thanks to Syncthing, even if the house goes up in flames, taking the NAS and the USB HDD next to it, I still have the most critical files on yet another device: the phone. I should make work of periodically making a copy of that USB HDD to put at my parents' house. The thought of uploading all that data to a random cloud---even if it's encrypted-at-rest and I own the key---doesn't really speak to me. I know Mike Harley syncs his Synology backups with his friend's, which sounds like a much better deal to me.

Thanks to Restic, I can open up and inspect incremental backups on any machine, even if it's my old 32-bit retro Linux rig. That was the biggest reason to get rid of Apple's Time Machine.

Or, to put it more generally, thanks to local-first and/or open-source software such as Restic, Obsidian, and PhotoPrism, there's much less chance of a gnarly vendor lock-in. Don't like PhotoPrism? Fine, your photos are still just folders on your drive. Don't like Obsidian? Fine, your notes are still just Markdown files on your drive.

The whole system does have one huge downside: where do you put all those encryption keys? In your KeePass database? But that file is also (double-)encrypted as part of the backup... Do you keep a print-out of your password database somewhere hidden in a safe behind a painting, as recommended by any security best practices guide?

Let me know if there's an obvious hole in our strategy and feel free to share your own super-secret backup plans!


  1. It's not impossible to use Restic with Android, but it's not officially supported and requires jumping through hoops. ↩︎