Blog

ZFS Snapshots and Scrubs: Data-Integrity Gymnastics

ZFS snapshots and scrub on a home NAS

Data-Integrity Gymnastics: A Home NAS Maintenance Calendar

What draws me to ZFS isn't that it's a trendy filesystem — it's the checksum-plus-snapshot combo: silent corruption can be caught, and mistakes can be rolled back. So this post follows the maintenance calendar — what to do on which day, and what signals mean it's time to escalate — instead of another command cheat sheet.

First Sunday of every month: scrub day

zpool scrub tank
zpool status tank

I run a full-pool scrub monthly, scheduled during low-traffic hours. It eats IO, so don't collide it with full-disk backups or media-library scraping. The calendar reminder just says "start a scrub"; the thing that actually matters is reading zpool status afterwards: any checksum errors, is it stuck in progress too long, has a disk already been offlined.

If you find checksum errors, move into drive-replacement or repair assessment — don't clear the log and pretend you didn't see it. With a mirror or raidz still holding redundancy, replace per the docs; if redundancy is already gone, make sure the backups are readable before you touch anything. A scrub is not a backup; it only answers "does the pool read back self-consistent right now." Also make a note if a scrub suddenly takes twice as long as last month — the pool may be fuller, a disk may be struggling, or there's just other heavy IO going on. Correlated, not necessarily a fault, but worth keeping an eye on.

Weekly: glance at snapshots and space

Important datasets: hourly / daily / weekly tiered retention; the media library: daily, short retention. Use sanoid or a simple cron. Name snapshots with the date and the reason, e.g. @before-immich-upgrade; manual snapshots shouldn't pile up as forgotten snap1, snap2.

Once a week, look at:

zfs list -o space,name
zfs list -t snapshot

"I deleted files but it's still full" usually means snapshots still reference the old data blocks. df won't drop the way your intuition expects. Understand the reference relationships, then clean up stale snapshots; blindly rm-ing inside the dataset won't reclaim snapshot space. Mark a "snapshot prune day" on the calendar, aligned with application major upgrades — take a named manual snapshot before the upgrade, watch for a week afterwards, then let it fall into auto-expiry.

Quarterly: snapshot-boundary refresher + send/recv spot check

In-pool snapshots protect against accidental deletion and rollback before ransomware spreads. They don't protect against a whole-chassis failure, or pool-level config disasters. Keep restic offsite, or zfs send | zfs recv to another machine. Every quarter, do an incremental send spot check: does the receiver have enough disk, does the incremental report errors, can the receiving side read-only mount and spot-check files.

The first full send takes a while; incrementals are fast. Replicating doesn't mean you can retire the source pool's snapshot policy — think about retention on each side separately. The receiver is also "another disk that might go bad," so it doesn't replace the semantics of an encrypted offsite backup; what it solves is "the pool is gone, but the recent snapshot chain lives on another box." When spot-checking, randomly open a few real files and verify them — more reassuring than just reading zfs list. I once hit an edge case where the incremental reported success but a few large files failed to read on the receiving side; the root cause was a network blip mid-transfer and an incomplete re-run strategy.

Anytime: the gate before upgrades and high-risk operations

Changing compression, changing recordsize, migrating a dataset, major app upgrades: take a named snapshot before, and log "who did it, why" in the calendar afterwards. Keep a high-risk operations list pinned to the rack or the top of your notes: double-check the name before destroying a dataset; don't experiment on the wrong pool. How long your rollback window lasts depends on whether you still have unexpired snapshots — the retention policy on your calendar is the lifespan of your undo key.

Yearly: does the policy still match the data

A year later the data profile shifts: photos explode, surveillance footage joins, VM images get bigger. Re-evaluate which datasets deserve dense snapshots and which only short ones; whether the scrub frequency still makes sense; whether the offsite backup covers the blind spot where "snapshots feel safe but are actually same-site." If the media library already has another copy on object storage, local snapshots can be shorter; documents and databases are the opposite.

Data integrity is a marathon: scrub is the checkup, snapshots are the undo key, and the offsite backup is the safe. The point of a maintenance calendar is turning these three from "I'll do it when I remember" into "it just happens on schedule." I keep scrub summaries, snapshot space anomalies, and send spot-check results in the same ops notebook, and re-reading it every six months beats watching green monitor dots — green dots say you can ping right now; the notebook says whether you're actually managing risk.

A checklist tied to rest days

I pin ZFS maintenance to the calendar instead of "I'll get around to it":

  • First Sunday of the month: check whether the last scrub finished and whether any checksum-error email arrived
  • Every quarter: pick a minor dataset and rollback to a test clone to confirm the snapshot actually works
  • Before a disk swap or expansion: run zpool scrub once manually, confirm the pool is healthy, then touch anything

Name snapshots with the date and reason, e.g. manual-2026-02-01-before-immich-upgrade. When automated snapshots pile up, hand-written labels can save you in a panic. I set the space alarm threshold at 80%: ZFS gets ugly on performance and fragmentation when it's very full, and leaving headroom matters more than using every last TB.

← Back to Blog