ZFS storage

ZFS (OpenZFS) is the illumos filesystem and volume manager, and the substrate solnix’s upgrade model is built on. If you come from NixOS you may have used ZFS as an optional root filesystem; on illumos it is the filesystem — the root pool, the swap, the dump device, and every boot environment live on it. This page covers the concepts (pools, datasets, snapshots, clones) and the zpool and zfs commands you need, then ties it back to boot environments.

On illumos generally ZFS is mature and load-bearing — illumos is a founding OpenZFS platform. On solnix ZFS is inherited from the illumos gate; the commands below are the real illumos commands. What is planned on solnix is the tooling that ties Nix generations to ZFS boot environments (see that page).

The model: pools and datasets

Traditional systems layer a filesystem on a partition on a disk. ZFS collapses that: you build a pool (a zpool) out of one or more devices with a chosen redundancy, and then create datasets inside the pool that draw from its shared free space on demand. There are two kinds of dataset:

Filesystems — mountable trees of files (most datasets).
Volumes (zvols) — block devices carved from the pool, used for swap, the crash-dump device, and zone/VM disks.

The root pool is conventionally named rpool. Boot environments live under rpool/ROOT/.

`zpool` — manage pools

zpool list                       # pools, size, health, capacity
zpool status                     # per-device health, errors, scrub progress
zpool status -v                  # verbose, lists any damaged files
pfexec zpool scrub rpool         # verify every block against its checksum
pfexec zpool create tank mirror c1t0d0 c1t1d0   # a mirrored pool

zpool status is your first stop for disk health. A scrub reads every block and checks it against its checksum; in a redundant pool it repairs bad blocks from the good copy automatically (“self-healing”). Scrub on a schedule.

Redundancy is chosen at pool creation — mirror, raidz, raidz2, raidz3 — and ZFS integrates the volume-manager job (which disks, what redundancy) with the filesystem, so it always knows which copy is good.

`zfs` — manage datasets

zfs list                         # datasets, used/avail/refer, mountpoint
zfs list -t snapshot             # snapshots
pfexec zfs create rpool/data     # a new filesystem dataset
pfexec zfs set compression=on rpool/data      # a property
zfs get all rpool/data           # every property of a dataset

Datasets have properties (compression, quota, mountpoint, recordsize, …) that are inherited down the tree unless overridden — a clean way to manage a hierarchy of filesystems from one place.

Snapshots

A snapshot is a read-only, point-in-time image of a dataset. It is copy-on-write, so it costs nothing at creation and grows only as the live dataset diverges from it.

pfexec zfs snapshot rpool/data@before-upgrade   # take a snapshot
zfs list -t snapshot rpool/data                 # list them
pfexec zfs rollback rpool/data@before-upgrade   # revert the dataset to it
pfexec zfs destroy rpool/data@before-upgrade    # remove a snapshot

You can browse a snapshot’s contents read-only under the dataset’s hidden .zfs/snapshot/<name>/ directory without rolling back — handy for recovering one file.

Clones

A clone is a writable dataset created from a snapshot; it shares the snapshot’s blocks copy-on-write and only consumes space as it is written to. Clones are the mechanism behind boot environments:

pfexec zfs clone rpool/data@snap rpool/data-copy

`send` / `receive` — replication and backup

A snapshot can be serialized to a byte stream and applied elsewhere — the same pool, another disk, or another machine over the network:

pfexec zfs send rpool/data@snap | pfexec zfs receive tank/data
# incremental, only the delta between two snapshots:
pfexec zfs send -i @snap1 rpool/data@snap2 | ssh host pfexec zfs receive tank/data

This is the foundation of ZFS backup and disaster recovery — full and incremental, verifiable end to end.

How this ties to boot environments

Everything on the ZFS boot environments page is these primitives applied to the root filesystem:

A boot environment is a dataset under rpool/ROOT/.
Creating a BE takes a snapshot of the current root and makes a clone of it — which is why it is instant and nearly free.
The clones share blocks, so twenty generations do not cost twenty full roots.
beadm is a convenience layer over exactly these zfs operations, aware of the boot loader.

So the same integrity guarantees (checksums, scrub, self-healing) and the same send/receive backup story that protect your data also protect your generations.

The solnix handbook