Ceph or ZFS on Proxmox, one question to ask

It is one of those debates that fill homelab forums and meeting rooms alike and it drags on because both camps compare things that do not compare. ZFS and Ceph do not play in the same category: one is a local filesystem and volume manager of remarkable robustness, the other is distributed storage aiming at scale and self-healing and trying to separate them in the absolute makes no more sense than asking whether a 13mm wrench beats a drill. The right question is not “which is better”, it is “how many nodes and what real high availability” and once that one is answered, the rest follows almost mechanically.

The rule

One or two nodes: ZFS, no hesitation. Ceph needs a quorum two nodes cannot honestly form and at that scale you do not want its operational complexity anyway. ZFS gives you snapshots, compression, end-to-end integrity and send/recv, which covers most of what a small cluster expects from its storage.

Three nodes or more, with a genuine high-availability need, that is, the ability to lose a node and see VMs restart elsewhere on shared state: Ceph. Distributed, replicated, able to rebuild itself after a failure, it is made precisely for that scenario and does it well, provided you give it what it demands, which I come back to.

Three nodes or more, but where you prefer simplicity to strict HA: ZFS completed by pvesr replication. That asynchronous replication, specific to ZFS, offers an economical failover with a loss window equal to the replication interval, without the operating weight of Ceph. It is not realtime HA, it is an owned trade-off and for many fleets it is exactly right.

The traps that bite

The rule is simple, the traps are less so and they are what turn a clean deployment into a production nightmare. Ceph wants a fast, dedicated network, 10 GbE at the strict minimum, ideally 25 and note that latency matters to it more than raw throughput, so a Ceph cluster on shared 1 GbE is not slow, it is unusable. Ceph also wants its three replicas, size=3 and min_size=2 by default, which means your usable capacity is one third of your raw capacity and better to know that before ordering disks than after.

ZFS, for its part, eats RAM through its ARC and appreciates ECC and while raidz resizing relaxed with the expansion that landed in OpenZFS 2.3, one still plans the layout as if never touching it again, because that remains the best way not to trap yourself.

Finally and this holds for both: never, ever, put a hardware RAID between the disks and ZFS or Ceph, which want to see the bare disks, so an HBA in IT mode and nothing else. And plan enterprise SSDs with power-loss protection for everything touching Ceph’s journals and databases as well as ZFS’s SLOG, or watch latency collapse at the first serious fsync.

The rest is taste and context, but as long as you answered the node count and HA question first, you will not pick the wrong category and that is where ninety percent of the regrets play out.

[edit: 2026] The Ansible implementation of all this: the microceph and proxmox roles.