L2ARC
definitionThe instinct: the pool is slow, there's a spare fast drive (an SSD) in the drawer, so add it as a cache—more cache is always better. Actually, L2ARC—a second-tier cache on that fast drive—is not free speed. Here's the catch: ZFS must keep a catalog in memory of everything parked on the cache drive. Every piece cached on the SSD spends a little of the memory cache—the one that's actually fast. On a machine without much memory, you can end up trading excellent cache for mediocre cache and calling it an upgrade.
The honest decision procedure, in order:
- Max out RAM first. It's the better cache and it needs no index.
- Measure.
arc_summary—if the ARC hit rate is already high (homelabs: it usually is), there is nothing for an L2ARC to catch. - Only with real, sustained ARC misses on a working set that fits on the
SSD:
zpool add tank cache <ssd>.
Two softeners worth knowing: losing an L2ARC device is harmless (it's pure cache—the pool doesn't depend on it, unlike a SLOG mid-crash), and modern OpenZFS supports a persistent L2ARC that survives reboots, fixing the old cold-cache-every-boot complaint. Neither changes the arithmetic at depth 2.