compressed zfs, postgres dataset, how much dataset size will i experience with using…
- User
compressed zfs, postgres dataset, how much dataset size will i experience with using zstd compression (i assume default level is 3, openzfs) vs zstd-4 vs zstd-5. are there other algorithms? im looking for good compression but not too havy on the cpu.
- Assistant
For PostgreSQL datasets on ZFS, here's what you can expect:
zstd Compression Levels:
- zstd-3 (default): Typically 2.5-4x compression ratio for database data
- zstd-4: Usually adds 5-10% better compression over zstd-3
- zstd-5: Another 3-7% improvement over zstd-4
The gains between zstd-3 to zstd-5 are relatively modest - you might see your dataset go from 4x compressed to 4.3x compressed, but CPU usage increases incrementally with each level.
Other ZFS Compression Algorithms:
lz4 - Extremely fast, minimal CPU impact, typically 2-3x compression for database data. Great choice if CPU is your primary concern.
gzip-6 - Better compression (often 3-5x) but significantly more CPU intensive than zstd. Usually not worth it for active databases.
lzjb - Older algorithm, generally outperformed by lz4.
For PostgreSQL specifically, I'd recommend:
- lz4 if you want minimal CPU overhead and decent compression
- zstd-3 for the best balance - it's the sweet spot for most database workloads
- zstd-4 if you have CPU headroom and want to squeeze out a bit more space
PostgreSQL data tends to compress well due to repeated patterns in indexes and text data. Your actual ratios will depend on your specific data types and patterns, but zstd-3 is usually the best starting point for active database workloads.
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.