Zstandard (zstd) Compression Library

Overview

Zstandard (zstd) is a fast, lossless compression algorithm designed for real-time compression scenarios. It achieves zlib-level compression ratios while delivering significantly faster speeds, backed by the Huff0 and FSE entropy encoding libraries. The format is standardized in RFC8878 with multiple independent implementations available.

Key Characteristics

Zstd offers a configurable speed-to-compression trade-off through compression levels ranging from <0 (fastest) to 22 (strongest). Decompression speed remains consistent across all settings, a property shared with most LZ-based algorithms. The library is production-ready, deployed at scale by Meta and other major cloud infrastructures, and continuously fuzzed for security by Google's oss-fuzz program.

Core APIs

The library provides three primary compression approaches:

1. Simple API - Single-step compression/decompression for straightforward use cases
2. Explicit Context API - Reusable contexts for better memory efficiency when compressing multiple times
3. Streaming API - Unbounded multi-step compression for handling large or streaming data

Key functions include ZSTD_compress(), ZSTD_decompress(), ZSTD_compressStream2(), and ZSTD_decompressStream().

Dictionary Compression

For small data compression, zstd supports dictionary-based compression. Training on sample data creates a dictionary that dramatically improves compression ratios on similar data. This is particularly effective for the first few KB of data, after which the algorithm learns from previously decoded content.

Error Handling

The library provides comprehensive error handling through zstd_errors.h, which defines ZSTD_ErrorCode enum values. Functions return size_t results that can be tested with ZSTD_isError() to distinguish between successful sizes and error codes.

Build System

The primary build system is make, which generates both static and dynamic libraries. Alternative build systems include CMake, Meson, VCPKG, Conan, Visual Studio, Buck, and Bazel. The library supports multithreading through the ZSTD_MULTITHREAD macro and pthread compilation flags.

Library Organization

The lib/ directory is modular, containing:

• compress/ - Compression implementation
• decompress/ - Decompression implementation
• common/ - Shared utilities
• dictBuilder/ - Dictionary training
• legacy/ - Support for legacy format versions
• deprecated/ - Deprecated APIs

Loading diagram...

Licensing

Zstd is dual-licensed under BSD or GPLv2, allowing flexible integration into various projects. The reference implementation is provided as an open-source C library with command-line utilities supporting .zst, .gz, .xz, and .lz4 formats.

Architecture & Core Components

Zstd is built on a modular architecture with three main layers: match finding, sequence encoding, and entropy compression. Each layer is independently optimizable and can be adapted for different compression strategies.

Compression Pipeline

The compression process follows a clear pipeline:

1. Match Finding – Identifies repeated patterns in the input using hash tables or binary trees
2. Sequence Generation – Converts matches into a sequence of literals and match references
3. Entropy Encoding – Compresses sequences and literals using FSE and Huffman coding

Loading diagram...

Match Finding Strategies

Zstd supports multiple match-finding algorithms, selectable via compression level:

• Fast – Single hash table lookup, minimal memory, low latency
• Double Fast – Two hash tables for different match lengths
• Lazy – Evaluates multiple candidates before committing to a match
• Lazy2 – Extended lazy matching with better compression
• Binary Tree (BT) – Maintains sorted binary trees for optimal match discovery
• Optimal (BTOPT/BTULTRA) – Dynamic programming parser for near-optimal compression

The ZSTD_MatchState_t structure (in zstd_compress_internal.h) manages match tables, window state, and strategy-specific data. Row-based match finders use SIMD to accelerate tag comparisons.

Entropy Encoding

Zstd uses two entropy coders working in tandem:

• FSE (Finite State Entropy) – Encodes literal lengths, match lengths, and offsets with adaptive probability tables
• Huffman – Compresses literal bytes directly

The ZSTD_entropyCTables_t structure holds precomputed encoding tables. During decompression, entropy_common.c reconstructs these tables from the compressed frame header.

Decompression Architecture

Decompression mirrors compression but is simpler:

1. Parse frame header and entropy tables
2. Decode blocks sequentially
3. Reconstruct literals and apply match copies

The ZSTD_DCtx maintains state across streaming calls, including window buffers and entropy tables. Dictionary support is handled via ZSTD_DDict, which precomputes entropy tables for faster repeated decompression.

Key Data Structures

• SeqDef – Compact representation of a match: offset, literal length, match length
• SeqStore_t – Accumulates sequences during compression
• ZSTD_window_t – Manages the sliding window and dictionary boundaries
• ZSTD_cwksp – Workspace allocator for dynamic buffers

Compression Engine & Strategies

Zstd implements nine compression strategies, each optimizing for different trade-offs between speed and compression ratio. These strategies are selected based on compression level and input characteristics.

Strategy Overview

The strategies range from fastest to strongest:

1. ZSTD_fast - Hash table lookup with single match per position. Minimal overhead, maximum speed.
2. ZSTD_dfast - Double hash table (small and large) for better match quality at modest speed cost.
3. ZSTD_greedy - Hash chain search, takes first match found above minimum length.
4. ZSTD_lazy - Hash chain with one-position lookahead; compares current match against next position's match.
5. ZSTD_lazy2 - Extended lookahead (two positions) for better match selection.
6. ZSTD_btlazy2 - Binary tree search combined with lazy matching for improved match finding.
7. ZSTD_btopt - Optimal parsing using binary trees; evaluates multiple match sequences.
8. ZSTD_btultra - Enhanced optimal parsing with fractional-bit cost calculations.
9. ZSTD_btultra2 - Maximum compression with pre-analysis phase for entropy statistics.

Compression Level Mapping

The clevels.h header defines parameter presets for each compression level (1–22), selecting appropriate strategies based on input size:

/* Example: Level 1 uses ZSTD_fast, Level 6 uses ZSTD_lazy,
   Level 16 uses ZSTD_btopt, Level 22 uses ZSTD_btultra2 */

Parameters include window size (W), hash log (C), chain log (H), search depth (S), lazy depth (L), and target length (T).

Match Finding Mechanisms

Hash Table Strategies (fast, dfast, greedy):

• Use hash tables to index positions in the input
• Fast lookup but limited match exploration
• Suitable for real-time compression

Hash Chain Strategies (lazy, lazy2):

• Follow chains of positions with identical hash values
• Configurable search depth for quality vs. speed trade-off
• Better match quality than hash tables

Binary Tree Strategies (btlazy2, btopt, btultra, btultra2):

• Organize matches in binary trees sorted by content
• Enable comprehensive match exploration
• Support optimal parsing for best compression

Optimal Parsing (btopt, btultra, btultra2)

The optimal parser evaluates multiple encoding choices using dynamic programming. It calculates the cost of each potential match sequence using entropy-based pricing:

/* Cost = bits needed to encode literals + match lengths + offsets */

btultra2 adds a pre-analysis phase that scans the first block to gather entropy statistics, improving cost estimates for subsequent blocks.

Configuration Parameters

Each strategy uses compression parameters:

• windowLog - Match distance window size (10–31 bits)
• hashLog - Hash table size for fast/dfast strategies
• chainLog - Chain table size for lazy/binary tree strategies
• searchDepth - Maximum comparisons per position
• targetLength - Desired match length to stop searching

Higher values improve compression but increase memory and CPU usage.

Entropy Coding: FSE & Huffman

Zstandard uses two complementary entropy coding schemes: Finite State Entropy (FSE) and Huffman coding (HUF). These algorithms compress data by assigning variable-length codes to symbols based on their frequency distribution.

Finite State Entropy (FSE)

FSE is a modern entropy coder that uses a state machine approach. Unlike traditional Huffman coding, FSE maintains a state value that transitions based on encoded symbols, enabling more efficient compression.

Compression Pipeline:

1. Count symbols from input data
2. Normalize frequencies to sum to a power of 2 (2^tableLog)
3. Write normalized counts to output header using compact encoding
4. Build compression table (CTable) that maps symbols to state transitions
5. Encode data by processing symbols in reverse order, updating state and emitting bits

Key Implementation Details:

• Table size is configurable via tableLog (default 13, max 15)
• Symbols are spread across the state table using a step function to optimize cache locality
• Low-probability symbols (frequency <= 1) are stored in a separate high-threshold area
• Encoding uses two parallel states to maximize bit container utilization
• Decoding reverses the process: read normalized counts, build decode table, then decode symbols

Huffman Coding (HUF)

Huffman coding builds a binary tree where frequent symbols get shorter codes. Zstandard's implementation uses a canonical Huffman tree for efficient decoding.

Compression Pipeline:

1. Count symbol frequencies
2. Build Huffman tree using a priority queue (sorted by frequency)
3. Determine optimal table depth (max code length)
4. Write tree weights to output (compressed using FSE)
5. Encode data using 1X (single stream) or 4X (four parallel streams) variants

Key Implementation Details:

• Table log ranges from 11 to 12 bits (max 12)
• Weights are compressed using FSE for compact header storage
• Supports two decoding strategies: X1 (single symbol per lookup) and X2 (two symbols per lookup)
• 4X variant splits input into 4 independent streams for parallel decompression
• Decoder selection is automatic based on input size heuristics

FSE vs. Huffman Trade-offs

Integration in Zstandard

Both codecs are used strategically:

• FSE compresses literals and match lengths in most cases
• Huffman is selected when it provides better compression for literals
• FSE also compresses Huffman tree weights, creating a nested entropy structure
• Codec selection is automatic based on compression heuristics

Loading diagram...

Decompression & Block Processing

Zstandard decompression reverses the compression pipeline by reconstructing data from compressed blocks. The process involves frame parsing, block type handling, literal decoding, and sequence execution.

Block Types

Zstandard frames contain multiple blocks, each with a 3-byte header encoding the block type:

• Raw Block (bt_raw): Uncompressed data copied directly to output
• RLE Block (bt_rle): Single byte repeated N times, efficient for repetitive data
• Compressed Block (bt_compressed): Contains literals and sequences requiring full decompression
• Reserved (bt_reserved): Invalid, indicates corrupted data

The block header uses little-endian encoding: bit 0 is the last-block flag, bits 1-2 encode the type, and bits 3-23 encode the size.

Literals Decoding

Compressed blocks begin with a literals section. Literals can be stored in four ways:

1. Raw Literals (set_raw): Uncompressed bytes
2. RLE Literals (set_rle): Single byte repeated
3. Compressed Literals (set_compressed): Huffman-encoded with tree description
4. Treeless Literals (set_repeat): Huffman-encoded reusing previous tree

The ZSTD_decodeLiteralsBlock() function handles allocation and decompression. For Huffman-compressed literals, it calls either HUF_decompress1X_* (single stream) or HUF_decompress4X_* (four parallel streams) depending on literal size. Literals are buffered strategically to avoid overwriting the decompression window.

Sequence Decoding

After literals, the sequences section contains encoded match instructions. Each sequence specifies:

• Literal Length (LL): Bytes to copy from literals
• Match Length (ML): Bytes to copy from history
• Offset (OF): Distance back in history

FSE (Finite State Entropy) decoding tables for LL, ML, and OF are built from the sequence header or reused from previous blocks. The ZSTD_decodeSeqHeaders() function parses these tables. Sequences are decoded via ZSTD_decodeSequence(), which reads FSE-encoded symbols and applies base values plus additional bits.

Sequence Execution

The ZSTD_decompressBlock_internal() function executes sequences by:

1. Copying literal bytes to output
2. Copying match bytes from history at the specified offset
3. Updating repeat offsets for future sequences

Multiple implementations exist: ZSTD_decompressSequences_default() for general use, and optimized variants with prefetching and BMI2 support. The decoder maintains a sliding window of previous data for match copying.

Memory Management

Decompression contexts (ZSTD_DCtx) manage:

• Entropy tables: FSE and Huffman decoding tables
• Literal buffer: Strategically placed to avoid window conflicts
• Repeat offsets: Last three match offsets for repeat codes
• Workspace: Temporary space for table building

The ZSTD_allocateLiteralsBuffer() function intelligently places literals in the output buffer, extra buffer, or split between both to maximize efficiency while preserving the decompression window.

Loading diagram...

Dictionary Building & Training

Zstd dictionaries improve compression of small, similar files by capturing common patterns across a dataset. The dictionary builder module provides multiple algorithms to automatically generate optimized dictionaries from sample data.

Why Dictionaries Matter

Dictionaries are essential for compressing small files that individually lack repetition. When compressing many similar files (JSON records, logs, configuration files), a trained dictionary can find patterns across samples that a single file cannot. This dramatically improves compression ratios, especially for files under 100KB.

Dictionary Structure

A zstd dictionary has two components:

1. Header - Contains magic number, dictionary ID, and entropy tables (FSE and Huffman statistics)
2. Content - Raw bytes representing common patterns found in training samples

The header allows zstd to optimize encoding by pre-loading frequency tables, reducing frame overhead. Raw content dictionaries (just bytes, no header) are also valid but less efficient.

Training Algorithms

The module provides three training approaches:

ZDICT_trainFromBuffer() - Simple entry point that redirects to fastCover with default parameters (d=8, steps=4, f=20, accel=1). Requires ~6 MB memory.

COVER Algorithm - Segment-based approach from the paper "Effective Construction of Relative Lempel-Ziv Dictionaries" (Liao et al., 2016). Divides samples into epochs and selects representative segments. Parameters:

• k - Segment size (16-2048+)
• d - D-mer size (6-16)
• steps - Optimization iterations (higher = better but slower)

fastCover Algorithm - Optimized variant using frequency hashing instead of suffix arrays. Faster than COVER with comparable quality. Additional parameters:

• f - Frequency array size (log scale, 0-31)
• accel - Speed vs. accuracy tradeoff (1-10)

Dictionary Finalization

ZDICT_finalizeDictionary() converts raw content into a proper zstd dictionary by adding headers and entropy tables. It requires representative samples to compute accurate statistics for the target compression level.

Practical Guidelines

Dictionary Size - Aim for ~100 KB (zstd CLI default is 110 KB). Smaller dictionaries save memory; larger ones rarely improve beyond 100 KB.

Sample Count - Provide ~100x the dictionary size in total samples. A few thousand samples typically suffice. More samples improve quality but slow training.

Effectiveness - Test with zstd -b1e3 -r /path/to/files -D /path/to/dict to benchmark compression gains.

Retraining - Retrain when data patterns change significantly. Dictionary effectiveness naturally decays over time as your data evolves.

Implementation Details

The builder uses suffix arrays (via divsufsort) to find repeated patterns. It scores segments by compression savings and selects the best combination. Multi-threaded optimization tries parameter combinations in parallel to find the best k/d pair.

// Simple training example
size_t dictSize = ZDICT_trainFromBuffer(
    dictBuffer, dictBufferCapacity,
    samplesBuffer, samplesSizes, nbSamples
);
if (ZDICT_isError(dictSize)) {
    // Handle error - likely insufficient samples
}

Error Handling

Training fails gracefully when:

• Too few samples provided
• Samples too small (minimum 8 bytes)
• Data is incompressible or uniform

The module provides ZDICT_isError() and ZDICT_getErrorName() for error checking.

Streaming & Advanced APIs

Streaming compression and decompression enable processing of data in chunks without loading entire files into memory. This is essential for handling large datasets, network streams, and real-time compression scenarios.

Core Streaming APIs

ZSTD_compressStream2() is the primary streaming compression function. It accepts input and output buffers with position tracking, allowing incremental compression:

size_t ZSTD_compressStream2(ZSTD_CCtx* cctx,
                            ZSTD_outBuffer* output,
                            ZSTD_inBuffer* input,
                            ZSTD_EndDirective endOp);

The ZSTD_inBuffer and ZSTD_outBuffer structures track data positions:

typedef struct {
  const void* src;    // start of input buffer
  size_t size;        // buffer size
  size_t pos;         // current position (updated by zstd)
} ZSTD_inBuffer;

typedef struct {
  void* dst;          // start of output buffer
  size_t size;        // buffer size
  size_t pos;         // current position (updated by zstd)
} ZSTD_outBuffer;

ZSTD_decompressStream() mirrors compression for decompression:

size_t ZSTD_decompressStream(ZSTD_DStream* dctx,
                             ZSTD_outBuffer* output,
                             ZSTD_inBuffer* input);

Flush Directives

The ZSTD_EndDirective enum controls flushing behavior:

• ZSTD_e_continue – Accumulate data for optimal compression; no guarantee of output
• ZSTD_e_flush – Flush current block; frame continues; allows immediate decoding
• ZSTD_e_end – Finalize frame; no further data accepted until new frame starts

Multithreaded Compression

Enable parallel compression by setting ZSTD_c_nbWorkers parameter:

ZSTD_CCtx_setParameter(cctx, ZSTD_c_nbWorkers, 4);

When nbWorkers >= 1, ZSTD_compressStream2() becomes non-blocking: it distributes work to threads and returns immediately, allowing caller to continue. The return value indicates remaining data to flush.

Thread Pool Management

For applications with multiple compression contexts, share a thread pool to limit resource usage:

ZSTD_threadPool* pool = ZSTD_createThreadPool(numThreads);
ZSTD_CCtx_refThreadPool(cctx, pool);
// ... use cctx ...
ZSTD_freeThreadPool(pool);

This prevents thread explosion when multiple contexts use multithreading independently.

Advanced Buffer Modes

Experimental parameters optimize memory usage:

• ZSTD_c_stableInBuffer – Input buffer remains at same address; zstd avoids copying to internal window
• ZSTD_c_stableOutBuffer – Output buffer size never grows; zstd writes directly without intermediate buffering

These require strict adherence to buffer stability contracts but reduce memory overhead.

Practical Streaming Pattern

ZSTD_CCtx* cctx = ZSTD_createCCtx();
ZSTD_inBuffer input = {data, dataSize, 0};
ZSTD_outBuffer output = {compressed, compressedSize, 0};

while (input.pos < input.size) {
    ZSTD_compressStream2(cctx, &output, &input, ZSTD_e_continue);
    // write output.pos bytes from compressed buffer
    output.pos = 0;  // reset for next iteration
}

// Finalize
size_t remaining = 1;
while (remaining) {
    remaining = ZSTD_compressStream2(cctx, &output, &input, ZSTD_e_end);
    // write output.pos bytes
    output.pos = 0;
}

Decompression follows the same pattern with ZSTD_decompressStream(), automatically handling frame boundaries and dictionary attachment.

Command-Line Interface & Tools

The Zstandard CLI is a comprehensive command-line tool for compression, decompression, benchmarking, and dictionary training. It supports multiple compression formats and provides extensive configuration options for advanced users.

Architecture Overview

Loading diagram...

Operation Modes

The CLI supports five primary operation modes, determined during argument parsing:

1. Compression (zom_compress) - Default mode; compresses files using specified parameters
2. Decompression (zom_decompress) - Decompresses zstd and other supported formats
3. Benchmarking (zom_bench) - Measures compression speed and ratio across compression levels
4. Dictionary Training (zom_train) - Generates optimized dictionaries from training data
5. File Listing (zom_list) - Displays metadata about compressed files

Argument Parsing Strategy

The main function iterates through command-line arguments, handling both short (-x) and long (--long-word) options. Key parsing features:

• Aggregated short options: -b1e18i1 is equivalent to -b1 -e18 -i1
• Symlink detection: Program behavior changes based on invocation name (e.g., zstdcat, gzip, xz)
• File list handling: Supports stdin (-), file arguments, and --filelist for batch processing
• Parameter validation: Compression levels, thread counts, and memory limits are validated during parsing

File I/O Abstraction

The FIO_prefs_t and FIO_ctx_t structures encapsulate compression preferences and execution context:

• Preferences (FIO_prefs_t): Compression type, checksums, sparse writes, memory limits, threading
• Context (FIO_ctx_t): File count tracking, stdin/stdout detection, progress reporting

Functions like FIO_compressFilename() and FIO_decompressMultipleFilenames() handle the actual file operations, delegating to the compression library.

Benchmarking Module

The benchmark mode (benchzstd.c) measures compression performance:

• Loads files into memory to eliminate I/O overhead
• Tests compression levels from -b (start) to -e (end)
• Generates synthetic data if no files provided
• Reports ratio, speed (MB/s), and memory usage

Dictionary Training

Dictionary training supports three algorithms:

• COVER: Optimal but slower; parameters k and d control dictionary structure
• FastCOVER: Faster variant with acceleration parameter f
• Legacy: Simple selectivity-based approach

Training output is written to a file (default: dictionary) for use in compression/decompression.

Symlink Shortcuts

Invoking zstd through symlinks triggers format-specific behavior:

• zstdmt - Automatic multithreading
• zcat, zstdcat - Decompress to stdout
• gzip, gunzip, gzcat - Gzip format (if enabled)
• xz, unxz, lzma, unlzma - LZMA formats (if enabled)
• lz4, unlz4 - LZ4 format (if enabled)

Compilation Variants

The CLI can be compiled with different feature sets via Makefile variables:

• ZSTD_NOCOMPRESS - Compression-only binary
• ZSTD_NODECOMPRESS - Decompression-only binary
• ZSTD_NOBENCH - Exclude benchmarking
• ZSTD_NODICT - Exclude dictionary builder
• HAVE_THREAD - Enable/disable multithreading
• ZSTD_LEGACY_SUPPORT - Support for legacy zstd formats

Format Specification & Compatibility

Zstandard defines a stable, standardized compression format documented in RFC 8878. The format is independent of CPU type, operating system, and file system, making it suitable for file compression, streaming, and data communications.

Frame Structure

A Zstandard compressed stream consists of one or more independent frames. Each frame contains:

1. Magic Number (4 bytes): 0xFD2FB528 in little-endian format
2. Frame Header (2-14 bytes): Describes frame parameters and optional metadata
3. Data Blocks (variable): One or more compressed or uncompressed blocks
4. Content Checksum (0-4 bytes): Optional xxHash-64 checksum for corruption detection

Multiple frames can be concatenated, and each decompresses independently. The decompressed content of concatenated frames is simply the concatenation of each frame's decompressed content.

Block Types

Each block has a 3-byte header specifying its type and size:

• Raw Block: Uncompressed data stored as-is
• RLE Block: Single byte repeated multiple times (efficient for repetitive data)
• Compressed Block: Zstandard-compressed data with literals and sequences
• Reserved: Invalid; indicates corrupted data

Block size is limited to the smaller of Window_Size or 128 KiB, ensuring decoders can allocate fixed buffers.

Compressed Block Structure

Compressed blocks contain two sections:

1. Literals Section: Raw bytes to be copied during decompression, optionally Huffman-encoded
2. Sequences Section: Commands specifying literal lengths, match offsets, and match lengths, encoded with FSE

Legacy Format Support

Zstandard maintains backward compatibility with older format versions (v0.1 through v0.7) through the legacy decoder infrastructure:

ZSTD_isLegacy()              // Detect legacy format version
ZSTD_decompressLegacy()      // Decompress legacy frames
ZSTD_getDecompressedSize_legacy()  // Extract size from legacy headers

The ZSTD_LEGACY_SUPPORT compile-time flag controls which legacy versions are included. By default, versions 1-8 are supported, allowing seamless decompression of older compressed data.

Entropy Encoding

Two entropy codecs work together:

• FSE (Finite State Entropy): Encodes sequence metadata (literal lengths, match lengths, offsets)
• Huffman Coding: Compresses literal bytes with optional tree descriptions

FSE bitstreams are read backward (from end to beginning), while Huffman streams can use 1 or 4 parallel streams for faster decompression.

Window Size and Memory

The Window_Descriptor specifies minimum memory requirements:

windowLog = 10 + Exponent
windowBase = 1 << windowLog
windowAdd = (windowBase / 8) * Mantissa
Window_Size = windowBase + windowAdd

Minimum window size is 1 KB; maximum is 3.75 TB. Recommended decoder support is 8 MB for broad compatibility.

Dictionary Support

Frames can reference a dictionary via an optional Dictionary_ID field (0-4 bytes). Dictionaries enable better compression for small data by providing pre-trained probability tables and Huffman trees. Dictionary ID 0 means no specific dictionary is required.

Skippable Frames

Skippable frames (magic: 0x184D2A5?) allow embedding custom metadata in compressed streams. Decoders simply skip these frames and resume normal decompression, enabling watermarking and tracking without breaking compatibility.

Build System & Integration

Overview

Zstandard supports multiple build systems to accommodate diverse development and deployment environments. The primary build system is Make, with CMake and Meson as modern alternatives. This flexibility allows integration into various projects and platforms.

Make-Based Build System

The Make system is the traditional and most widely-used approach. The root Makefile orchestrates builds across three main components:

Library Build (lib/Makefile):

• Generates both static (libzstd.a) and dynamic (libzstd.so) libraries
• Supports modular compilation via feature flags: ZSTD_LIB_COMPRESSION, ZSTD_LIB_DECOMPRESSION, ZSTD_LIB_DICTBUILDER
• Handles multithreading: dynamic library defaults to multithreaded, static to single-threaded
• Platform-specific handling for macOS (.dylib), Windows (.dll), and POSIX systems

CLI Tool Build (programs/Makefile):

• Produces multiple variants: zstd, zstd-small, zstd-compress, zstd-decompress
• Auto-detects optional dependencies: zlib, lzma, lz4, pthread
• Supports symlink shortcuts (zstdmt, zcat, gzip, xz, etc.) for compatibility

Test & Examples:

• Comprehensive test suite in tests/Makefile
• Example programs in examples/Makefile
• Contrib projects (pzstd, seekable format, etc.)

Build Targets

Common Make targets:

make lib              # Build static and dynamic libraries
make lib-mt           # Force multithreaded build
make lib-nomt          # Force single-threaded build
make zstd             # Build CLI tool
make test             # Run comprehensive test suite
make check            # Quick validation tests
make install          # Install libraries and headers
make clean            # Remove build artifacts

CMake Integration

CMake provides a modern, cross-platform alternative. The root CMakeLists.txt delegates to build/cmake/:

cmake -S build/cmake -B cmakebuild
cmake --build cmakebuild
cmake --install cmakebuild

CMake supports the same modular options as Make and integrates with IDEs (Visual Studio, Xcode, etc.).

Meson Build System

Meson offers fast, parallel builds with clear dependency tracking:

meson setup build/meson mesonbuild
ninja -C mesonbuild
meson install -C mesonbuild

Modular Compilation

The library can be customized by disabling features:

• ZSTD_LIB_COMPRESSION=0 – Decompression only
• ZSTD_LIB_DECOMPRESSION=0 – Compression only
• ZSTD_LIB_DICTBUILDER=0 – No dictionary builder
• ZSTD_LIB_MINIFY=1 – Minimize binary size

Installation & Multiarch Support

Standard GNU installation conventions are supported:

make install PREFIX=/usr LIBDIR=/usr/lib/x86_64-linux-gnu

The build system generates libzstd.pc for pkg-config integration, enabling easy dependency resolution in downstream projects.

Cross-Compilation & Testing

The Makefile includes extensive cross-compilation targets:

• ARM, ARM64, PowerPC builds with QEMU testing
• Sanitizer builds (ASAN, MSAN, TSAN, UBSAN)
• Multiple compiler versions (GCC 5–8, Clang)
• C standard compliance testing (C89, C99, C11)

Platform-Specific Considerations

Windows: MinGW+MSYS support creates DLLs with import libraries. Visual Studio projects are available in build/VS* directories.

macOS: Uses .dylib extension with compatibility versioning. Fat binary support for ARM64 and x86_64.

POSIX: Full pthread support with automatic detection. Multiarch library paths for Debian/Ubuntu systems.