Zstandard file format
Zstandard-compressed data, a fast modern compression format from Meta.
- MIME type
- application/zstd
- Extensions
- .zst
- Category
- Archives & compression
Magic-number signature
| 0x00000000 | 28B52FFD | (·/· | Zstandard magic |
About the Zstandard format
Zstandard delivers gzip-class ratios at much higher speed with tunable levels, now widely adopted in package managers and filesystems. Its little-endian magic is 28 B5 2F FD. A .zst file is a sequence of frames, and each frame begins with a one-byte descriptor whose bits decide which of the following fields are present at all — so unlike gzip, whose header is a fixed ten bytes, a Zstandard header is between six and eighteen bytes long and no offset past the fifth byte is fixed. Reading one by hand means decoding that descriptor first; there is no shortcut.
Header layout, byte by byte
| Offset | Length | Field | Meaning |
|---|---|---|---|
| 0 | 4 | Magic number | 28 B5 2F FD on disk. The specification writes it as 0xFD2FB528 because every multi-byte value in Zstandard is little-endian, so the number and the byte order look reversed from one another. |
| 4 | 1 | Frame header descriptor | The byte everything else depends on. Bits 7-6 give the frame content size field's width, bit 5 is the single-segment flag, bit 4 is unused, bit 3 is reserved and must be 0, bit 2 turns on the trailing content checksum, and bits 1-0 give the dictionary id width. A2 and A4 are common values from the command-line tool. |
| 5 | 1 | Window descriptor | Present only when the single-segment flag at bit 5 of the descriptor is clear. It encodes the window size a decoder must buffer. When the flag is set — as it is whenever the whole input fits in one segment — this byte does not exist and the content size starts here instead. Getting this conditional wrong is the classic zstd parsing bug. |
| variable | 0-4 | Dictionary id | Zero, one, two or four bytes, chosen by the low two bits of the descriptor. A non-zero value means the frame was compressed against a shared dictionary and cannot be decoded without it, however intact the bytes are. |
| variable | 0-8 | Frame content size | The decompressed length, little-endian, in zero, one, two, four or eight bytes. A 200000-byte input is stored as 40 0D 03 00 and a 48-byte input as a single byte 30. It is absent entirely when the encoder was streaming and did not know the length in advance. |
| variable | 3 | Block header | A 24-bit little-endian field starting each block: bit 0 marks the last block, bits 1-2 give the type (0 raw, 1 RLE, 2 compressed, 3 reserved) and bits 3-23 the size. Incompressible input produces raw blocks — 00 00 10 is a 131072-byte raw block — while 6D 00 00 is a final compressed block of 13 bytes. |
| -4 | 4 | Content checksum | The low 32 bits of an XXH64 over the original data, present only when bit 2 of the frame header descriptor is set. It is on by default, and a file written with the check disabled simply ends with the last byte of the final block instead. |
Telling it apart from its neighbours
| Easily confused with | How to tell |
|---|---|
| skippable frames | A .zst does not have to begin with 28 B5 2F FD at all. Skippable frames use magic numbers 0x184D2A50 through 0x184D2A5F, stored little-endian as 50 2A 4D 18 through 5F 2A 4D 18, followed by a four-byte little-endian size; decoders jump over them and applications use them to embed their own metadata. So a detector that insists on the compressed-frame magic at offset 0 will reject perfectly valid files. |
| XZ | The direct rival for the same jobs, and the comparison behind most 'which should I use' searches. FD 37 7A 58 5A 00 against 28 B5 2F FD. XZ still wins on ratio at maximum settings; Zstandard wins enormously on speed at both ends and holds its own at level 19. |
| GZIP | 1F 8B against 28 B5 2F FD. Zstandard was designed explicitly to match gzip's ratio at several times the speed, so they occupy the same niche and get swapped in place — which is exactly how a .zst ends up named .gz in a build script. |
| ZIP | ZIP compression method 93 is Zstandard, so the magic can appear inside a ZIP entry rather than at offset 0. Support for it outside a few writers is thin, so such archives frequently fail to open in tools that read every other ZIP fine. |
| TAR | A .tar.zst is the ordinary shape for package payloads. Nothing in the frame distinguishes it from a compressed single file — decompress and check offset 257 of the output for 'ustar'. |
How these files break
- Truncation with the length still declared. If the frame content size was present in the header, tools report the full expected output size and then fail partway with an unexpected end of frame, so the numbers look right up until the moment the data stops.
- A frame written with the checksum disabled. The trailing four bytes are simply absent, so nothing verifies the output; corruption inside a raw block then produces wrong bytes with no error at all, because raw blocks are copied verbatim and have no entropy coding to derail.
- A missing dictionary. When the dictionary id field is non-zero the frame is undecodable without that exact dictionary, even though every byte on disk is intact. It fails with a dictionary mismatch rather than a corruption error, which sends people looking for damage that is not there.
- Concatenated frames read as one. Joining .zst files is legal and the standard tool decompresses all frames in sequence; a reader that stops after the first frame's last block silently truncates the result, and skippable frames between them make it worse.
- Text-mode transfer. The header survives, the declared content size still reads correctly, and decoding fails inside the first compressed block — so the file identifies as valid Zstandard right up until it is used.
Where the format stands today
Zstandard has become the default general-purpose compressor of the last few years and is still spreading. Arch and Fedora packages, Debian package payloads, btrfs and ZFS transparent compression, container image layers, Kafka and RocksDB internals, and the compression Linux uses for its own initramfs images all sit on it, and it is a published internet standard in RFC 8878 rather than a single project's format. The reason is the shape of its trade-off curve rather than any single number: at its default level it roughly matches gzip's ratio while compressing several times faster, and decompression is fast and nearly level-independent, so choosing a higher level costs the writer and never the readers. It also has dictionary support, which no other common format offers, and that transforms compression of many small similar payloads. Where it still loses is maximum ratio for archival, where xz and LZMA remain ahead, and long-tail compatibility, where gzip remains unbeatable.
Open Zstandard files on this site
Every one of these runs in your browser — the file never leaves your machine.
Related formats
Formats that are easy to confuse with Zstandard — compare their leading bytes.