pigz is the parallel implementation of gzip, written by Mark Adler in 2007, and the name is simply the first letters of that phrase. It slices the input into blocks of 128 KB, hands them to several cores at once, and writes the result back as one standard gzip file. The version sitting on the official source page is 2.8, dated 19 August 2023.

After 2.8, the version number stopped for three years

The source page carries the line pigz source code version 2.8 (19 Aug 2023), with a tarball of 118K beside it, next to a SHA-256 checksum and a GPG signature. The commits in the code repository also stop in August 2023, and the message on that last one is a single line: version 2.8.

Three years without a release does not mean a stalled project. The DEFLATE format was settled long ago, decompressors exist everywhere, and a tool whose only job is spreading compression over cores has no reason to change as long as what goes in and what comes out stay the same. Version 2.8 touched details: writing the modification time only on ordinary files, accepting the two part zlib version number, and a few repairs around assertions and null pointers. Once that kind of patch is done, the feature surface is closed.

A block of 128 KB, compressed, then stitched back

The method is not roundabout. The input is cut into 128 KB blocks, each is compressed on its own, and the checksum of each is computed in parallel along the way. A writing thread joins the finished pieces in the original order, merges the block checksums into one at the end, and wraps a gzip header and trailer around the lot. What comes out is the sort of file any decompressor recognises, with nothing to show that it was compressed in pieces.

Compressing in blocks usually costs some ratio, since a later block cannot see the context of an earlier one. pigz compensates: before a block starts, the last 32K of the previous block is fed in as a preset dictionary, which is the same history window a single threaded deflate would have had for free. Each block has to end on a byte boundary when the pieces are joined, which adds four or five bytes of overhead; against a block of over a hundred kilobytes, that disappears.

Two knobs: thread count and block size

The number of compression threads follows the online processors by default, one per core, and -p changes it; writing -p 1 removes threading entirely and returns to the plain gzip path. Block size defaults to 128K and -b changes it. Larger blocks help the ratio and hurt parallelism; smaller blocks fill the cores better and surrender a little on both ratio and overhead. These two are the main means of trading speed against size.

It asks very little of the environment. The official README says it depends only on zlib and pthread, with no third party code tucked inside, which is why it builds on every flavour of Unix. zlib must be at least 1.2.3, and the maker prefers something after 1.2.6, because the overhead between blocks is squeezed smaller there.

Decompression does not parallelise

A deflate stream locks together front and back, so a decoder must walk it in order, unless the compressor split the stream into blocks that can each be decoded alone. So pigz decompresses with a single main thread and starts three others for reading, writing and checksum work. That can be faster in some situations, but it is not spreading decompression across every core. To force everything single threaded, write -dp 1 or -tp 1.

This explains a common misunderstanding: a file compressed with pigz does not decompress any faster. The time saved on the compressing side cannot be recovered on the other. Both sides gain only if the compressor used the independent block mode described next.

What -i buys is random access and tolerance

With -i or --independent, blocks no longer reference each other's dictionaries and each can be decrypted on its own. Two benefits follow: if one block of data is damaged, the rest still reads, and jumping straight into the middle of a file needs no decoding from the start. So a decoder can find the boundaries, nine fixed bytes are inserted before each block, which in hexadecimal are 00 00 FF FF 00 00 00 FF FF. The price is a ratio back at the level of plain block compression, with the preset dictionary compensation gone.

From -0 to -11

Compression levels match gzip: -0 stores without compressing, -9 squeezes hardest and slowest, and the default lands on -6. pigz adds a -11, which runs Jyrki Alakuijala's zopfli algorithm over the data repeatedly, pulling out a few more percent at a severe cost in execution time, to use the manual's own phrase. Around -11 sits a group of dedicated options: fifteen iterations by default, fifteen maximum blocks by default, and the choice either not to split at all or to iterate first and split after.

The output format is one of three: gzip by default, zlib's .zz with -z, a single entry zip with -K. There is also -R, which makes the split points follow the content, so that after a small edit the unchanged parts land in the same places and an incremental transfer carries only what moved. Bulk log compression and backup jobs, where a large batch goes through at once, are where blocks and cores really pay for themselves.

A licence of three sentences

pigz uses the permissive zlib licence: anyone may use the software for any purpose, change it, redistribute it. Only three things are asked. Do not claim you wrote it, mark altered versions, and leave the notice in place. Mark Adler is also one of the two authors of zlib, and the zlib site says he owns the decompression half; the block and dictionary trick in pigz comes from the same drawer as the library he keeps.

The official page explains how to say the name: pig-zee, not the plural of pig. Finding that question answered on the front page says something about the temperament of the project. The documentation is one page, the options live in the manual, and the rest is left to the source.

Spreading compression over cores is the only thing pigz sells, and it is also why twenty years have needed only small repairs: the gzip format has not changed, so what the tool can do has not changed either. The honest complaint is that decompression is still one thread, and the speed is all on the compressing end.