Compression in Malware: The Detection Gap

When a malware analyst encounters a Base64-decoded binary blob, the first check is compression magic bytes. GZip starts with 1F 8B. Zlib starts with 78 9C. If neither matches, the standard playbook moves on to XOR, AES, or raw shellcode analysis. But what if the blob starts with 28 B5 2F FD? That's Zstandard. Or 04 22 4D 18? That's LZ4. Most analysis tools don't check for these. Most YARA rules don't match them. Most analysts have never encountered them in a malware context. That's the blind spot. Not a failure of skill or diligence, but a gap in the default toolkit. Security tools were built when GZip and Deflate were the only compression formats that mattered. The world moved on. The tooling, largely, did not.

The Two Formats Everyone Detects

GZip and zlib/Deflate dominate malware compression because they dominate everything else. They're the default in HTTP content encoding, ZIP archives, PDF streams, and PNG image data. Every programming language has built-in support. Every security tool has signatures for their magic bytes.

This universality made them the natural choice for malware authors. Why use an obscure format when the target machine already has the decompression library installed? PowerShell's [IO.Compression.GZipStream] and .NET's System.IO.Compression.DeflateStream are one-liners. The malware compresses its payload with GZip, Base64-encodes it, and decompresses at runtime.

The security industry responded with comprehensive detection. YARA rules match GZip headers. Network inspection devices decompress GZip content inline. Email gateways scan inside compressed attachments. The detection coverage is mature. And that maturity is exactly why the gap matters.

The Formats Nobody Checks

While security tooling focused on GZip and zlib, the software industry adopted a new generation of compression algorithms. These formats offer better compression ratios, faster decompression, or both. They're not experimental. They're in production at scale.

Why the Gap Exists

Detection tooling is reactive. YARA rules get written for compression formats after they're observed in malware samples. Network signatures get added after incidents. The problem is that this reactive cycle has a bootstrapping gap: if nobody detects the format, nobody observes it in malware, so nobody writes detection rules, so nobody detects it.

Meanwhile, the legitimate adoption of these formats ensures that the decompression libraries are already present on target systems. Any Python environment can install zstandard and lz4 in seconds via pip. Any browser can decompress Brotli. Any JavaScript runtime has access to LZString via npm or a CDN. The attacker doesn't need to ship a decompression library with their payload.

The Detection Gap in Numbers

The left side of this table is well-defended territory. The right side is open ground. An attacker switching from [IO.Compression.GZipStream] to Zstandard in their delivery script gains immediate evasion against every tool that relies on magic byte detection.

How Multi-Layer Chains Exploit the Gap

Compression rarely appears alone in malware delivery. It's typically one layer in a multi-layer obfuscation chain. A common pattern: Base64-encode the compressed payload, optionally XOR-encrypt it, then embed it in a script that reverses the chain at runtime.

When the compression layer uses GZip, automated tools handle the full chain: decode Base64, detect GZip magic, decompress, continue analysis. When the compression layer uses Zstandard or LZ4, the chain breaks at the compression step. The tool decodes Base64 successfully, encounters a binary blob with unrecognised magic bytes, and either stops or misclassifies the output.

The analyst sees "decoded binary data" and may assume it's encrypted payload requiring a key. In reality, it's compressed payload requiring the right decompression algorithm. The intelligence is one library call away, but the tool doesn't know which library to call.

This pattern is especially prevalent in Python loaders, where the standard library exposes zlib, bz2, and lzma without external dependencies. Modern Python obfuscators chain these compression algorithms with exec() and Base64 to produce loaders where each peeled layer reveals another compressed-then-encoded payload. Decoders that recognise GZip but not zlib (raw, no header) or lzma will stall on the first inner layer.

LZString: The Invisible Format

LZString deserves special attention because it breaks every assumption that magic-byte detection relies on.

Most compression formats produce binary output. LZString produces printable character strings. Its compressToBase64 variant outputs valid Base64 characters. Its compressToEncodedURIComponent variant outputs valid URL-encoded characters. A compressed LZString payload looks like a long string of random text, not a binary blob.

This means there are no magic bytes to detect. No binary signature to match. The compressed payload passes through text-based filters, email gateways, and WAF rules without triggering any alert. An eval(LZString.decompress(...)) call in JavaScript looks like a function call with a string argument. The compressed content is invisible to everything except a tool that specifically knows to try LZString decompression.

Preparing for the Shift

The compressed-malware picture will shift. Not because of a specific threat actor announcement, but because the economics are too favourable for attackers to ignore. Switching compression formats is trivial. The evasion benefit is immediate. And the target systems already have the decompression capability installed.

Defenders can prepare now, before the shift becomes a wave. Practical steps:

• Add magic byte rules for Zstandard (28 B5 2F FD) and LZ4 (04 22 4D 18) to YARA rule sets. Even without confirmed malware samples, these rules create early warning for format adoption.
• Ensure your deobfuscation pipeline handles non-GZip compression in multi-layer chains. Base64(Zstd(payload)) should resolve the same way Base64(GZip(payload)) does.
• Monitor for Base64-decoded binary blobs with unrecognised magic bytes. If your tool decodes Base64 and hits unknown binary, check whether it's Zstd, LZ4, or Brotli before assuming encryption.
• Test LZString decompression against JavaScript payloads that contain long string arguments to eval() or Function() calls. The absence of binary data doesn't mean the absence of compression.

The Precedent: How GZip Became Standard

GZip didn't start as a malware technique. It became one because it was everywhere in legitimate software. The same Python stdlib that serves web content was available for compressing payloads. The same .NET class that handles HTTP responses was available for decompressing in a PowerShell dropper.

Zstandard, LZ4, and Brotli are on the same trajectory. They're in CDNs, container runtimes, databases, and web servers. The decompression code is already on the target machine. The only thing preventing widespread adoption in malware is inertia and the fact that GZip still works for most campaigns. When detection pressure on GZip increases sufficiently, the migration will be fast.

The window to build detection before that migration is now.