A post on Lobsters AI — linked from a blog by Nathan — is exploding right now in AI underground circles, and the question it raises is deceptively simple: can gzip be a language model?

Spoiler: yes, in a way — and it should make you think.

Here's the thing. Compression and language prediction are really two sides of the same coin. When gzip compresses text, it looks for patterns and repetitions within a sliding window of recently seen data. What compresses well is what is statistically predictable. What compresses poorly is what is unexpected. That is exactly what a language model does — except gzip does it without training data, parameters, or GPU bills.

An algorithm from 1992 implicitly does what billion-parameter models are trained for months to learn.

What really made people raise their eyebrows was a research paper (Jiang et al.) that used gzip combined with Normalized Compression Distance and kNN for text classification — entirely without machine learning parameters. On low-resource languages such as Kinyarwanda, Kirundi, and Swahili, the method outperformed large neural networks. Not because gzip is clever, but simply because the LLMs did not have enough training data in those languages.

gzip beats GPT: Old zip algorithm holds its own as a language model - Bilde 1

Is this perfect? Absolutely not. On large, well-defined datasets like YahooAnswers, the gzip method drops around 7 percentage points behind neural alternatives. It does not understand semantics, synonyms, or context at any deep level. And the kNN approach scales poorly — O(n²) complexity is no fun in production.

But that is not the point. What makes this discussion interesting is what it says about our assumptions. We tend to assume that intelligent results require complex systems. The gzip experiment reminds us that much of what we call "understanding" in LLMs may be extremely sophisticated pattern recognition — and that simple, well-defined pattern matchers already exist.

For people working with low-resource languages, edge deployments without access to large models, or who simply appreciate elegant minimalism in system design — this is a signal worth paying attention to.

Keep in mind that this is an early signal from community sources, not peer-reviewed consensus. But the discussion is real, and it points to something fundamental that the major AI labs rarely say out loud: perhaps we do not always need more scale. Sometimes an old-fashioned zip is enough.