Can gzip Be a Language Model? Compression, Prediction, and the Limits of the Experiment
Gzip outperformed some LLMs on next-token prediction by exploiting compression-prediction equivalence. But its 32KB window and lack of abstraction reveal why it