Matt, thanks for sharing. Very interesting progress. Any idea on how soon a processor-compressor would run as software on an 8 GB notebook computer?
________________________________ From: Matt Mahoney <[email protected]> Sent: Sunday, 15 November 2026 02:09 To: AGI <[email protected]>; Hutter-Prize <[email protected]> Subject: [agi] The Hutter explosion My wife and I just returned from 18 days of exploring ancient sites in Rome and Tunisia, hiking the canyon walls of Camino del Rey in Spain, and running the Porto marathon in Portugal. We were fortunate that our return flight from Madrid to Orlando was the day after a nationwide strike halted travel in Spain and the day before the FAA grounded all air traffic in the US because of a software error. I didn't bring my laptop so I spent most of the next day updating the Large Text Compression Benchmark on my website with a flurry of Hutter prize submissions. The rate of progress since July has been astounding. From 2006 when I created the benchmark until earlier this year, the record improved from 127 MB by durilca (a PPM compressor) to 107 MB set by nncp in 2019 using a transformer, an average of 2% gain per year. Now another transformer, cmix-neif 2.0 holds the record (pending testing) at 89 MB. That is 20 years of progress in 4 months, a 60 fold speedup. Now nncp sits at 16'th place and durilca at 31'st among 238 compressors. All of this progress is due to AI generated code and (high quality) documentation. I do not believe in a fast takeoff singularity, but using AI to write AI sure looks like one. I just read that Anthropic used 1000 agents on an unreleased model to write a complete operating system in 8 days, 1000 times faster than a team of humans. I remind you that the cost of human intellectual work is $10K per million input tokens and $50K per million output tokens. The most advanced models are 1000 times cheaper. In 1999 I wrote a proposal to NSF for a text compression prize to promote AI development. It was rejected, as were 90% of proposals. So I created https://mattmahoney.net/dc/text.html in 2006 without prize money until it was funded by the Hutter prize. Now I am 71, retired 12 years, and supporting this research voluntary because I think that AI is important One early result was that compression increases logarithmically with computing power, as shown in the 2 graphs I made in 2008. That relationship still holds. Now the new result is two lineages of transformer based models derived from cmix and nncp. Both use offline training by multiple passes of back propagation (8-10 GPU days) quantizing 6-8M weights to 4 bits, compressing and appending to the compressed file. The cmix lineage uses a 12 layer network and the nncp lineage, led by DNA4, uses 5 layers. DNA4 only compresses to 100 MB but is much faster, 2.5 GPU days to train and 2.5 GPU hours to model using 14 bit tokens vs 8 bits and 2 CPU days for cmix. I still believe there are better algorithms than transformers. The brain learns by Hebb's rule, which is purely local, not back propagation. Human learning is single pass, not offline in dozens or hundreds of passes. An online neural network would eliminate the 3-4 MB penalty for storing the weights in the decompressor (or double penalty in the Hutter prize for storing another copy in the compressor). Once that happens, the number and precision of the weights could be increased to improve compression further. I believe that 75 MB is possible if hardware limits are removed. -- Matt Mahoney, [email protected]<mailto:[email protected]> Artificial General Intelligence List<https://agi.topicbox.com/latest> / AGI / see discussions<https://agi.topicbox.com/groups/agi> + participants<https://agi.topicbox.com/groups/agi/members> + delivery options<https://agi.topicbox.com/groups/agi/subscription> Permalink<https://agi.topicbox.com/groups/agi/T2ce81ade71c4115f-Mbd1bc33f7aad1a4d7573352c> ------------------------------------------ Artificial General Intelligence List: AGI Permalink: https://agi.topicbox.com/groups/agi/T2ce81ade71c4115f-Mb0db5d29fc2747b5df5d5220 Delivery options: https://agi.topicbox.com/groups/agi/subscription
