Matt, thanks for sharing. Very interesting progress. Any idea on how soon a 
processor-compressor would run as software on an 8 GB notebook computer?

________________________________
From: Matt Mahoney <[email protected]>
Sent: Sunday, 15 November 2026 02:09
To: AGI <[email protected]>; Hutter-Prize <[email protected]>
Subject: [agi] The Hutter explosion

My wife and I just returned from 18 days of exploring ancient sites in Rome and 
Tunisia, hiking the canyon walls of Camino del Rey in Spain, and running the 
Porto marathon in Portugal. We were fortunate that our return flight from 
Madrid to Orlando was the day after a nationwide strike halted travel in Spain 
and the day before the FAA grounded all air traffic in the US because of a 
software error.

I didn't bring my laptop so I spent most of the next day updating the Large 
Text Compression Benchmark on my website with a flurry of Hutter prize 
submissions. The rate of progress since July has been astounding. From 2006 
when I created the benchmark until earlier this year, the record improved from 
127 MB by durilca (a PPM compressor) to 107 MB set by nncp in 2019 using a 
transformer, an average of 2% gain per year. Now another transformer, cmix-neif 
2.0 holds the record (pending testing) at 89 MB. That is 20 years of progress 
in 4 months, a 60 fold speedup. Now nncp sits at 16'th place and durilca at 
31'st among 238 compressors. All of this progress is due to AI generated code 
and (high quality) documentation.

I do not believe in a fast takeoff singularity, but using AI to write AI sure 
looks like one. I just read that Anthropic used 1000 agents on an unreleased 
model to write a complete operating system in 8 days, 1000 times faster than a 
team of humans. I remind you that the cost of human intellectual work is $10K 
per million input tokens and $50K per million output tokens. The most advanced 
models are 1000 times cheaper.

In 1999 I wrote a proposal to NSF for a text compression prize to promote AI 
development. It was rejected, as were 90% of proposals. So I created 
https://mattmahoney.net/dc/text.html in 2006 without prize money until it was 
funded by the Hutter prize. Now I am 71, retired 12 years, and supporting this 
research voluntary because I think that AI is important

One early result was that compression increases logarithmically with computing 
power, as shown in the 2 graphs I made in 2008. That relationship still holds. 
Now the new result is two lineages of transformer based models derived from 
cmix and nncp. Both use offline training by multiple passes of back propagation 
(8-10 GPU days) quantizing 6-8M weights to 4 bits, compressing and appending to 
the compressed file. The cmix lineage uses a 12 layer network and the nncp 
lineage, led by DNA4, uses 5 layers. DNA4 only compresses to 100 MB but is much 
faster, 2.5 GPU days to train and 2.5 GPU hours to model using 14 bit tokens vs 
8 bits and 2 CPU days for cmix.

I still believe there are better algorithms than transformers. The brain learns 
by Hebb's rule, which is purely local, not back propagation. Human learning is 
single pass, not offline in dozens or hundreds of passes. An online neural 
network would eliminate the 3-4 MB penalty for storing the weights in the 
decompressor (or double penalty in the Hutter prize for storing another copy in 
the compressor). Once that happens, the number and precision of the weights 
could be increased to improve compression further. I believe that 75 MB is 
possible if hardware limits are removed.

-- Matt Mahoney, [email protected]<mailto:[email protected]>












Artificial General Intelligence List<https://agi.topicbox.com/latest> / AGI / 
see discussions<https://agi.topicbox.com/groups/agi> + 
participants<https://agi.topicbox.com/groups/agi/members> + delivery 
options<https://agi.topicbox.com/groups/agi/subscription> 
Permalink<https://agi.topicbox.com/groups/agi/T2ce81ade71c4115f-Mbd1bc33f7aad1a4d7573352c>

------------------------------------------
Artificial General Intelligence List: AGI
Permalink: 
https://agi.topicbox.com/groups/agi/T2ce81ade71c4115f-Mb0db5d29fc2747b5df5d5220
Delivery options: https://agi.topicbox.com/groups/agi/subscription

Reply via email to