I'm criticizing the systems thinking about compression, not compression itself. IMO, the Shannon limit functions as an allegorical lighthouse for compression, but it's seemingly being treated as contextual noise.
Caveat. Currently, compression algorithms resolve locally. What it doesn't resolve for coherently, it discounts as noise. All that contextual information is discounted. Similar to what IBM did in the historical anomalous dataset, which Mandelbrot subsequently identified as core structure and not anomaly. Gell-Mann physically placed the AIS, or optimal entropic data set, within that context. By default human consciousness resolves locally as a closed system. This seems naturally inescapable. However, in theory AI could bypass this human-functional limitation. Hence, the apparent industry obsession with it. What if "noise" was in fact structure, and lossiness proved itself to be a localized systems thinking problem? On Wed, 22 Jul 2026, 23:38 Matt Mahoney, <[email protected]> wrote: > You can't just propose a solution to the biggest problems confronting > humanity and then walk away if you want people to take you seriously. You > propose using algorithmic information theory to evaluate models in > sociology. OK prove it. Give me an example of a question you want to answer > and the data you would compress to solve it. > > I gave an example of predicting future population from historical data. > This actually works pretty well for a decade or two. But you can see the > problems when you go further out. If you have n data points, you can fit an > n degree polynomial exactly, but it has no predictive power because the > coefficients take up as much space as the original data. It overfits, > resulting in wild swings between points. If you start with a low order > polynomial, then each added coefficient adds some bits to store it but > reduces the bits needed to encode smaller residual errors. The break even > point is around log n. Now the leading coefficient is very near 0, hovering > on a fine line between a Kardashev level civilization and human extinction. > > What about adding more data? Claude compressed 20 TB of human knowledge, > so it should be able to reason about fertility rates by country, advances > in medicine, the economy, and so on. For 2100 it cites several forecasts > and says 8.8 to 10.4 billion people. For 2200 it says that's too far into > the future to predict. > https://claude.ai/share/57292ed8-455f-4d73-92a7-8cc9a0c51d56 > > Can you beat that? Or is there another problem you can suggest? > > I'm not attacking AIT. We accept Newton's law of gravity because it uses a > simpler model to explain the motions of the planets than the epicycye model > where the planets go around the Earth but sometimes reverse direction. It > is the foundation of machine learning, explaining the optimal number of > coefficients for fitting a polynomial or the number of parameters in a > neural network. > > But it doesn't work for everything. Compression works great for evaluating > language models but not vision models because the latter is overwhelmed by > noise. It works great for physics but not the social sciences for the same > reason. > > -- Matt Mahoney, [email protected] > > > > On Tue, Jul 21, 2026, 8:33 PM James Bowery <[email protected]> wrote: > >> >> >> On Tue, Jul 21, 2026 at 4:21 PM Matt Mahoney <[email protected]> >> wrote: >> >>> ... >>> So my question is what questions do you want to answer and what data >>> would you compress? >>> >> >> That's not for me to decide, as I lack the authority to determine the >> narratives influencing decisions about whether calls for "optimism" by >> "influencers" are realistic. >> >> However, Forrester is someone whose warnings should not be ignored when >> he says that people routinely cause the problems they're trying to solve. >> > *Artificial General Intelligence List <https://agi.topicbox.com/latest>* > / AGI / see discussions <https://agi.topicbox.com/groups/agi> + > participants <https://agi.topicbox.com/groups/agi/members> + > delivery options <https://agi.topicbox.com/groups/agi/subscription> > Permalink > <https://agi.topicbox.com/groups/agi/T5b58bcc51c493d41-M8dde40c12166e1a7c4bce99b> > ------------------------------------------ Artificial General Intelligence List: AGI Permalink: https://agi.topicbox.com/groups/agi/T5b58bcc51c493d41-M9073f5b6e5ba7cf07467f86c Delivery options: https://agi.topicbox.com/groups/agi/subscription
