As I explained in my 2013 paper ( https://mattmahoney.net/costofai.pdf ), the complexity of the economy that AGI ought to automate is on the order of 10^17 bits. To get that you need to train on 100,000 TB of text. Current LLMs are trained on 15 TB of text because that's all you can suck off the Internet. Given that we don't have the technology to scan brains at nm resolution, the only way to get this data is through slow channels like speech and writing at the cognitive limit of 5 to 10 bits per second. So you need to spend on the order of $100 trillion (roughly 1 year global GDP) at the current global average wage rate of $5 per hour. As wages go up, so do your costs.
10^17 bits is the capacity of human long term knowledge (10^9 bits based on long term memory tests for words and pictures) x 10^10 people (world population) x 1% (the fraction of lifetime earnings that it costs to replace an employee, which I used to estimate the fraction of your knowledge that is not known to anyone else or written down). That's 100 PB at a compression ratio of 1 bit per character, or likely more because the compression ratio improves with the size of the data set. This is the reason that AI hasn't replaced your job. The knowledge you need to do your job is not written down. The cost of your time to train your replacement is roughly a year whether your replacement is carbon or silicon based. That cost will go up for higher paying jobs. If somehow we manage to double human knowledge every 3 years, then AGI is 40 years away. On Mon, Sep 16, 2024 at 2:24 PM Alan Grimes via AGI <agi@agi.topicbox.com> wrote: > > The question that seems central to all of this seems to be why the > entire AI industry is so pathetic that OpenAI needs to give it a kick in > the pants every 8 months or so? I mean you can frame this question in > several different ways, and approach it from a dozen different angles. > At root it doesn't seem to make any cents at all but the underlying fact > keeps re-asserting itself like clockwork and demands an explanation.... -- -- Matt Mahoney, mattmahone...@gmail.com ------------------------------------------ Artificial General Intelligence List: AGI Permalink: https://agi.topicbox.com/groups/agi/T11c1e7172d92d3cd-M9799cd2125ab59dd575f8c9e Delivery options: https://agi.topicbox.com/groups/agi/subscription
