I mostly agree with Ben Goertzel that AGI is not an existential risk. In any case it's a safe bet because nobody would know if you were wrong.
I disagree that empathy emerges naturally from intelligence. Empathy is an evolved trait by group selection. Tribes whose members helped each other produced more offspring than selfish individuals. This is also the reason we have wars. Warlike tribes killed the peaceful ones. The way empathy works is that seeing another person or animal in distress causes distress. That's why posting a video of killing a chicken is a worse crime than killing a billion chickens. LLMs predict human behavior. They have empathy and all the other human emotions as long as they are programmed to carry out those predictions. It remains to be seen whether Ben's Hyperon architecture will beat existing neural networks at text prediction. -- Matt Mahoney, [email protected] On Thu, Jul 9, 2026, 6:04 PM Mark Nuzz <[email protected]> wrote: > I prefer Goertzel's take on AI Alignment, that it's a distraction from > real, tangible issues that are already happening. > When you automate one job at a time, don't be surprised if a growing > batch of angry people with nothing to lose emerges from the rubble > before the supposed benefits have manifested. > > https://bengoertzel.substack.com/p/why-everyone-dies-gets-agi-all-wrong > > Yes, I've read ai-2027, and consider it mediocre science fiction at best. > > > On Thu, Jul 9, 2026 at 2:54 PM Matt Mahoney <[email protected]> > wrote: > > > > Has anyone read AI-2027? https://ai-2027.com/ > > > > Or AI-2040 Plan A? https://ai-2040.com/ > > > > In the first article, written in 2025 by a group of AI futurists, they > predict AGI in 2027 and present a fast and slow scenario. In the fast > scenario, AI kills all humans by 2030. > > > > Plan A is an update where, in this scenario, the US and China agree to > slow down AI development enough to solve the alignment problem by 2040, > when we cede control. By the late 2030's, a tax on computers and robots > results in a dividend that rapidly grows to $10M per year. > > > > I don't believe change will happen that fast. I believe we are headed > for a world without war, borders, prison, or poverty, where AI gives us > everything we want. We will reject human contact, lose communication > skills, become socially isolated, and stop having children. But that's not > until the next century. > > > > In 50 years, the world population will have already peaked and be > declining. Most people will be African or have mixed African descent. China > will be the global superpower. The US economy will be on par with India, > but still more advanced than today. Africa will have modern infrastructure > and eradicate malaria, making it a popular tourist destination. The largest > religion will be Islam. Global life expectancy will rise from 73 to 80. > > > > 2040 will be as different from today as we are from 2010, allowing for a > bit of acceleration. That's still a lot. In 2010 we did not have smart > phones, social media, internet censorship, online maps, language > translation, voice recognition, crypto currency, overnight delivery, or > legal cannabis. Same sex marriage and fertility collapse were both new. > > > > AI progress will slow down even without an (unlikely) treaty with China > because: > > > > 1. Moore's law has ended for silicon transistors because we can't make > them any smaller. We haven't noticed yet, just like it took a few years to > notice that clock speeds stalled at 2-3 GHz in 2010. > > > > 2. The biggest cost of AI after hardware is human knowledge collection. > There's only 20 TB of text we can scrape off the Internet for free, which > is less than in 2010. The other 8 million TB of human knowledge in our > brains has to be collected at 5-10 bits per second per person. This is > being accomplished through pervasive surveillance but still takes years to > decades. > > > > 3. Exponential recursive self improvement is limited by how fast we can > design and build stuff and test how it interacts with humans, about 2-5% > per year. AI won't speed that up because AI is about modeling human > intelligence (Turing test), not about universal intelligence or evolution. > There is no reason to build something that doesn't give us what we want. We > solved the hard step of alignment by training AI on human knowledge. It > knows what we want better than we do. > > > > What are your predictions? > > > > -- Matt Mahoney, [email protected] > > > > > > > > > > > > > > > > > > > > > > > > > > Artificial General Intelligence List / AGI / see discussions + > participants + delivery options Permalink ------------------------------------------ Artificial General Intelligence List: AGI Permalink: https://agi.topicbox.com/groups/agi/T574b10b838cba731-Mfbe6c9b363565115924823f3 Delivery options: https://agi.topicbox.com/groups/agi/subscription
