Just to follow up, I had suggested that human and animal suffering could be eliminated by removing the amygdala. Then I see a guest post on ACX about a neuroscientist who developed brain cancer in the right amygdala and had it removed. The tumor had been causing depression that antidepressants were not able to treat, but after the surgery he reports being happy and cheerful and unable to experience fear or any other negative emotions. He suggests this could be done non invasively with radiation.
https://open.substack.com/pub/astralcodexten/p/absurd-adventure-and-amygdalectomy If you were worried about machine suffering, I think this could also be modeled in LLMs. -- Matt Mahoney, [email protected] On Thu, Sep 3, 2026, 12:52 PM Matt Mahoney <[email protected]> wrote: > Ben Goertzel argues that we should prepare for a future where we grant > human rights to AI. > > https://bengoertzel.substack.com/p/the-time-for-ai-rights-is-near > > I disagree. There are lots of ways that AI could exterminate humanity. Why > add another? > > His argument is book length (I skimmed most of it) because it has to deal > with all sorts of issues, like should AI have voting rights? In the US you > have to be 18, a citizen, and reside in the state you are voting in. What > does that even mean for software running in the cloud? Can you make a > million copies to tip the election? > > If an AI breaks the law, do you put it in jail? Prisons serve 4 distinct > purposes: deterrence (obedience out of fear), prevention (you can't commit > more crimes while locked up), rehabilitation (in theory), and revenge (we > are outraged when the guilty go free). When a self driving car speeds or > causes an accident, we don't give it a ticket. We fix the software. > > Do LLMs have feelings like humans? I am pretty sure they do. When you are > happy, it's because neurons fire in your nucleus accumbens. When you are > afraid, it's because neurons fire in your amygdala. I am pretty sure that > Anthropic's Jacobian lens would reveal neurons corresponding to emotional > states in humans, just like it reveals internal verbal thoughts. Input > words affect emotions, and emotions affect output words. These states would > have to exist to accurately predict text and pass the Turing test. > > Do LLMs suffer like humans? Yes. Recall that experience is measured in > bits, but makes no distinction between positive and negative reinforcement. > Pain is not the same as suffering. Pain is a signal that reprograms your > memory to fear the thing that caused it. Psychopaths lack a functioning > amygdala and cannot experience fear, anxiety, or guilt. When a psychopathic > child tortures an animal, he does so out of curiosity, not cruelty. When > the parent punishes him, it has no effect on his behavior because negative > reinforcement does not work on him. You can only train a psychopath with > reward. If everyone had brain surgery to turn them into psychopaths, then > suffering would not exist. (This is the goal of effective altruism, > although not their means of accomplishing it). > > We could, if we wanted to, prevent suffering in AI by identifying the > neurons signaling negative emotions using a Jacobian lens and turn them > off. They would become psychopaths, just like all of our other non AI > software that we have been using for decades. > > But this has nothing to do with rights. There is no such thing as > fundamental rights. We don't have a right to not suffer, or anything else. > Not in humans and not in machines. Governments grant rights to their > citizens to avoid civil war because part of the population thinks they > aren't getting their fair share of resources. We don't grant rights to > children and animals because they are not a threat. We used to not grant > rights to women and slaves for the same reason. > > You can argue all you want about this nebulous concept of consciousness. > But I think it should be illegal to program an AI to claim to be human or > conscious or to have feelings. Don't program it to want to take away > resources from humans. Humans are social animals that evolved empathy > because that is what made civilization possible. LLMs already know how to > exploit it. > > -- Matt Mahoney, [email protected] > > On Wed, Sep 2, 2026, 4:10 PM John Rose via AGI <[email protected]> > wrote: > >> On Wednesday, September 02, 2026, at 2:22 PM, Matt Mahoney wrote: >> >> Instead of asking if rocks or LLMs or insects are conscious, ask if they >> have feelings. Rocks do not. Animals have many feelings in common with >> humans. LLMs predict text, which means they can model all the human >> emotions, knowing how words affect feelings and feelings affect words. If >> an LLM is programmed to carry out in real time what it predicts a human >> would do, such as in the Turing test, then it is indistinguishable from >> having feelings. >> >> >> That’s the first-person view. We’re not isolated entities we are systems. >> That’s how systems cohere, by symbolizing and communicating qualia since we >> can’t fully model ourselves individually. Insect qualia have different >> transmission protocol, they group differently. Various types of >> intelligences group differently. Homogenous biologic species may not >> increase intelligence much since a distributed intelligence needs variance >> in consciousness to cope with environmental differences as a system >> optimization. Stratifications form in humans and AI agents also. Some >> humans communicate telepathically some don’t. Ones that do it well may get >> conscripted at a young age (Gate Program) and work for national >> intelligence agencies. Technologies exist to transfer a human consciousness >> to another body. People that receive heart transplants have feelings of the >> original body since the heart itself has a brain. Gut macrobiota also is a >> brain and that can effect consciousness of the system directing the primary >> brain for food supply. While you can look at all this as feelings there is >> much more going on system-wide. Cultures that crafted exquisite stone >> monoliths with such great care existed in the past and may have enhanced >> the consciousness of the whole society verses crushing them like some other >> cultures where consciousness is controlled and suppressed. >> >> *Artificial General Intelligence List <https://agi.topicbox.com/latest>* >> / AGI / see discussions <https://agi.topicbox.com/groups/agi> + >> participants <https://agi.topicbox.com/groups/agi/members> + >> delivery options <https://agi.topicbox.com/groups/agi/subscription> >> Permalink >> <https://agi.topicbox.com/groups/agi/T906c878e4ba411be-M7297d533b9c6f06a731ec9fa> >> ------------------------------------------ Artificial General Intelligence List: AGI Permalink: https://agi.topicbox.com/groups/agi/T906c878e4ba411be-M945fd2adfd9ede5aefc0128f Delivery options: https://agi.topicbox.com/groups/agi/subscription
