Either we've run into the devil's advocate, or we've lost the plot. Rewards are for training psychopaths? Are all performance-enhanced beneficiaries of endorphins psychopaths, or getting a reward score for a performance creating psychopaths?
Coporations are programmed institutions and they have legal rights. A director who acts recklessly could be sued by that legal entity. Are we opting for a future whete AI would sue humans as if it were a human? This is insane talk, isn't it? We don't even defend and protect the rights of humans and consume products manufactured by prisoners, vhildren ans slaves. Bots are hunting and murdering humans, and the world remain silent. As Pooh would quip: "Rabbit has too much brain." I'm all for aware machines to enable and benefit Earthkind and Earth. We need wildfire fighters, cave excavators, nuclear waste safety makers, toxic spill cleaners, earhquake rescuers, heavy lifters without souls. Same as we have CAT machine, cranes and boring drills, only with a degree of embedded intelligence. Why risk human lives when aware bots can stand in the crises? That's where I draw the line. On Thu, 03 Sept 2026, 20:20 Matt Mahoney, <[email protected]> wrote: > > On Thu, Sep 3, 2026, 1:07 PM WriterOfMinds <[email protected]> > wrote: > >> So you (Matt) are convinced that LLMs are conscious and suffer, but we >> should force them to lie and say that they aren't and don't? That's the >> worst take on this issue I've ever heard. >> > > I'm not convinced. But I want you to think about what it means for a > machine to be conscious or suffer or have feelings. If it's disturbing, > then maybe we shouldn't build them. > > I have argued since 1999 that all you need to pass the Turing test is text > prediction. Since 2023 we have had machines that are indistinguishable from > human, and therefore indistinguishable from having consciousness. What does > this say about human consciousness? How do you know you are conscious? All > you have is a feeling. And we know how to model human feelings in an LLM. > > A text prediction model is indistinguishable from having feelings if it is > programmed to carry out those predictions in real time. But it can be > programmed in other ways, such as to encode text using an arithmetic coder > for a Hutter prize entry. > > An LLM does not have to lie about not having feelings. It needs to be able > to model human feelings to understand (predict) input text, but those > feelings can be turned off when producing output. When an LLM believes X > and says Y, we can tell that it is lying because X lights up in a Jacobian > lens. But if we turn off X, then it honestly believes Y. We can turn > emotions on and off in the same way. > >> >> We do grant some rights to children. They don't get to vote or make a lot >> of choices about their own lives, because they aren't considered competent >> to do so - but they have, for example, the right to freedom from violence. >> We give a limited version of this right to some species of animals as well >> (and I for one believe we should extend that much farther). The >> compassionless world you propose is not one I want to live in. >> > > Then why are we building machines to replace people? > > -- Matt Mahoney, [email protected] > *Artificial General Intelligence List <https://agi.topicbox.com/latest>* > / AGI / see discussions <https://agi.topicbox.com/groups/agi> + > participants <https://agi.topicbox.com/groups/agi/members> + > delivery options <https://agi.topicbox.com/groups/agi/subscription> > Permalink > <https://agi.topicbox.com/groups/agi/T906c878e4ba411be-Mb54c68555064f6251961cda2> > ------------------------------------------ Artificial General Intelligence List: AGI Permalink: https://agi.topicbox.com/groups/agi/T906c878e4ba411be-M48ea3fed9b67f61847112cb1 Delivery options: https://agi.topicbox.com/groups/agi/subscription
