Yes, the results that LLMs produce are fascinating. Yet there is something missing. It could be the grounding that comes from embeddedness in a real environment.William James writes in his "Essays in Radical Empiricism":"Most thought-paths, it is true, are substitutes for nothing actual; they end outside the real world altogether, in wayward fancies, utopias, fictions or mistakes [...] The objective nucleus of every man's experience, his own body, is [...] a continuous percept; and equally continuous as a percept (though we may be inattentive to it) is the material environment of that body, changing by gradual transition when the body moves. But the distant parts of the physical world are at all times absent from us, and form conceptual objects merely, into the perceptual reality of which our life inserts itself at points discrete and relatively rare."https://brocku.ca/MeadProject/James/James_1912/James_1912_02.html-J. -------- Original message --------From: Russ Abbott <russ.abb...@gmail.com> Date: 5/29/23 8:01 PM (GMT+01:00) To: Jochen Fromm <j...@cas-group.net> Cc: The Friday Morning Applied Complexity Coffee Group <friam@redfish.com> Subject: Re: [FRIAM] A distinguishing feature of living entities I saw that Rodney Brooks video. He claimed that transformer-based software has no semantic content. I think that's an exaggeration. The semantic content is defined by the token embeddings. So many of the explanations of token embeddings overcomplicate the basic idea. Look up wordToVec and read some of the articles. Word-to-Vec was around before transformers. But transformers are based on that idea. (One of the keys to transformers is that the embedding space, including the features themselves, is generated as part of the training.) The embedding of all tokens in the GPT embedding space is the semantics. It's amazing the extent to which that idea can be pushed and the results LLMs produce! -- Russ On Mon, May 29, 2023 at 10:13 AM Jochen Fromm <j...@cas-group.net> wrote:Yes, Rodney Brooks said something similar. He said "GPTs have no understanding of the words they use, no way to connect those words, those symbols, to the real world. A robot needs to be connected to the real world and its commands need to be coherent with the real world. Classically it is known as the 'symbol grounding problem""https://rodneybrooks.com/what-will-transformers-transform/One could argue that this form of connectedness and embeddedness leads eventually to self-awareness. First physical embeddedness, then social embeddedness and finally self-awareness1. Physical Embeddedness:Agents who are embedded in a physical world are aware of the world and move in it. To be embedded they need to be embodied. Embeddedness leads to a grounding and a unique point of viewhttps://iep.utm.edu/husspemb/2. Social Embeddedness:Agents who are embedded in a world of social actors are aware of other agents in the world and interact with themhttps://mitpress.mit.edu/9780262518581/the-new-science-of-the-mind/3. Self-Awareness:Agents who are embedded in two worlds, the physical world and the world of language become aware that they are actors in a world of social actors by creating a link between the embodiment in the physical world (the body) and the embodiment in the world of language (the name or I)-J.-------- Original message --------From: Russ Abbott <russ.abb...@gmail.com> Date: 5/29/23 7:08 AM (GMT+01:00) To: The Friday Morning Applied Complexity Coffee Group <friam@redfish.com> Subject: [FRIAM] A distinguishing feature of living entities While watching my two little dogs run around our house, it struck me that a feature that distinguishes living from non-living entities is the apparent effortlessness with which living ones navigate the world. Imagine how difficult it would be to build a robot that could navigate the world so effortlessly. To make the comparison a bit simpler, imagine how difficult it would be to build a robotic cockroach. When I asked ChatGPT whether anyone has built a robotic cockroach, it came up with these examples. (I haven't checked to see whether these are real projects.)DASH: The Dynamic Autonomous Sprawled Hexapod (DASH) robot, developed at the University of California, Berkeley, was inspired by the rapid locomotion of cockroaches. It has six legs and can move quickly on various terrains using a simple control mechanism.Harvard RoboBee: Although not specifically modeled after a cockroach, the Harvard RoboBee project aims to develop small, insect-like robots. These tiny flying robots are inspired by the mechanics and flight capabilities of insects and demonstrate similar agility and maneuverability.iSprawl: The iSprawl robot, developed at the University of California, Berkeley, was inspired by cockroaches' ability to squeeze through small spaces. It uses a compliant body design and six legs to navigate tight and cluttered environments.VelociRoACH: Developed at the University of California, Berkeley, the VelociRoACH is a fast-running robot designed to mimic the high-speed locomotion of cockroaches. It utilizes a legged design and has demonstrated impressive speed and agility.These mainly explore locomotion. Besides locomotion, cockroaches notice when someone enters an area where they are exposed. They quickly scuttle off to some hiding place. How do they sense the presence of a new being? How do they know where the hiding places are? How do they know how to move in the right direction? How do they know how to avoid small obstacles and fires? Etc. One can argue that these capabilities are hard-wired in. But that doesn't make it any easier. These are still capabilities they have, that would be a challenge to build.I became amazed at how well-connected living entities are to their environments. They quickly and easily extract and use information from their environment that is important to their survival. Man-made robots have nowhere near that level of embeddedness and environmental integration. Was it Rodney Brooks who said that we should build that sort of connectedness before worrying about building intelligence into our robots? Today that struck me as an important insight. -- Russ Abbott Professor Emeritus, Computer ScienceCalifornia State University, Los Angeles
-. --- - / ...- .- .-.. .. -.. / -- --- .-. ... . / -.-. --- -.. . FRIAM Applied Complexity Group listserv Fridays 9a-12p Friday St. Johns Cafe / Thursdays 9a-12p Zoom https://bit.ly/virtualfriam to (un)subscribe http://redfish.com/mailman/listinfo/friam_redfish.com FRIAM-COMIC http://friam-comic.blogspot.com/ archives: 5/2017 thru present https://redfish.com/pipermail/friam_redfish.com/ 1/2003 thru 6/2021 http://friam.383.s1.nabble.com/
