Thoughts: a few of many, some saved; perhaps too complex. *Briefly:* superficial and deceptive unjustified logic. What exactly is an agent, without a definition, the implied ones are flat, like a single elementary scalar "utility function" and a trivial straight goal with no side effects, a 1-bit representation and one-single centralized "agent" which controls everything (that's supposedly the Universe, yet that control has different manifestations at different scales and ranges).
*Matt"**1. If an agent is smarter than you, then you can't predict its actions, and therefore cannot control it."* *Todor:* (Various seeds of thoughts) 1. You can't predict it also if it's a random process, by definition. You can't predict a toss of a coin. "It's not an agent" - why not, it's a part of other "agentic systems", including you, e.g. you play gamble by tossing coins, and depending on the outcome you play Russian roulette, LOL. The coin "controls you" - no matter the outcome of the "test shot", and no matter if you just toss coins and observe the number of heads and tails: this process is "controlling you", your focus, behavior, what you record, see etc. So a trivial random process resulting in one bit of information is (somewhat, quantitively) "predictive for your behavior" and controls the gazillion bits describing your body (if one analyzes it this way; 1-bit doesn't *represent* precisely these changes and it is a "virtual control" (see Theory of Universe and Mind), however THE SAME goes with humans who believe "they control" anything - outputting 10 bits per second and "controlling" even an Apple//, what about a PC with 64 GB RAM, 100B transistors chips etc. - how exactly your press of a button "controls" it - rather IT controls you if these amounts of information are used in the "requisite variety" way, the content of the screen controls where you will click and look; in fact all are parts of a process and "control" in the fearful way from these aligners is an obsession in simplifications and "channeling" a hierarchical, feedback, interactive, multi-layer, multi-scale, ... processes into simple "chain of command" where the control-freaks are on top). 2. You (not personally) don't distinguish POWER from intelligence. If anyone is "smarter than you", but has no corresponding power, the entity with power "controls" it when it crosses its boundaries. It may not be able to control it at the highest possible or higher than ... resolution of causality-control, similar to Ashby's requisite variety, but it doesn't matter and the force (virtual agent) with higher power may not care. Are the presidents of USA the smartest Americans, or maybe the policemen, the special agents or the bosses in any company? By "definition" they are supposed to "control you", they are "in charge", they can give you orders and you are forced to execute them (so they predict you with the resolution that is sufficient for them: one aspect of prediction is causation). You can't give them orders or if you do, they won't obey. You could make them do what you want, breaking the hierarchy, only if you CHEAT or if you destroy the hierarchy and use "unallowed" means, i.e. ones which the other "more intelligent" by your definitions (actually being charged with more POWER) believe are "not allowed" (they "wouldn't predict them) - e.g. if someone uses physical force, manipulate some systems, data etc. In the "normal" condition where the obedient ones down the hierarchy are "aligned" with the hierarchy, if the human individuals are taken as "the agents" in the mundane way of segmentation, therefore, according to your definitions, they should be models for "highest intelligence" (perhaps that's why the services are called FB Intelligence (the intelligence of Facebook) and C-Intelligence-A (Cerebral I. Association) and we should take lessons from George Bush Jr., Biden, Trump, IQ's 99999 (measuring the POWER they had and what they caused or could cause to the world and "predict" the future of their victims, e.g. predicting and causing a million or millions of people to die or suffer in a war or due to the war). The congressmen, senators etc. who invited or would invite AI representatives to discuss these problems (one was reciting the same confused triviality about "if it's more intelligent, you can't control it --> "therefore" "it want to kill us"), who "controls you" and the world by deciding on laws or sending military troops to "civilize" the others, they are supposed to possess "higher intelligence" than us (literally) or the others. How come? Perhaps they haven't HEARD, because they didn't have curiousity about, of any technicality related to AI, until ChatGPT or they can't understand a burst of 5 machine code or Assembly instructions or higher math, perhaps they are not quite talented in ANY really deep, analytical, creative, mathematical or whatever field and compared to the talented and geniuses in these fields they are fools or retards. However THEY CONTROL the geniuses and talented ones, i.e. they "can predict" that if these masters decide a law or enforce something, the smarter ones will have to obey (if not: there will be consequences, enforced by the "even smarter" policemen or if needed: the army: the smartest of all: a machine gun or a rocket can PREDICT that you will be destroyed in 1 second - you may either be able to "predict" that (seeing the gun pointed at you or the rocket falling to you), but not be able to prevent it: prediction is not enough, you need appropriate causal POWER over the phenomena you predict in order to change them, and be able to chain them etc.; it is not just about "prediction" in isolation and flat). The same goes for anyone who has more financial power, in a world where things are bought, compared to someone with less or none financail power. The financially "powered" one can PREDICT the behavior of the ones who sell him goods: he pays, they hand him the item or do what he has ordered. If you have no money or power, the other agents will not do it, no matter how ingenious you are. The bacteria and viruses are supposedly "dumber" than humans in human measures, or say gamma radiation or temperature, but at low level at molecular scale, they are more powerful and "more intelligent" by the given definition, they have "higher predictive" (and more importantly CAUSAL) power over molecules in their locality than the molecules of the human body, so they can kill humans and humanity with 0 points on the LLM tests. Another force is the intention to CHEAT others which is known in many "democratic" societies or most, as well as your will "to control" (if you can't control it, "it may want to kill you" - why??? because that's how *YOU* or the part of your civilization used to behave with others who it couldn't take the resources it demanded). The most intelligent persons rarely have power or aim at it in human terms in scales corresponding to their intelligence, neither they try to CHEAT. And many "successful" individuals, persons, in financial, "social", "human", "political" measures are notoriously DUMB in a serious scale. "Entrepreneurs" with obvious or diagnosed ADHD, dyslexia & dysgraphia* and generally not curious, not learning much, besides that they "want the success" (higher profit) and they chase this (*or very poor general reading and writing skill, poor abstract thinking, poor working memory capacity). That resembles the simplified notion of "utility" that is "maximized": flat and scalar, not requiring intelligence (and "intelligence" may "emerge" as a side effect of other forces and locations which work for achieving the maximization of this scalar utility function, as it happens in one way or another with ML models with implicit creation of as-if world-models) *"Matt: *If an agent is smarter than you, then you can't predict its actions, and therefore cannot control it." *Todor:* You may be unable to predict it with sufficient or with ANY precision beyond chance if it has ZERO intelligence and acts "completely randomly". You can be able to predict its behavior, however lacking POWER to cause a change of his behavior, while it may have that power and be dumb as f* yet forcing you to do whatever it wants and simplifying your behavior down to what its complexity is. You may be able to predict it using SIMULATORS which are "smarter than you" (you can't do it in your mind or without the technology - that's the case without modern "AI" anyway). You may be unable to predict the exact content (most humans are worse than GPT2 in what it does), but you may be able to predict that it won't be harmful in some range of time, space, domains. (...) *Matt: "4.* The goal of all agents in a finite universe is a state of maximum utilitiy, where any thought or perception is unpleasant because it would result in a different state. Your goal is death. You just don't know it because evolution programmed you to fear death." *Todor: *This agent is too simplistic, like maximizing only a single scalar value, and it is also to SERVE. What about the goal as TO BE, to exist, or to progress. This is already discussed about at least two types of "rewards" in a reward model: sensual and cognitive. The maximum "pleasure" state is the sensual sub-system in living organisms, they want to "feel good". The cognitive system wants something else and if its state (at some resolution) doesn't change that's boring and cause change of the state; in FEP/AIF/Mark Solms that's "babbling". Yet, these systems work in agents which are "in tact", at lower level and the virtual and field-defined causality-control units it may be different; generally at lower levels systems, looked from a higher ones, seem as aiming at doing exactly what their intention/design is, complete match (like being in that state), but these are "micro" states and micro agents from a macro view, and their "state" is "static" as say "existing" as an atom, a molecule, a cellular organelle, but this is not death, they don't "want" to dissolve, and at the higher levels or higher resolution of causality-control they have parameters which change, e.g. coordinates, temperature or other kind of energy, aspects of their "internal structure" which this "agent" itself may be unaware of etc. (Humans as agent may believe "they are the same" although they vastly change at low levels in any picoseconds and in longer periods they are "completely different", yet they may believe they aim to "preserve themselves".) ... *Matt: "4.* *The goal of all agents in a finite universe is a state of maximum utilitiy, where any thought or perception is unpleasant because it would result in a different state. Your goal is death. You just don't know it because evolution programmed you to fear death." * *Todor: *This is also the problem of deadlock and one of its solutions is explained e.g. in the Theory of Universe and Mind 21+ years ago: the agents, causality-control units, as Universe, have to be hierarchical and multi-scale, multi-range, multi-resolution of causality-control, working at different time, space and "embeddings" spaces and horizons, and no single (or an ensemble of) causality-control units should be allowed to take the power of the causal units, the effectors, for an infinite periods. In such a system there is no one global utility function (if the system is regarded as mutli-agent); it was explained also that there's no objective single unified self for humans, but an integral of infinitesimal selves, mind is "mutli-agent", the agents/causality-control units are "fluid" and swith their "identity", at certain level of representations they become incomplete agents, in others they are like fields where different segmentations can be recognized depending on the evaluator-observer's choices and views. The fear of death is bullshit, it's not what humans or living beings fear "the most". What babies fear? NOTHING, except loud noises or physical, immediate, automatic signals for tissue damage, in case their pain-signals are in tact. Babies are the authentic "humans" or what "evolution" taught it as an individual "entity", excluding the "software" and humans as field-defined-agents with a distributed locus of control which is going way beyond the boundaries of the obvious mechanical biological body. So babies and humans initially fear only PAIN and suffering, maybe also the fear itself, for most people. They fear it only if they feel it (the respective subsystems work that way, there are such believes), or they believe they will feel it. Children who don't feel pain for genetic and medical reasons may burn themselves, break their bones, cut themselves, poke in their eyes and continue to play. If the pain system is broken, many axioms of life go to hell: it stops wanting to "preserve itself" and to "fear death". (...) Todor Arnaudov The Sacred Computer: Thinking Machines, Creativity and Human Developmen Self-Improving General Intelligence 2025: the second oldest AGI "conference" https://github.com/twenkid/sigi-2025 ------------------------------------------ Artificial General Intelligence List: AGI Permalink: https://agi.topicbox.com/groups/agi/Te0af3a0c35a03987-Ma98d11de0e4c208394796967 Delivery options: https://agi.topicbox.com/groups/agi/subscription
