I suppose what I am looking for is really in that space beyond the benchmark tests, in which clearly more than one decision is arguably valid within acceptable boundaries. How does the machine gauge what such acceptable boundaries are? What does the machine judge in cases with a scarcity of evidence in multiple dimensions?
Most of the emphasis on large model testing is on "understanding and reasoning" (two words appear repeatedly in papers) but not really judging. Judging is what we do about the output of the AI. But ultimately we want the machine to really judge within acceptable boundaries given a scarcity of objective evidence. Now the models usually output something like "I am not comfortable answering that" or "I am so and so model but don't do that" or such. Some of this comes down to intuition and gut feel in humans -- that is, when faced with a novel situation. On Wed, Jan 24, 2024 at 1:31 PM Mike Archbold <jazzbo...@gmail.com> wrote: > James, > > Thanks for the lead. I know the general nature of AIXI but haven't read > the paper. Basically what you are arguing, I think, is that everything done > by a machine is a judgment, since ultimately it's only subjective. So, we > cannot readily distinguish "fact" from "judgment" in a machine, and the > point is argued by Brian Smith in "The Promise of AI Reckoning and > Judgment." > > But the climate of opinion and practical nature of modern AI is in meeting > benchmarks in test, so there is some objectivity anyway, like it or not... > the benchmark tests are more or less inescapably "objective" I think. > > On Tue, Jan 23, 2024 at 2:55 PM James Bowery <jabow...@gmail.com> wrote: > >> There are two senses in which "subjective" applies to AGI, and one must >> very carefully distinguish between them or you'll end up in the weeds: >> >> 1) One's observations (measurement instruments) are inescapably >> "localized" within the universe hence are, in that sense, "subjective". >> See Hutter's paper "A *Complete* Theory of Everything (will be >> subjective)". But note that one may nevertheless speak of the "ToE" which >> one constructs from one's "subjective" experiences, as an "objective" >> theory in the sense that one may shift one's perspective and measurement >> instruments without losing what one might think of as the canonical >> knowledge about the world aka "world model" that is abstracted from such >> localization parameters. >> >> 2) One's "judgements" as you call them, or "decisions" as AIXI calls them >> via Sequential *Decision* Theory, are inescapably subjective in a the >> vernacular sense of "subjective" where one places *values* on one's >> experiences via the *utility function* that parameterizes SDT. >> >> If you're going to depart from AIXI or elaborate it in some way, then it >> is important to understand where, in its very concise formalization, one is >> performing one's amputation and/or enhancement. >> >> >> On Tue, Jan 23, 2024 at 3:55 PM Mike Archbold <jazzbo...@gmail.com> >> wrote: >> >>> Hey everybody, I've been doing some research on the topic of judgments >>> in AI. Looking for some leads on where the art/science of decision making >>> is heading in AI/AGI. Note: by "judgment" I mean situations which have a >>> decision that is open to values within boundaries, not that can be >>> immediately and objectively correct or incorrect. >>> >>> Lately I have been studying LLM-as-a-Judge theory. I might do a survey >>> or such, not sure... looking for leads, comments etc. >>> >>> Thanks Mike Archbold >>> >> *Artificial General Intelligence List <https://agi.topicbox.com/latest>* >> / AGI / see discussions <https://agi.topicbox.com/groups/agi> + >> participants <https://agi.topicbox.com/groups/agi/members> + >> delivery options <https://agi.topicbox.com/groups/agi/subscription> >> Permalink >> <https://agi.topicbox.com/groups/agi/T5edfab21647324f7-Md98cb4028534a4e2216815aa> >> ------------------------------------------ Artificial General Intelligence List: AGI Permalink: https://agi.topicbox.com/groups/agi/T5edfab21647324f7-M043c9f5a6e5f4419090ad6e3 Delivery options: https://agi.topicbox.com/groups/agi/subscription
