Excellent.

Summarizing: the idea of "understanding" something (in this case a fragment of (written) natural language) involves many representations being constructed on many levels simultaneously (from word recognition through syntactic parsing to story-archetype recognition). There is nothing in this picture to indicate that "understanding" is a simple process, or that it can be defined in terms of a single optimization function.

When I see attempts to capture the meaning of "understanding" in an expansive way, like the one you have laid out here (involving a catalog of many different processes that work cooperatively to give the whole), I feel like we are making progress.

On the other hand, when I hear people trying to narrow down a definition to some canonical core like "an agent that can find the most optimal actions to manipulate the world in the service of its goals" (or some other nonsense of that sort), I feel like I am listening to content-free gibberish. Definitions like that use words in such a way as to say nothing.

Josh's elaboration of some of the stuff that goes on in the understanding of a paragraph reminds me of a book that I have mentioned here before (Levelt's "Speaking") in which the author takes apart a single conversational exchange consisting of a couple of short sentences.


Richard Loosemore






J. Storrs Hall, PhD. wrote:
"It was a true solar-plexus blow, and completely knocked out, Perkins
staggered back against the instrument-board. His outflung arm pushed the
power-lever out to its last notch, throwing full current through the
bar, which was pointed straight up as it had been when they made their
landing."
This is an essay about natural language understanding. The above quote
is from the middle of a book, and what I intend to do is to unravel the
process by which someone reading the book could be said to understand
it. Largely the concern is about what kind of mental structures are
being built and what structures must have been built by reading the
previous half of the book for the passage to do what it does in the mind
of the reader.

Without further ado, let us jump into the quote, which starts at the
beginning of a paragraph:

"It was a true solar-plexus blow,"

There are two sources for the comprehension of this clause. First is the
preceding paragraph, where a fight is described. A scene and script have
been built up, like a movie in the mind. In particular, one man is
holding a girl (who is struggling to escape) and another is trying to
tie her feet. She kicks the second man, and that is the blow that's
being referred to.

Unlike a cinematic movie, however, much that would be evident on the
screen has been left out. The specific positions of the bodies, the
clothing in some cases, and many aspects of the background have been
left to the imagination. In other words, the "movie" is a sequence of
*abstractions.*
It is in no sense simply a pile of predicates, however. When I read
this, I come away with a semi-visual motion script, such as could be
used to orchestrate a re-enactment by action-figure dolls, even though
the text doesn't come close to specifying the actual positions or
motions I imagine.

The second source is the reader's memories of pertinent experiences,
either of watching fights or having been in them. In the multi-level
abstraction structure that's being built, by and large, at least in the
hands of a skillful writer, the things that get mentioned are the things
that you'd pay attention to if watching the scene. It's well
established, for example in studies of eyewitness acounts in
criminology, that people confabulate what happens between such points in
their memory of actual events, much less from verbal stories. So to that
extent, the structure of a story reflects that of memory.

If you've ever taken a hard blow to the solar plexus, you'll have a much
deeper understanding of this passage than someone who hasn't. I have,
and the sensation is unique; nothing else in my experience feels the
same or has the same effects. If you have, note that among the few
descriptions of clothing that were provided was that the girl was
wearing riding boots...

At a higher level, the scene is part of an attempted abduction of the
girl by the men. On this level, the reader is on tenterhooks to discover
whether the abduction will succeed, given the girl's spirited and at
least partially efficacious resistance.

"and completely knocked out,"

Syntactically, this is a bit of a garden path; we expect it to be a
conjunct of the previous predicate until we see the comma. This phrase
appears to be there for those readers who have not experienced solar
plexus blows. It describes the effect well enough to follow the action
sensibly, but doesn't really capture the experience.
This points out that there can be different amounts of actual
understanding going on in different readers each of whom would claim to
have understood the passage: there can be ties to emotions, sensations,
memories, and/or mental models in various combinations and strengths.

"Perkins staggered back against the instrument-board."

Perkins is the second man, and his staggering back is completely
predictable from the description of the action so far. In fact, it's
predictable that staggering back is part of the process of his
collapsing, which isn't, and doesn't need to be, stated explicitly. The ability to predict is one of the key elements of understanding, so
we can propose that there is a model of collapsing after being knocked
out (or struck in the solar plexus) that abstracts away from any
particulars about the individual (or his specific position) that allows
the extrapolation of the "movie," if need be.

It's the instrument-board that is the new item. In order to understand
this, the reader has to pull into play a much broader background
structure than heretofore. The action is taking place in the control
room of a spaceship, and the "instrument-board" is its control
panel. One is reminded of a programming language variable being looked
up in a containing context after being found unbound in the local one.

"His outflung arm pushed the power-lever out to its last notch,"

Now we see that there are multiple disconnected levels of abstraction,
as well as disconnected items of physical description, that need
interpolation. We hear about the arm and the lever, which is local and
concrete. We can imagine an arm striking a lever and pushing it. We
still don't know anything about how big the board is, where the lever is
on the board, whether the board is horizontal, vertical, or tilted. We
don't know where Perkins is with respect to the pilot's (and
co-pilot's?) seat(s). On the other hand, we do know much higher-level
things about control panels, and power levers (I, for example, call to
mind typical airplane cockpits as well as the spaceship control rooms in
SF movies I've seen.) Although various things about the spaceship have
been mentioned before in the book, no good description of the control
room has been given; we have to assemble this as some useful level of
abstraction as we read this passage.

"throwing full current through the bar,"

This is the key development, not only of the passage, but of the entire
book. Note that without the model that the reader will have built up by
that point, the phrase means virtually nothing. You might well think
that there is a place where drinks are served on the ship.

The book is, as the perspicacious reader may have guessed, E. E. Smith's
Skylark of Space, and its premise is that a bar of copper plated with a
(fictional) transuranic metal can convert its mass to pure kinetic
energy with the application of a current. The bar is the drive motor of
the spaceship. As far as I know, there's no other context, even in
science fiction, where the motor of any vehicle is referred to as a
bar. It certainly doesn't happen in reality, and will not have been the
case in any reader's experience. Many of us have seen bars of copper,
and to that extent the symbol is "grounded;" but in the *salient* sense
of the story it is not. Its meaning comes from the model that has been
built up out of pure words.

The really remarkable thing about this phrase is that it throws
implications across virtually every level at which the book is to be
understood.
At the physical level, it describes the closing of a circuit and the
application of voltage to a piece of metal.
At the technological level, using the (fictional) understanding of space
drives built up before, that means that the ship will be placed under
very heavy acceleration.

Back at the physical bodies level in which we were understanding the fight
before, it means that the parties will be thrown to the floor and unable
to move.

The fight will be at least suspended. At something closer to the plot
level, the parties, pinned to the floor by acceleration, will be unable
to stop the ship until the fuel runs out, stranding them far away in
space.

This converts them from abductors and victim, where the conflict is all
interpersonal, to fellow lifeboat passengers facing a common doom. There
is room for various character development as they adjust to the shift in
circumstances.

In a larger context, from the outside it will seem as if the abduction
succeeded. This will force the girl's fiance to give chase in his own
spaceship (already a hackneyed plot by 1928, to be sure, but thereby all
the more predictable for the reader).
But the fact that the ship will fly at top acceleration until its fuel
is exhausted implies that the succeeding action will be far-removed from
the familiar terrestrial scenes it has taken place in so far. In fact it
converts the story from one of personalities and intrigue (a la the
Illiad) to a true voyage of the imagination (like the Odyssey).

If you are want to understand things deeply, you typically want to call
in comparable things which can illuminate them by analogy. In this case
the arm on the lever is like the tornado in Wizard of Oz (or of course
the storm in the Odyssey); it not only throws the protagonists into a
strange new world, but motivates their subsequent adventures with the
quest to get home.

There is NO knowledge representation and inference scheme in the NLP
field today that has even a snowball's chance of capturing all this. But
a human reader with a good grounding in the classics can see that this
sentence is the turning point and spark of the whole book on something
like five different levels simultaneously.

"... which was pointed straight up as it had been when they made their
landing."

In something of an anticlimax, Smith is keeping the reader up to date
with the physical model of the ship, in case someone wonders why, having
gone down to land, it didn't keep going down when the juice was turned
back on. It had come down backwards, hanging on its thrust. Yet another
model -- and level of abstraction.

--Josh

-----
This list is sponsored by AGIRI: http://www.agiri.org/email
To unsubscribe or change your options, please go to:
http://v2.listbox.com/member/?list_id=303



-----
This list is sponsored by AGIRI: http://www.agiri.org/email
To unsubscribe or change your options, please go to:
http://v2.listbox.com/member/?list_id=303

Reply via email to