https://elanbarenholtz.github.io/#evidence

This guy seems to be on the right track.

The Morphosyntax Experiment

If syntax is distributional structure over high-leverage tokens, then
function words (THE, WAS, TO) and morphology (-ING, -ED, -LY) should
constrain predictions even when surrounded by nonsense. We tested this by
measuring next-token entropy in language models across four conditions:
Real Sentences:
"The teacher was explaining the concept clearly"
Jabberwocky:
"The blicket was florping the daxen grentily" (function words + morphology
intact)
Stripped:
"Ke blicket nar florp ke daxen grenti" (all nonwords, no morphology)
Random Nonwords:
Completely unstructured

*Results:*

Sentences (7.45 bits) < Jabberwocky (8.04) < Stripped (9.07) < Random (9.27)

Morphosyntax alone reduces entropy by ~1 bit (p < 0.0001, d = -1.75).
Function words and morphological markers constrain prediction independently
of semantic content — exactly as predicted by the distributional account.

------------------------------------------
Artificial General Intelligence List: AGI
Permalink: 
https://agi.topicbox.com/groups/agi/T33aa28f274d02422-Ma259f1137fb92f305bbbb1c7
Delivery options: https://agi.topicbox.com/groups/agi/subscription

Reply via email to