On Sun, Feb 9, 2025, 9:01 PM Rob Freeman <[email protected]> wrote:
> Matt, > > Interesting about the lexical clustering techniques also working for whale > song. An old curiosity of mine. I've always wanted to extend language to > this. What technique did you use? Entropy maximization, repeated sequences? > See my paper. I just counted ngrams. But re. your statement for the reason NNs have been so successful > developing language models being because "Language evolved to be learnable > by neural networks one layer at a time, segmentation first, then vocabulary > ... then semantics, then grammar." I dispute that. You're giving people a > bum steer if they read that and imagine it is "settled science". Not least > the hierarchical attribution of grammar over semantics (possibly influenced > by cognitivist/functionalist linguistic dogma? Or is that still a Hutter > Prize idea, that semantics will provide a minimal representation for > grammar?) > You need to understand words before you can parse sentences. How do you parse the following? I ate pizza with pepperoni. I ate pizza with a fork. I ate pizza with Bob. Children learn semantics before grammar. They start to learn the meanings of words at age 1. They learn content words like "ball" or "milk" before high frequency function words like "the" or "of". They learn sentences around age 2-3. That's more training data. To understand whale semantics, you have to observe how their behavior associates with different calls. Whales may have different languages. We don't know. Lexical structure (segmentation and Zipf's law) is language independent but semantics is not. How would we find out without more data? ------------------------------------------ Artificial General Intelligence List: AGI Permalink: https://agi.topicbox.com/groups/agi/T73fe79f7d09a903a-Md383fcca082bd2993941a907 Delivery options: https://agi.topicbox.com/groups/agi/subscription
