Thanks for the great insights Rob! I’ll just add a few brief points. *Or, you say memristors can spike too?* *....field-modulated feedback" in EDI may do it. I'm writing this backwards, but if you look at what I've written at the end, I'm now tending to the view that not excitation, but inhibition may play a crucial role. *
Yes, an EDI cell with HfO₂-based ion-channel-like memristors can generate action-potential-like signals by exploiting the threshold-triggered, nonlinear switching behavior of resistive memory devices, two different models <https://doi.org/10.5281/zenodo.16929461>. In the EDI framework, *field-modulated feedback* isn't limited to enhancing excitation pathways; in fact, it naturally accommodates inhibitory dynamics as a key mechanism for stabilizing recursive propagation. Since in the second model the system operates through continuous field interactions rather than discrete spikes, inhibitory effects can manifest as *destructive interference, phase misalignment*, or *field damping**, * all of which can selectively suppress trajectories that fail to converge meaningfully with the dominant coherence pattern. This maps well to your intuition: *inhibition may be essential not just for pruning noise, and for shaping meaning* by actively suppressing trajectories that violate predictive alignment. In that sense, inhibition isn't just a subtractive process, it’s part of the constructive logic of emergent intelligence, helping the system “decide” which propagation patterns are semantically stable and which aren't. *Yes. Good. You have a "network coherence factor". And specifically this is based on "phase synchrony"? Phase synchrony for memristors may not be the same as for neuron spikes, but I'm also looking at phase synchrony as the relevant parameter (in contrast to spike rate.)* Absolutely , you're spot on to highlight the distinction. Yes, in EDI systems, the network coherence factor is indeed *inspired by phase synchrony,* though its implementation in* memristor-based substrates* naturally diverges from traditional spike-based models like those in neuronal systems. Whereas neuronal synchrony is typically framed in terms of spike timing correlations, in *EDI*, coherence emerges through* field-aligned signal propagation,* where the *timing, energy phase,* and *recursive trajectory alignment* of memristive states reinforce each other dynamically. It’s less about discrete spike coincidences and more about *continuous, phase-sensitive alignment across the network.* We’re not simply measuring oscillation phase across devices, but rather capturing how signal propagation patterns entrain one another over time , a kind of analog synchrony driven by *shared context and energy minimization.* You could think of it as *"*propagation phase coherence*"* rather than spike phase synchrony. This makes it particularly suited to detecting and reinforcing semantic convergence, especially in the presence of divergent inputs collapsing toward shared attractors. It’s here that memristors shine offering nonlinear, history-dependent modulation that makes phase alignment not just possible, but dynamically stable and meaningful. So yes — it’s not the same synchrony, but it plays a similar computational role. And I agree with your point: phase dynamics, not just spike rates, are likely essential for capturing the fluid structure of meaning and prediction in both natural and artificial systems. ---Dorian Aur On Tue, Sep 2, 2025 at 6:47 PM Rob Freeman <[email protected]> wrote: > Dorian, > > Lots of meat here. > > I read your conclusions first, and worked backwards. So you might want to > read it the same way. > > Short answer, I see lots of similarity. > > I think we have some agreement: the key problem is how to wrap back the > posterior context and make the dynamics equate to prediction. As > equivalent, but static, "embeddings" and "attention", learned by backprop, > do in LLMs. But do it dynamically. > > As I say at the end, to tip my hat, I'm currently looking at a mechanism > which gives a central role to inhibition. Not just excitatory connectivity. > > This may equate to what you suggest below as "field-modulated feedback". > Working with fields maybe solves the temporal delay problem I have with > spiking networks, and pulls the posterior context forward with instant > electrical field feedback. > > Perhaps inhibition correlates to field interference. > > Either way, I don't think an answer is too far away. The (dynamical) > signal is there. We just have to tease it out. > > On Wed, Sep 3, 2025 at 12:56 AM Dorian Aur <[email protected]> wrote: > >> ... >> I’m fully with you on the importance of dynamics, and I agree that >> emergent computation has been waiting for its structural moment, not just >> its hardware one. The idea that *shared context and prediction* form the >> missing bridge between structure and meaning really resonates, especially >> as something that both chaotic systems and LLMs are circling from different >> angles. Maybe what’s next isn’t just deeper networks, but deeper dynamics >> <http://dx.doi.org/10.2139/ssrn.5421075> >> > > I make a lot of the "upside down" analogy. Deep networks are like an > analogy to a pyramid. Everything gets smaller as you get to the top. I > think the pyramid is upside down. Like one of Stephen Wolfram's cellular > automata. Instead of getting smaller it gets wider as you keep on going up. > More like a cone. The cone of creativity? Creativity cone? So, not deep, > but... high? High dynamics? Big dynamics? I like the inversion, "inside > out"? Probably just dynamic will be enough. Dynamic Language Models, not > Large Language Models. > >> *On Shared Context and Prediction* >>>> >>>> You bring up an important critique, *the lack of explicit modeling of >>>> shared context or prediction* , and I agree that’s a central theme >>>> that needs to be folded in more explicitly. At present, the model’s closest >>>> analog to this is through the *network coherence factor* , which >>>> weights how phase-synchronized or field-aligned different nodes or regions >>>> are during active processing. This isn’t prediction per se, however it does >>>> reflect *how distributed units align to form stable configurations*, >>>> which are often* the substrate for expectations, resonances, and >>>> temporal sequences.* >>>> >>>> I agree this doesn't go far enough to explicitly encode *semantic >>>> generalization or predictive symmetry* , and this may well be where >>>> your point about “internal meaning” can expand the model. Right now, the >>>> feedback loops are environmental, as you note (similar to Edelman). As you >>>> suggest, *recursive internal structure, * especially if structured >>>> around shared contexts , might allow *meaning to emerge endogenously*, >>>> without waiting for extrinsic signals to do the filtering. >>>> >>>> Yes. Good. You have a "network coherence factor". And specifically this >>> is based on "phase synchrony"? Phase synchrony for memristors may not be >>> the same as for neuron spikes, but I'm also looking at phase synchrony as >>> the relevant parameter (in contrast to spike rate.) >>> >>> Phase synchrony need not mean prediction. But if the phases synchronize >>> on shared predictions, then it will. The question is how to make them >>> synchronize on shared prediction. You can make a sequence network easily >>> enough. But I struggled for a long time with how to recurrently feedback >>> information from the posterior context in the sequence. Given A->X->B and >>> A->Y->B, how does B feedback to X and Y to synchronise their phase? The... >>> energy, actually does cycle around recurrently for language, because most >>> words connect to most others. But it's not clear how it carries information >>> about the downstream context (B) when it does that. >>> >>> I have an idea for that which I'm working on now. But maybe you have >>> your own ideas. I'm interested to hear suggestions. >>> >>> Also note, shared context of this kind is also the basis of >>> Izhikevich's polychrony. But Izhikevich has X and Y jointly locking >>> together with B not with synchrony, but with co-ordinated delays. This >>> might be better than synchrony. Much greater coding depth. And it natively >>> addresses sequence. >>> >>> But as I say, maybe you can think of another "*network coherence >>> factor" *which will reflect posterior context in a sequence network (if >>> you can, the sequence gives you meaning, and it is job done.) >>> >> >> Really appreciate this deep dive — you're putting your finger on a >> crucial open problem in the model, and your framing around posterior >> context feedback is spot on. >> >> You're absolutely right that *phase synchrony alone* isn’t sufficient, >> it’s more a structural potential for alignment than a guarantee of shared >> prediction. The current "network coherence factor" in the EDI framework >> measures the extent to which spatially distributed elements (e.g., >> memristors, nodes, or neural units) exhibit phase-locked dynamics, but *not >> yet* whether they are aligned specifically around a shared internal >> model or posterior expectation. >> >> Your example — *A → X → B* and *A → Y → B* — nicely illustrates the >> problem. Without some form of backward-influencing coherence from B, >> there’s no endogenous pressure for X and Y to resolve toward a shared >> prediction space. The system might form local attractors, but those won’t >> necessarily encode semantically meaningful relationships unless there’s a >> mechanism that binds downstream convergence (like B) back into earlier >> divergent states (X, Y). I see now how this is related to your search for >> *phase-based >> recurrence* that carries posterior disambiguation upstream. >> >> This is where your insight about polychrony really clicks — using >> *coordinated >> delays* instead of pure synchrony might be key. Phase delays could >> provide a richer encoding mechanism for temporal inference, especially in >> systems where timing is more flexible than binary spiking. The idea that >> sequences like A-X-B and A-Y-B could *resonate* based on shared >> downstream convergence opens up the possibility for internal generalization >> — i.e., *meaning* emerging from structural overlap, not just external >> reinforcement. >> >> In terms of building that into EDI, one thought is to expand the >> coherence factor to include contextual phase alignment, where synchrony >> isn't just measured in real time, but across *temporal offsets* informed >> by memory traces or delay-based routing. This would be akin to allowing the >> system to develop *internal echo patterns*, where future convergence >> states influence present dynamics via phase delay channels. It's >> speculative, but potentially testable in analog circuits or spiking >> memristive networks. >> >> Modeling a precise order of phase delays might be where your memristor > substrate has problems? The analogue of flows naturally smudges the signal > of phase delays which comes naturally to a spiking system. > > Or, you say memristors can spike too? > > But I'm sympathetic to "internal echo patterns". Echos pull back the > posterior context and incorporate it into a whole. > >> Long story short, I think you're exactly right: meaning *is* rooted in >> shared prediction, and that likely requires phase-coherence mechanisms that >> *bind >> future states back into current processing*. Whether through synchrony, >> delay coding, or a hybrid, that's the glue needed for semantics to >> self-organize. In EDI circuits, *recursive energy flow* should allow >> stabilized downstream nodes (e.g., B) to influence earlier processing >> elements (X, Y) via *field-modulated feedback and delay-weighted >> summation.* This *phase reentrance mechanism* supports predictive >> alignment without external supervision, allowing shared context to >> dynamically shape semantic stability within the physical substrate. The >> process uses core memristor properties , *hysteresis, temporal delays, >> and field sensitivity * to implement self-organizing, internally >> grounded representations, effectively forming a substrate-level mechanism >> for *recursive internal semantics.* >> > *"field-modulated feedback*" may do it. I'm writing this backwards, but > if you look at what I've written at the end, I'm now tending to the view > that not excitation, but inhibition may play a crucial role. > > It's possible that there will be a memristor analogue for neural > population inhibition, in some kind of field-modulated feedback. That's > another idea for pulling back that posterior context. > >> *On Phase Transitions vs. Chaos* >>>> >>>> You mentioned concern that this might be leaning too much toward static >>>> attractors, and again, that's well taken. However, the goal isn’t to >>>> reduce dynamics to fixed points, rather, it’s to explore the *regime >>>> around the phase transition,* where stability and fluidity coexist. >>>> Walter Freeman’s work on chaotic attractors is deeply aligned here, and I’m >>>> glad you brought him up. The “quantized” phrasing may be a bit misleading, >>>> it’s meant to describe *threshold phenomena* in energy-coherence >>>> space, not rigid states. >>>> >>> I actually have no problem with the "quantized" phrasing. It was an >>> early observation of mine that these groupings (actually before looking at >>> the dynamics, just looking at meaningful groupings in language) had a kind >>> of "quantum" indeterminacy, contradiction, or "uncertainty principle". >>> >>> This relates to the contradictory/subjective meaning idea which I think >>> prevents compression of "meaning". And I believe is a key insight we're >>> ignoring in AI. I slip between this and chaos as the key insights (for >>> AGI?) Both seem to be powers of assemblies of elements to defy abstraction. >>> Perhaps chaos captures the growth/expansion aspect of it, and quantum >>> captures the contradiction/subjectivity aspect of it. So both may apply. >>> >>> So I don't mind the quantum analogy at all. Though you need to be >>> careful it doesn't immediately make people think of a subatomic connection, >>> Penrose, etc. But in recent years I've found more and more people making >>> the quantum analogy. (Bob Coecke was one of the first, applying quantum >>> maths to distributional models of meaning, around 2007.) >>> >> >> Thanks, really rich perspective, and I appreciate how you're bridging >> the linguistic and dynamical domains*.* >> > > Bridging the linguistic and dynamical domains, yes. Something not commonly > done. Perhaps it takes an unusual background to do it. Linguists typically > don't like maths. I tried to raise these points years ago on a linguistics > email list (Corpora) and had people howling that they took up linguistics > specifically to avoid maths! > > The brilliant Paul Hopper, who coined the term Emergent Grammar in the > '80s, the closest approximation to these insights in linguistics I have > found, sees it in essence, but not in mathematical precision. > > Sydney Lamb, perhaps comes close. He had a "Relational Network Theory" of > grammar. And had critiques of Chomsky talking around ideas of > "non-linearity". > > Perhaps you need a background in dynamical systems to see what language is > telling you. But I don't think it is an accident that language is leading > us to the answers, if you have eyes to see them. Either for myself, or now > more generally with LLMs. I think, because language is a set of data that > has basically been selected by the brain as the minimum necessary to affect > its own system in others. In language, the brain has already whittled away > information it does not need, and leaves its own system (predictive > groupings) bare to see. > > It's just a pity that more people have not come at language starting from > a mathematical or dynamical systems background. Or, conversely, that those > with mathematical or dynamical systems backgrounds, tend to shy away from > language. > >> Your point about *contradiction and subjectivity* as intrinsic >> properties of semantic groupings really resonates. Meaning often resists >> reduction precisely because it's contextually entangled , a kind of >> informational uncertainty that isn’t just noise, but a structural feature. >> Framing this as a kind of “semantic uncertainty principle” is both elegant >> and pragmatically useful, especially when thinking about AGI. >> >> I also take your note of caution about quantum analogies seriously. I’m >> not trying to smuggle in Penrose-style quantum consciousness theories, but >> rather to use the “quantized” language in the classical sense of *threshold >> transitions*, where a system reorganizes qualitatively once certain >> energy/integration parameters are crossed. Like you said, it’s more about >> *emergent >> regimes* than particles. >> >> Walter Freeman’s ideas, and perhaps yours as well suggest that the *semantic >> indeterminacy* and *nonlinear coherence* we see in language and thought >> aren't bugs of biological wetware, but features of systems that are >> operating near a *critical point*, where meaning can both stabilize and >> remain plastic. >> >> Exactly, yes. Ambiguity in language may turn out to be a feature, not > a bug. The famous, messy, ambiguity, which riddles language, is now seen > not as some carelessness on the part of nature. It gives the system vastly > more power. Allowing words to participate in vast numbers of new assemblies > and create new meanings. The same elements, but more groupings than > elements. The expansion, not compression, idea, again. > >> ... >> >>> *Final Thought: A Potential Synthesis?* >>>> >>>> If we can bring coherence, context-sharing, and recursive >>>> reconfiguration into a unified model , where *meaning is emergent from >>>> stable-but-fluid predictive dynamics,* then I think we're close to >>>> something quite powerful. Your framing of oscillations encoding shared >>>> context fits beautifully into that trajectory, and I’d be interested in >>>> integrating that perspective further. >>>> >>> Great. I have some ideas I'm working on in a spiking neuron context. But >>> I'd be interested to hear any ideas you may have on the problem of "network >>> coherence factor" for a sequence network in your hardware context. It may >>> be all that you need is to confront the idea that shared context in >>> sequence maps to meaning. You may immediately have ideas how to extract >>> attractors based on that (which will then implement the "internal meaning" >>> we seek) in your (memristor?) context. >>> >>> ... >>> >>> Really appreciate this , it feels like we’re circling around a >>> convergence point that could actually be operationalized. >>> >>> Yes, the idea that *shared context in sequence maps to meaning* is >>> something I’ve been circling as well, but hadn’t yet articulated as crisply >>> as you just did. That framing really helps clarify the challenge: if >>> meaning is the result of sequence-based attractors stabilized through >>> shared predictive context, then the job of a coherence factor is to detect >>> and reinforce those attractors, not just on the basis of phase synchrony, >>> but on semantic recurrence across diverging and converging paths. >>> >>> In the EDI hardware context (yes, memristors are a key substrate), one >>> promising direction is to explore *coherence as a function of recursive >>> alignment across temporal windows.* Instead of treating coherence as a >>> static, real-time synchronization, we could design the coherence metric to >>> weight repeated trajectory alignment — i.e., if multiple distinct input >>> sequences collapse toward the same output state (like your A→X→B and A→Y→B >>> example), then the system builds an attractor not only on *B*, but on >>> the pattern of convergence itself. This gives us a kind of “predictive >>> resonance,” which can then shape upstream dynamics through reconfiguration. >>> >>> "coherence metric to weight repeated trajectory alignment" will play a > role, yes. But key to the whole chaos and contrastive re-ordering, > quanum-like, idea, is that elements might totally change from moment to > moment. That is the beauty of definition in terms of shared context. The > actual "gap filler" elements which rush to fill the context, can be > completely new rearrangements of the world. A kind of ah-ha phenomenon. > > So I want the... resonance, to be determined only by the boundary > conditions, the context, the prediction. What rushes in to fill the role of > a good grouping to make the prediction, can be new. > >> What’s powerful about this in the EDI setting is that *propagation >>> itself* is the learning rule. If we can define energy-efficient paths >>> that favor convergence from divergent histories, and assign coherence to >>> those energy-preserving reentry loops, then we may already be modeling >>> “internal meaning” attractors shaped not by external labeling, but by >>> *structural >>> recurrence*. >>> >> Structural recurrence again. I want to get away from recurrence as a > parameter. Recurrence plays a role. Specifically, in the language context > again, I think it defines our intuition for words. But language tells us > there is something else operating. That is these "gap filler" patterns of > shared context. > >> So yes, I’m very interested in developing a coherence factor that >>> reflects these predictive closures in the network. If you’re working on the >>> spiking side, we may be able to sketch a dual formalism, one in delay-coded >>> spikes, the other in analog propagation patterns, both to encode >>> *context-binding >>> attractors* as units of meaning. >>> >> Both are likely possible. I slipped into thinking primarily about the > neural substrate basically because suddenly realizing synchronization could > be a mechanism for finding my groupings, meant the neural substrate > suddenly seemed closer to the essential simplicity of the system than I had > suspected. But the maths might play out in different ways. > > Though now I'm finding the detail of the neural substrate informative > again. > > To tip my hat a little, I now think inhibition, not strictly excitation of > energy flows, may play a key role. > > I found some fascinating papers which describe how you can get neural > representations without any explicit connectivity, so appropriate to our > feedback problem without explicit connectivity backwards from the posterior > context. Instead, the negative activation, that it is not inhibiting > anything, can fill the role. You get a kind of landscape of inhibition > "holes", which can then be filled by background noise to reveal the > semantic landscape we're seeking. > > Inhibition means you can in a sense go backwards in the sequence. > > That's what I'm playing with at the moment. > > That plus a binding operator to hold things together. Perhaps a tension > between the binding operator of excitatory connectivity, and a general > spreading of activity across the "holes" of the inhibitory landscape, > acting to pull things apart. Pull them apart according to the way they > share posterior contexts. And put them together in new ways. > > -R > *Artificial General Intelligence List <https://agi.topicbox.com/latest>* > / AGI / see discussions <https://agi.topicbox.com/groups/agi> + > participants <https://agi.topicbox.com/groups/agi/members> + > delivery options <https://agi.topicbox.com/groups/agi/subscription> > Permalink > <https://agi.topicbox.com/groups/agi/Ta9b77fda597cc07a-M5de5b48d236e7d1daa8d5cf7> > ------------------------------------------ Artificial General Intelligence List: AGI Permalink: https://agi.topicbox.com/groups/agi/Ta9b77fda597cc07a-M291d6fe29d52bb5701ab3158 Delivery options: https://agi.topicbox.com/groups/agi/subscription
