I'll kick in one bit of 2c here: Statefulness is the enemy of distributed systems - it's why all the clever objects-with-one-foot-on-each-side-of-the-wire frameworks that have come and gone have gone; it's why distributed databases are hard and choose different sets of trade-offs in terms of what they do in the case of different flavors of failure; it's why ntp and clock syncing is terribly important in those systems.
Nupic is all about state - state that can't be minimized or reduced because fundamentally what Nupic *does* is compute that state. If you can design something where Machine-A computes something and throws it over the wall to Machine-B, which will not feed anything back to Machine-A, that will work. If not, you could easily end up with something where the amount of bookkeeping and stops for coordination completely swamps the benefit of distributing the work, or slows work down to some percentage of the speed of the slowest machine or network connection in the group. I'm not saying don't do it, but think through the parts that would need to touch each other or wait for each other, or be synchronized with each other across machines very carefully, up-front, or it's possible to waste a lot of effort. Scale-out is cooler than scale-up, but not all problems are amenable to it. -Tim
