Rektide, I still am very much interested in technical discussion around this > feature, know it abstractly but not deeply, and the cause of it being > shelved remains shrouded in mystery. >
I made an independent effort to implement isolates - in the sense of multiple node instances in a process, not TAGG - before the isolates effort by the core team. I wasn't privy to the discussion that led to the decision to remove the isolates feature, but I can give an overview of what I think the issues are in getting that feature implemented, stable and usable in practice. Before diving in, I should say that this feature really is very different from TAGG, although they share the same underpinnings in terms of the v8 features used. Nonetheless, it is certainly confusing the whole discussion for them to be put into the same bucket. My own use-case and constraints were quite specific, but the general motivations, as others have commented, were the possibility of: - lower cost of message-passing between instances, avoiding a data copy. Nobody imagined, realistically, that passing object references from one instance to another was possible, but copies of primitive datatypes, buffers and serialised objects could in principle be enabled between instances in the same process; - reduced context-switch times; - reduced instance start-up time; - miscellaneous reduced overheads from sharing of resources such as libev thread pool, fds, or whatever. There's a summary of a number of implementation issues in this issue: https://github.com/paddybyers/node/issues/16. The core team's implementation followed a similar path although there were differences in detail. All of this sounds quite achievable, and there's no reason why this set of features cannot be completed and stabilised with a lot of careful attention to detail. Don't underestimate the effort needed, and the time it will take to iron out all thread-insecurity problems with existing code, but possible. However, that's only really the beginning. Once you've done this you confront three sets of issues: 1) Lots of expected behaviours simply aren't achievable. This includes state that is shared that people expect not to be shared: process.cwd(), process.env, process.setuid(); now you have to study your app - and reverse-engineer the modules you're using - to understand where these differences in behaviour will affect you. Child_process, SIGCHLD handling, and signal handling generally cannot be implemented completely faithfully. This is all OK if you're prepared to define, document, and accommodate the changed semantics, but it's a huge effort to migrate all existing code to a new set of assumptions - and only worthwhile if there's a big payoff. 2) Existing native modules now operate in quite a different regime, and all existing modules will be broken. When there is only a single v8 isolate you know it's OK to keep v8 handles in statics. You can also get away with being a bit lazy and/or buggy in the way you release native resources. None of that is possible when you have isolates; you need to track and release every resource explicitly and exactly, and you have to have a way of putting your object handles into thread-local storage or some other isolate-wide structure. Every single native module will fail when you use it in the second isolate in a process, which means that every single real-world app that uses a number of such modules will fail in unexpected ways. The maintainers of those modules might not have an interest in your use-case and that's a real barrier to getting any high-quality migration to the new regime. 3) The compelling performance gains you had as motivation for doing any of this are marginal at best. In fact, since you're now using thread-local storage to get context throughout the codebase, it's potentially slower most of the time. The shared memory that you thought was going to be the performance win turns out to be the thing that kills you, and you realise how powerful process separation, with the hardware assistance it gives you, really is. More or less every place you thought of using shared memory - and where it's a practical bottleneck - there's a way of getting pretty much the same thing between processes using shared memory or whatever. Add these things to be backdrop of having other pressing reasons to get v0.8 out, and you can see why the decision was made to abort the development. I'm sure the decision wasn't taken lightly given the effort that had been sunk into it. My own use-case for isolates still remains, however, and I do intend to migrate the work I did to v0.8+. I'm optimistic that the performance degradation from thread-local storage can be kept well below the 2-8% that Ben stated, and can be made negligible in the case that you only ever use a single default isolate. But that's speculation at the moment. Please don't take any of the comments above to apply to TAGG, and I'm not commenting, either positively or negatively, on that. Paddy -- Job Board: http://jobs.nodejs.org/ Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines You received this message because you are subscribed to the Google Groups "nodejs" group. To post to this group, send email to nodejs@googlegroups.com To unsubscribe from this group, send email to nodejs+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/nodejs?hl=en?hl=en
