Great explanation Isaac, thanks for taking the time to write it. +1 to having nextTick work in a predictable way and updating the documentation to reflect that
An idle event or async API would be useful to have. -- Dick On Jun 1, 2012, at 12:19 PM, Isaac Schlueter wrote: > First of all, Mikeal Rogers is not a "core guy who doesn't write > apps". Mikeal Rogers has written several apps with node, and has > authored very few patches to node core. He has a startup, and has > been building applications with Node since the 0.1 days. > > And yes, I work on the node core project, but I'm also a Joyent > employee. My thoughts on this came from debugging node in production > applications at scale, especially the memory leaks and HTTP errors > that Voxer has seen on a regular basis. Also, some of these problems > have manifested in the node programs that Joyent uses to run data > centers for real customers, pushing loads of network traffic. > > Our complicated nextTick semantics are causing real problems for real > applications. There is no ivory-tower-ism here. Node exists in the > real world. That's why we need to make nextTick work properly for the > cases where it's actually used. Complying with some notion of the > "correct" meaning of "tick" and "next" is completely not a priority. > > I didn't ask for feedback as a joke. I asked for feedback because I > wanted to know what problems we would encounter if we changed nextTick > so that it actually works for its intended use case. The conclusion > that I eventually drew (the plan for node v0.9) is informed by that > feedback, and was not known at the outset. > > There were two use cases presented in objection to the proposal: > 1. Using process.nextTick to "break up" CPU intensive operations. > 2. Using process.nextTick as an idle listener. > > #2 is a valid use case for which there is no reasonable API at the > moment, but nextTick is *not* an idle listener anyway, since it will > frequently be fired in advance of pending IO. setTimeout(fn) is > better, but we probably ought to implement setImmediate or something > similar. That was new information, and is very useful. Thank you, > everyone. > > #1 is not a valid use case. nextTick is horrible for this, I just > can't put it any other way. Either the operations are fast enough to > be done in a single thread, and a loop is fine, or they're not, and > you need to put them into a child process. > > Meanwhile, we have lots of real world cases where nextTick is causing > actual problems for real applications. It's adding latency and > causing errors to be thrown. Phone conversations are interrupted by > it, and error pages are showing up in web browsers and applications. > It's an insidious bug that only manifests under load, and it must be > fixed. > > Also, it seems that the documentation of process.nextTick needs to be > improved along with this change, because there is widespread > misunderstanding about how it ought to be used. The impression I'm > getting is that a lot of people treat nextTick a bit like a background > fork, and it's really not a good idea to use it that way. > > Are there other new use cases not already brought up in this thread? > If so, please share them. They may make a big difference. > > For those who feel disenfranchised by this decision, please consider > where I'm coming from. Half the people in node assumed that it worked > one way, and the other half believed it worked the other way, and in > fact, both were at least partly wrong. It's failing at its intended > purpose, and causing http to be subtly broken in some cases, which is > causing real world problems. > > We can make node much more reliable for high-traffic HTTP servers > (which is what node is explicitly for), at the expense of making it > slightly worse at some approaches to high-CPU use cases (which is what > the child process API is for), and make node's internals simpler in > the process. > > We can't please everyone. But please don't be so presumptuous to > assume that this is somehow about library authors being out of touch > with real world application developers. This is about choosing which > real world application developers to please, and which developers are > going to have to change their code.
