First of all, Mikeal Rogers is not a "core guy who doesn't write
apps".  Mikeal Rogers has written several apps with node, and has
authored very few patches to node core.  He has a startup, and has
been building applications with Node since the 0.1 days.

And yes, I work on the node core project, but I'm also a Joyent
employee.  My thoughts on this came from debugging node in production
applications at scale, especially the memory leaks and HTTP errors
that Voxer has seen on a regular basis.  Also, some of these problems
have manifested in the node programs that Joyent uses to run data
centers for real customers, pushing loads of network traffic.

Our complicated nextTick semantics are causing real problems for real
applications.  There is no ivory-tower-ism here.  Node exists in the
real world.  That's why we need to make nextTick work properly for the
cases where it's actually used.  Complying with some notion of the
"correct" meaning of "tick" and "next" is completely not a priority.

I didn't ask for feedback as a joke.  I asked for feedback because I
wanted to know what problems we would encounter if we changed nextTick
so that it actually works for its intended use case.  The conclusion
that I eventually drew (the plan for node v0.9) is informed by that
feedback, and was not known at the outset.

There were two use cases presented in objection to the proposal:
1. Using process.nextTick to "break up" CPU intensive operations.
2. Using process.nextTick as an idle listener.

#2 is a valid use case for which there is no reasonable API at the
moment, but nextTick is *not* an idle listener anyway, since it will
frequently be fired in advance of pending IO.  setTimeout(fn) is
better, but we probably ought to implement setImmediate or something
similar.  That was new information, and is very useful.  Thank you,
everyone.

#1 is not a valid use case.  nextTick is horrible for this, I just
can't put it any other way.  Either the operations are fast enough to
be done in a single thread, and a loop is fine, or they're not, and
you need to put them into a child process.

Meanwhile, we have lots of real world cases where nextTick is causing
actual problems for real applications.  It's adding latency and
causing errors to be thrown.  Phone conversations are interrupted by
it, and error pages are showing up in web browsers and applications.
It's an insidious bug that only manifests under load, and it must be
fixed.

Also, it seems that the documentation of process.nextTick needs to be
improved along with this change, because there is widespread
misunderstanding about how it ought to be used.  The impression I'm
getting is that a lot of people treat nextTick a bit like a background
fork, and it's really not a good idea to use it that way.

Are there other new use cases not already brought up in this thread?
If so, please share them.  They may make a big difference.

For those who feel disenfranchised by this decision, please consider
where I'm coming from.  Half the people in node assumed that it worked
one way, and the other half believed it worked the other way, and in
fact, both were at least partly wrong.  It's failing at its intended
purpose, and causing http to be subtly broken in some cases, which is
causing real world problems.

We can make node much more reliable for high-traffic HTTP servers
(which is what node is explicitly for), at the expense of making it
slightly worse at some approaches to high-CPU use cases (which is what
the child process API is for), and make node's internals simpler in
the process.

We can't please everyone.  But please don't be so presumptuous to
assume that this is somehow about library authors being out of touch
with real world application developers.  This is about choosing which
real world application developers to please, and which developers are
going to have to change their code.

Reply via email to