You're not "streaming" data into the stream. You're pumping it full as quickly as possible and letting it drain in its own time. Obviously this is sub-optimal.
On Tue, Mar 20, 2012 at 3:20 PM, C. Mundi <[email protected]> wrote: > > Hi. I am trying to learn how to use streams properly in node. > > The attached script stakes a single argv parameter N and writes 2^N bytes > to a file via a stream. > > I expect time behavior O(2^N). But what I am seeing is scaling faster. > > I collected data like this in bash on Linux 3.0.0.-16 x64: > > for i in {0..20}; do time node simple.js $i; done > > and plotted the results in the attached spreadhseet (OpenOffice format). > > Watching it execute, I see that almost all of the time for N>10 is spent > waiting for the stream to close. It seems to be draining very slowly! The > maximum memory use by node at any time is 180 MB, less than 10% of physical > memory. For N greater than about 10, node takes 100% CPU while closing the > stream. The system is otherwise not under load. The system is not > swapping much if at all. > > The straight lines in the plot are a guide to the eyes -- not a fit! You > can see that the time behavior is superlinear on the log-log plot. > > If I were using a lot of memory, I would expect the GC to kick in and then > all bets are off. But my biggest run (N=20) is only a megabyte. > > Could the pushback be coming from the Linux VFS? The scaling of the > system time makes me wonder. > > Please help me understand why I should not do what I've done and where to > learn about how to use streams efficiently in node. > > Note: The code is a demonstration. I would not typically create files > this way. > > Thanks for cluing the noob (again). > > > -- > Job Board: http://jobs.nodejs.org/ > Posting guidelines: > https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines > You received this message because you are subscribed to the Google > Groups "nodejs" group. > To post to this group, send email to [email protected] > To unsubscribe from this group, send email to > [email protected] > For more options, visit this group at > http://groups.google.com/group/nodejs?hl=en?hl=en > -- Job Board: http://jobs.nodejs.org/ Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines You received this message because you are subscribed to the Google Groups "nodejs" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/nodejs?hl=en?hl=en
