Hi. I am trying to learn how to use streams properly in node. The attached script stakes a single argv parameter N and writes 2^N bytes to a file via a stream.
I expect time behavior O(2^N). But what I am seeing is scaling faster.
I collected data like this in bash on Linux 3.0.0.-16 x64:
for i in {0..20}; do time node simple.js $i; done
and plotted the results in the attached spreadhseet (OpenOffice format).
Watching it execute, I see that almost all of the time for N>10 is spent
waiting for the stream to close. It seems to be draining very slowly! The
maximum memory use by node at any time is 180 MB, less than 10% of physical
memory. For N greater than about 10, node takes 100% CPU while closing the
stream. The system is otherwise not under load. The system is not
swapping much if at all.
The straight lines in the plot are a guide to the eyes -- not a fit! You
can see that the time behavior is superlinear on the log-log plot.
If I were using a lot of memory, I would expect the GC to kick in and then
all bets are off. But my biggest run (N=20) is only a megabyte.
Could the pushback be coming from the Linux VFS? The scaling of the system
time makes me wonder.
Please help me understand why I should not do what I've done and where to
learn about how to use streams efficiently in node.
Note: The code is a demonstration. I would not typically create files this
way.
Thanks for cluing the noob (again).
--
Job Board: http://jobs.nodejs.org/
Posting guidelines:
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en
Stream Memory Pressure.ods
Description: application/vnd.oasis.opendocument.spreadsheet
simple.js
Description: JavaScript source
