Thanks for all the answers. I almost forgot to look back at this thread as the custom writeStreams have exceeded the high expectation I had already for Streams 2. For me, the reference manual was a little confusing, as there are complete examples on using the read method, no mention of "reading" through a writeStream endpoint.
Marco, I agree that that read has more detailed control of minimum incoming content. However I wonder if it would be more efficient to default pipe.chunkSize to a "lowWatermark" of the receiver (if defined). This lowWatermark could be adjusted dynamically and the callback in the writable should keep sequence of events under control? Anyway, thanks Node team, I'm very impressed! On Wednesday, March 20, 2013 4:45:32 AM UTC-4, Marco Rogers wrote: > > @Nathan's response is right. Creating a writable stream is preferable in > most cases. But I wanted to add a little context to that. If you're dealing > with a base readable stream, it's just pushing chunks of data at you off > the wire. Your first task is to collect those chunks into meaningful data. > So IMO the reason creating a writable stream is preferable is because it > prompts you not just read off the stream, but to create semantics around > what the new stream is supposed to be. The api reflects this opinion and > that's why creating writable streams feels like the more natural way, and > the ugliness of dealing with read() is wrapped up in the pipe() method. It > was kind of designed that way. > > But the read() api was also designed for a use case. It's meant to handle > low/high water marks effectively, as well as enable more optimized special > parsing by reading off specific lengths of chunks. These were things that > people kept needing, but the old api didn't support well. If you were > writing a library for a special parser, you might write a custom Writable > stream and inside it you would be using the read(n) api to control *how* > you read data off the socket. I hope that makes sense. > > :Marco > > On Monday, March 18, 2013 11:06:48 AM UTC-7, Sigurgeir Jonsson wrote: >> >> The new streams have excellent support for high/low watermarks and >> auto-pausing/resuming, but the documentation confuses me a little... >> particularly the read method. >> >> When I read the new docs for the first time I was under the impression >> that the optimal way to become a user of a stream is to write loops around >> the read functio. However in practice I find myself simply writing custom >> writeStreams and use the callback to control upstream pressure (in addition >> to source Watermarks if needed). Here is an example where I move the >> output to a queue that executes a custom function in parallel (i.e. >> uploading to a database) https://gist.github.com/ZJONSSON/5189249 >> >> Are there any benefits to using the read method directly on a stream vs. >> piping to a custom Writable stream? >> > -- -- Job Board: http://jobs.nodejs.org/ Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines You received this message because you are subscribed to the Google Groups "nodejs" group. To post to this group, send email to nodejs@googlegroups.com To unsubscribe from this group, send email to nodejs+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/nodejs?hl=en?hl=en --- You received this message because you are subscribed to the Google Groups "nodejs" group. To unsubscribe from this group and stop receiving emails from it, send an email to nodejs+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
