You *must* try to read from it. Otherwise it's likely to remain "paused."
On Mar 25, 2013, at 1:50PM, Dan Milon <danmi...@gmail.com> wrote: > You're right, my bad. > > But still, data stay in the buffer until someone tries to `.read()`. > So, if you're being passed a stream that you dont know whether the > first `readable` event has fired, you can try to actually read from > it. If it returns null, then you wait for `readable`. > > On 03/25/13 22:42, Michael Jackson wrote: >> readable is emitted after you've actually started reading. >> >> >> That's not what it says in the docs >> <http://nodejs.org/api/stream.html#stream_event_readable>. >> >> ### Event: 'readable' When there is data ready to be consumed, >> this event will fire. When this event emits, call the read() method >> to consume the data. ### >> >> Calling stream.read *before* you get the "readable" event is >> totally counterintuitive. >> >> -- Michael Jackson @mjackson >> >> In your example, you dont ever `response.read()`, so no readable >> event is ever emitted. >> >> As you said, streams start in paused state and ready to be read. >> >> On 03/25/13 22:28, Michael Jackson wrote: >>> Is it correct to assume that a Readable won't emit the >>> "readable" >> event >>> until you're registered for it? >>> >>> Reading through the streams2 docs, I was under the impression >>> that all streams start out paused and don't start emitting data >>> until you add either a "data" (for old streams) or a "readable" >>> listener. For new streams, this should mean that they don't emit >>> "readable" until at >> least >>> one listener is registered. Otherwise we still need to do some >> buffering >>> in order to capture all the data. >>> >>> For example, this code misses the readable event on node 0.10: >>> >>> var http = require('http'); >>> >>> http.get('http://www.google.com', function (response) { >>> console.log('got response with status ' + response.statusCode); >>> >>> setTimeout(function () { response.on('readable', function () { >>> console.log('readable'); }); >>> >>> response.on('end', function () { console.log('end'); }); }, 5); >>> }); >>> >>> Here's my shell session: >>> >>> $ node -v v0.10.0 $ node http-test.js got response with status >>> 200 $ >>> >>> Is this the correct behavior? >>> >>> -- Michael Jackson @mjackson >>> >>> >>> On Thu, Mar 21, 2013 at 4:27 PM, Isaac Schlueter <i...@izs.me >> <mailto:i...@izs.me> >>> <mailto:i...@izs.me <mailto:i...@izs.me>>> wrote: >>> >>> re old-mode >>> >>> Yes, that's fine. If you just want to get all the data asap, use >>> on('data', handler). It'll work great, and it's still very fast. >>> pause()/resume(), the whole bit. (The difference is that it >>> won't emit data until you're listening, and pause() will >>> *actually* >> pause.) >>> >>> >>> Re read(cb) >>> >>> It's problematic for reasons that I've discussed all of the >>> places where it's been brought up. That horse is dead, let's >>> stop >> beating >>> it. (There were a few other proposals as well, btw. >> Reducibles and >>> some other monadic approaches come to mind.) >>> >>> >>> Re pipe() vs looping around read() vs custom Writable vs >> on('data') >>> >>> Whatever works for your case is fine. It's flexible on >> purpose, and >>> allows more types of consumption than streams1, and creating >> custom >>> writables is easier than it was in streams1. >>> >>> If you find something that the API can't do for you, or find >> yourself >>> doing a lot of backflips or overriding a lot of methods to get >> your >>> stuff working, then let's chat about it in a github issue. >> You might >>> be missing something, or you might have found a genuine >> shortcoming in >>> the API. >>> >>> >>> >>> On Thu, Mar 21, 2013 at 2:01 PM, Sigurgeir Jonsson >>> <ziggy.jonsson....@gmail.com >> <mailto:ziggy.jonsson....@gmail.com> >> <mailto:ziggy.jonsson....@gmail.com >> <mailto:ziggy.jonsson....@gmail.com>>> >>> wrote: >>>> Thanks for all the answers. I almost forgot to look back at >>>> this >>> thread as >>>> the custom writeStreams have exceeded the high expectation I >>>> had >>> already for >>>> Streams 2. For me, the reference manual was a little >>>> confusing, as >> there are >>> complete >>>> examples on using the read method, no mention of "reading" >> through a >>>> writeStream endpoint. >>>> >>>> Marco, I agree that that read has more detailed control of >> minimum >>> incoming >>>> content. However I wonder if it would be more efficient to >> default >>>> pipe.chunkSize to a "lowWatermark" of the receiver (if >>>> defined). >>> This >>>> lowWatermark could be adjusted dynamically and the callback >> in the >>> writable >>>> should keep sequence of events under control? >>>> >>>> Anyway, thanks Node team, I'm very impressed! >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> On Wednesday, March 20, 2013 4:45:32 AM UTC-4, Marco Rogers >> wrote: >>>>> >>>>> @Nathan's response is right. Creating a writable stream is >>> preferable in >>>>> most cases. But I wanted to add a little context to that. If >>> you're dealing >>>>> with a base readable stream, it's just pushing chunks of >> data at >>> you off the >>>>> wire. Your first task is to collect those chunks into >> meaningful >>> data. So >>>>> IMO the reason creating a writable stream is preferable is >> because it >>>>> prompts you not just read off the stream, but to create >> semantics >>> around >>>>> what the new stream is supposed to be. The api reflects this >>> opinion and >>>>> that's why creating writable streams feels like the more >> natural >>> way, and >>>>> the ugliness of dealing with read() is wrapped up in the >>>>> pipe() >>> method. It >>>>> was kind of designed that way. >>>>> >>>>> But the read() api was also designed for a use case. It's >>>>> meant >>> to handle >>>>> low/high water marks effectively, as well as enable more >>> optimized special >>>>> parsing by reading off specific lengths of chunks. These >>>>> were >>> things that >>>>> people kept needing, but the old api didn't support well. >> If you were >>>>> writing a library for a special parser, you might write a >> custom >>> Writable >>>>> stream and inside it you would be using the read(n) api to >>> control *how* you >>>>> read data off the socket. I hope that makes sense. >>>>> >>>>> :Marco >>>>> >>>>> On Monday, March 18, 2013 11:06:48 AM UTC-7, Sigurgeir >> Jonsson wrote: >>>>>> >>>>>> The new streams have excellent support for high/low >> watermarks and >>>>>> auto-pausing/resuming, but the documentation confuses me a >> little... >>>>>> particularly the read method. >>>>>> >>>>>> When I read the new docs for the first time I was under >>>>>> the >>> impression >>>>>> that the optimal way to become a user of a stream is to >>>>>> write >>> loops around >>>>>> the read functio. However in practice I find myself >>>>>> simply >>> writing custom >>>>>> writeStreams and use the callback to control upstream >>>>>> pressure >>> (in addition >>>>>> to source Watermarks if needed). Here is an example >>>>>> where I >>> move the >>>>>> output to a queue that executes a custom function in >> parallel (i.e. >>>>>> uploading to a database) >> https://gist.github.com/ZJONSSON/5189249 >>>>>> >>>>>> Are there any benefits to using the read method directly >>>>>> on a >>> stream vs. >>>>>> piping to a custom Writable stream? >>>> >>>> -- -- Job Board: http://jobs.nodejs.org/ Posting guidelines: >>>> >> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines >> >> >> >>> You received this message because you are subscribed to the >> Google >>>> Groups "nodejs" group. To post to this group, send email to >>>> nodejs@googlegroups.com >> <mailto:nodejs@googlegroups.com> >>> <mailto:nodejs@googlegroups.com >>> <mailto:nodejs@googlegroups.com>> >>>> To unsubscribe from this group, send email to >>>> nodejs+unsubscr...@googlegroups.com >> <mailto:nodejs%2bunsubscr...@googlegroups.com> >>> <mailto:nodejs%2bunsubscr...@googlegroups.com >> <mailto:nodejs%252bunsubscr...@googlegroups.com>> >>>> For more options, visit this group at >>>> http://groups.google.com/group/nodejs?hl=en?hl=en >>>> >>>> --- You received this message because you are subscribed to >>>> the >> Google >>> Groups >>>> "nodejs" group. To unsubscribe from this group and stop >>>> receiving emails >> from it, >>> send an >>>> email to nodejs+unsubscr...@googlegroups.com >> <mailto:nodejs%2bunsubscr...@googlegroups.com> >>> <mailto:nodejs%2bunsubscr...@googlegroups.com >> <mailto:nodejs%252bunsubscr...@googlegroups.com>>. >>>> For more options, visit >> https://groups.google.com/groups/opt_out. >>>> >>>> >>> >>> -- -- Job Board: http://jobs.nodejs.org/ Posting guidelines: >>> >> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines >> >> >> > You received this message because you are subscribed to the >> Google >>> Groups "nodejs" group. To post to this group, send email to >>> nodejs@googlegroups.com >> <mailto:nodejs@googlegroups.com> >>> <mailto:nodejs@googlegroups.com <mailto:nodejs@googlegroups.com>> >>> To unsubscribe from this group, send email to >>> nodejs+unsubscr...@googlegroups.com >> <mailto:nodejs%2bunsubscr...@googlegroups.com> >>> <mailto:nodejs%2bunsubscr...@googlegroups.com >> <mailto:nodejs%252bunsubscr...@googlegroups.com>> >>> For more options, visit this group at >>> http://groups.google.com/group/nodejs?hl=en?hl=en >>> >>> --- You received this message because you are subscribed to the >>> Google Groups "nodejs" group. To unsubscribe from this group and >>> stop receiving emails from it, send an email to >>> nodejs+unsubscr...@googlegroups.com >> <mailto:nodejs%2bunsubscr...@googlegroups.com> >>> <mailto:nodejs%2bunsubscr...@googlegroups.com >> <mailto:nodejs%252bunsubscr...@googlegroups.com>>. >>> For more options, visit >>> https://groups.google.com/groups/opt_out. >>> >>> >>> >>> -- -- Job Board: http://jobs.nodejs.org/ Posting guidelines: >>> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines >> >>> >>> >> You received this message because you are subscribed to the Google >>> Groups "nodejs" group. To post to this group, send email to >>> nodejs@googlegroups.com >> <mailto:nodejs@googlegroups.com> >>> To unsubscribe from this group, send email to >>> nodejs+unsubscr...@googlegroups.com >> <mailto:nodejs%2bunsubscr...@googlegroups.com> >>> For more options, visit this group at >>> http://groups.google.com/group/nodejs?hl=en?hl=en >>> >>> --- You received this message because you are subscribed to the >>> Google Groups "nodejs" group. To unsubscribe from this group and >>> stop receiving emails from it, send an email to >>> nodejs+unsubscr...@googlegroups.com >> <mailto:nodejs%2bunsubscr...@googlegroups.com>. >>> For more options, visit >>> https://groups.google.com/groups/opt_out. >>> >>> >> >> > > -- > -- > Job Board: http://jobs.nodejs.org/ > Posting guidelines: > https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines > You received this message because you are subscribed to the Google > Groups "nodejs" group. > To post to this group, send email to nodejs@googlegroups.com > To unsubscribe from this group, send email to > nodejs+unsubscr...@googlegroups.com > For more options, visit this group at > http://groups.google.com/group/nodejs?hl=en?hl=en > > --- > You received this message because you are subscribed to the Google Groups > "nodejs" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to nodejs+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/groups/opt_out. > > -- -- Job Board: http://jobs.nodejs.org/ Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines You received this message because you are subscribed to the Google Groups "nodejs" group. To post to this group, send email to nodejs@googlegroups.com To unsubscribe from this group, send email to nodejs+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/nodejs?hl=en?hl=en --- You received this message because you are subscribed to the Google Groups "nodejs" group. To unsubscribe from this group and stop receiving emails from it, send an email to nodejs+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
