On Jul 28, 2012, at July 28, 201212:05 PM, Tim Caswell <[email protected]> wrote:
> FWIW, I actually like Bruno's proposal. It doesn't cover all the use > cases, but it makes backpressure enabled pumps really easy. > > One use case missing that's easy to add is when consuming a binary > protocol, I often only want part of the input. For example, I might > want to get the first 4 bytes, decode that as a uint32 length header > and then read n more bytes for the body. Without being able to > request how many bytes I want, I have to handle putting data back in > the stream that I don't need. That's very error prone and tedious. > So on the read function, add an optional "maxBytes" or "bytes" > parameter. The difference is in the maxBytes case, I want the data as > soon as there is anything, even if it's less than the number of bytes > I want. In the "bytes" case I want to wait till that many bytes are > available. Both are valid for different use cases. The early stuff I saw included a "length" option. > > Also streams (both readable and writable) need a configurable > low-water mark. I don't want to wait till the pipe is empty before I > start piping data again. This mark would control how soon writable > streams called my write callback and how much readable streams would > readahead from their data source before waiting for me to call read. > I want to keep it always full. It would be great if this was handled > internally in the stream and consumers of the stream simply configured > what the mark should be. I think you're missing how this works. Nobody automatically asks for data so watermarks aren't strictly necessary. You ask for data if it's available and you read as much as you can handle. There is no "readahead". If someone stops calling read() then the buffer fills and, if it's a TCP stream, it's asked to stop sending data. Remember that when the "readable" event goes off it's expected that the pending data is read in the same event loop cycle.
