On Wed, May 16, 2012 at 04:15:22PM -0400, Steven Schveighoffer wrote: > On Wed, 16 May 2012 15:38:02 -0400, H. S. Teoh > <hst...@quickfur.ath.cx> wrote: [...] > >One direction that _could_ be helpful, perhaps, is to extend the concept > >of range to include, let's tentatively call it, a ChunkedRange. > >Basically a ChunkedRange implements the usual InputRange operations > >(empty, front, popfront) but adds the following new primitives: > > > >- bool hasAtLeast(R)(R range, int n) - true if underlying range has at > > least n elements left; > > > >- E[] frontN(R)(R range, int n) - returns a slice containing the front n > > elements from the range: this will buffer the next n elements from the > > range if they aren't already; repeated calls will just return the > > buffer; > > > >- void popN(R)(R range, int n) - discards the first n elements from the > > buffer, thus causing the next call to frontN() to fetch more data if > > necessary. > > > > On such ranges, what would popFront and front do? I'm assuming since > frontN and popN are referring to how many elements, and since the most > logical definition for elements is bytes, that front gets the next > byte, and popFront discards the next byte. This seems useless to me.
How so? It's still useful for implementing readByte, for example. > I still don't get the need to "add" this to ranges. The streaming API > works fine on its own. > > But there is an omission with your proposed API regardless -- > reading data is a mutating event. It destructively mutates the > underlying data stream so that you cannot get the data again. This > means you must double-buffer data in order to support frontN and > popN that are not necessary with a simple read API. > > For example: > > auto buf = new ubyte[1000000]; > stream.read(buf); > > does not need to first buffer the data inside the stream and then > copy it to buf, it can read it from the OS *directly* into buf. [...] The idea is that by asking for N elements at a time instead of calling front/popFront N times, the underlying implementation can optimize the request by creating a buffer of size N and have the OS read exactly N bytes directly into that buffer. // Reads 1,000,000 bytes into newly allocated buffer and returns // buffer. auto buf = stream.frontN(1_000_000); // Since 1,000,000 bytes is already read into the buffer, this // simply returns a slice of the same buffer: auto buf2 = stream.frontN(1_000_000); assert(buf is buf2); // This consumes the buffer: stream.popN(1_000_000); // This will read another 1,000,000 bytes into a new buffer auto buf3 = stream.frontN(1_000_000); // This returns the same buffer as buf3 since we already have // the data available. auto buf4 = stream.frontN(1_000_000); T -- English has the lovely word "defenestrate", meaning "to execute by throwing someone out a window", or more recently "to remove Windows from a computer and replace it with something useful". :-) -- John Cowan