On Fri, Mar 08, 2013 at 09:30:30PM -0500, Jonathan M Davis wrote: > On Saturday, March 09, 2013 01:59:33 Stewart Gordon wrote: > > On 07/03/2013 12:07, Steven Schveighoffer wrote: > > <snip> > > > > > I don't really understand the need to make ranges into streams. > > > > <snip> > > > > Ask Walter - from what I recall it was his idea to have range-based > > file I/O to replace std.stream. > > > > Thikning about it now, a range-based interface might be good for > > reading files of certain kinds, but isn't suited to general file > > I/O. > > In general, ranges should work just fine for I/O as long as they have > an efficient implementation which underneathbuffers (and preferably > makes them forward ranges). Aside from how its implemented internally, > there's no real difference between operating on a range over a file > and any other range. The trick is making it efficient internally. > Doing something like reading a character at a time from a file every > time that popFront is called would be horrible, but with buffering, it > should be just fine. Now, you're not going to get a random-access > range that way, but it should work fine as a forward range, and > std.mmfile will probably give you want you want if an RA range is what > you really need (and that, we have already). [...]
I think the new std.stream should have a low-level stream API based on reading & simultaneously advancing by n bytes. This is still the most efficient approach for low-level file I/O. On top of this core, we can provide range-based APIs which are backed by buffers implemented using the stream API. Conceptually, it could be something like this: module std.stream; struct FileStream { File _impl; ... // Low-level stream API void read(T)(ref T[] buffer, size_t n); bool eof(); } struct BufferedStream(T, SrcStream) { SrcStream impl; T[] buffer; size_t readPos; enum BufSize = ...; // some suitable value this() { buffer.length = BufSize; } // Range API T front() { return buffer[readPos]; } bool empty() { return impl.eof && readPos >= buffer.length; } void popFront() { if (++readPos >= buffer.length) { // Load next chunk of file into buffer impl.read(buffer, BufSize); readPos = 0; } } } Suitable adaptor functions/structs/etc. can be used for automatically converting between streams and range APIs via BufferedStream, etc.. As for making ranges into streams: it could be useful for transparently substituting, say, a string buffer for file input for generic code that operates on streams. I'm not sure if ranges are the right thing to use here, though; if all you have is an input stream, then generic code that uses BufferedStream on top that would be horribly inefficient. It may make more sense to require an array. Another approach could be to extend the idea of a range, to have, for lack of a better term, a StreamRange or something of the sort, that provides a read() method (or maybe more suitably named, like copyFrontN() or something along those lines) that is equivalent to copying .front and calling popFront n times. But we already have trouble taming the current variety of ranges, so I'm not sure if this is a good idea or not. Jonathan probably will hate the idea of introducing yet another range type to the mix. :) T -- "How are you doing?" "Doing what?"