Andrei Alexandrescu schrieb:
On 10/11/2010 07:49 PM, Daniel Gibson wrote:
Andrei Alexandrescu schrieb:
Agreed. Maybe this is a good time to sart making a requirements list
for streams. What are the essential features/feature groups?
Andrei
Maybe something like the following (I hope it's not too extensive):
* Input- Output- and InputAndOutput- Streams
- having InputStream and OutputStream as an interface like in the old
design may be a good idea
- implementing the standard operations that are mostly independent from
the data source/sink
like read/write for basic types, strings, ... in mixin templates is
probably elegant to create
streams that are both Input and Output (one mixin that implements most
of InputStream and
one that implements most of OutputStream)
So far so good. I will point out, however, that the classic read/write
routines are not all that good. For example if you want to implement a
line-buffered stream on top of a block-buffered stream you'll be forced
to write inefficient code.
So what's a possible alternative to the classic read/write routines?
Also, a requirement that I think is essential is separation between
formatting and transport. std.stream does not have that. At the top
level there are two types of transport: text and binary. On top of that
lie various formatters.
Ok, one should differ between text and binary streams. I was mostly focused on binary streams
(because that's what I use).
So there might be a hierarchy like
* Input/Output-Stream (interface for all those read/write operations)
- BinaryStream // abstract class implementing writeInt() etc using
write(void* buf, size_t len)
* BinarySocketStream
* BinaryFileStream
* ...
- TextStream // abstract class implementing writeInt() etc using something like to!string and
write(char[])
* TextFileStream
* ...
(This for both Input- and Outputstreams)
* Two kinds of streams:
1. basic streams: reading/writing from/to:
* network (socket)
* files
* just memory (currently MemoryStream)
* Arrays/Ranges?
* ...
2. streams wrapping other streams:
* for buffering - buffer input/output/both
- with the possibility to peek?
* to modify data when it's read/written (e.g. change endianess -
important for networking!)
* custom streams.. e.g. could parse/create CSV (comma seperated values)
data or similar
Would these be streams be different in their interface?
No. I just wanted to point out that it must be possible (and should be easy) to
wrap streams.
* Also there are different types of streams: seekable, resettable (a
network stream is neither), ...
Agreed. Question: is there a file system that offers resettable but not
seekable files? I'm thinking of collapsing the two together.
As mentioned before in other branches of this thread: Probably no file system, but maybe archive
files (zip, ...)
* functionality/methods needed/desirable:
- low level access
* void read(void *buf, size_t len) // read *exactly* len bytes into buf
* void write(void *buf, size_t len) // write *exactly* len bytes from
buf to stream
- convenient methods to read/write basic types in binary (!) from/to
stream
Again, binary vs. text is a capability of the stream. For example, a tty
can never transport binary data - programs like gzip refuse to write
binary data to a terminal. (Then of course a binary stream can always
accommodate text data.)
* <type> read<Type>() (like int readInt()) or T read(T)() (like int
read!int())
Templates will be difficult for a class hierarchy.
Ok.
Another issue, as you mentioned line based streams: of course read(T)()/write(T)() would be quite
messy on them (endless static if(is(T==int)) { ... } else static if(is(T==float)) {...} etc).
(here were a lot of templated methods)
* void writeString(char[] str) // same for wchar and dchar
- could write str into the stream with its length (as ushort xor uint
xor ulong,
_not_ size_t!) prepended
* char[] readString() // same for wchar and dchar
- read length of the string and then the string itself that will be
returned
Many of these capabilities involve template methods. Is a template-based
approach preferable to a straight class hierarchy? I tend to think that
in the case of streams, classic hierarchies are most adequate.
Ok agreed.
I forgot that templated methods can't be overridden.
As writeArray(T)() etc is hardly possible, at least for strings there should be write(char[])
write(dchar[]) and write(wchar[]) (maybe with some const-stuff added). These should just write the
string to the stream, without its length.
(Analogous for read(...))
- all that array/string/low level stuff but reading *at most* len (or
array.length) values
and returning the amount actually read ( readUpTo() ?)
* useful e.g. for parsing http (you don't know how long the header is
etc)
* the same for write? don't see much use for that though..
- some way to determine whether the stream
* is at its definite end (eof on file, socket closed or something like
that)
* currently empty (for input stream) - just doing a read() would block ?
- Output streams need flush()
- for Input streams skip(size_t noBytes) or even skip(T)(size_t noElems)
may be
handy to just throw away data we're not interested in without having it
copied around - especially for non-seekable streams (network..)
OK, that's a good start. Let's toss this back and forth a few times and
see what sticks.
Andrei
Cheers,
- Daniel
