Re: Streaming library

[Andrei Alexandrescu]

On 10/13/10 14:02 CDT, Denis Koroskin wrote:
On Wed, 13 Oct 2010 20:55:04 +0400, Andrei Alexandrescu
<seewebsiteforem...@erdani.org> wrote:
http://www.gnu.org/s/libc/manual/html_node/Buffering-Concepts.html
I don't think streams must mimic the low-level OS I/O interface.


I in contrast think that Streams should be a lowest-level possible
platform-independent abstraction.
No buffering besides what an OS provides, no additional functionality.
If you need to be able to read something up to some character (besides,
what should be considered a new-line separator: \r, \n, \r\n?), this
should be done manually in "byLine".

This aggravates client code for the sake of simplicity in a library that 
was supposed to make streaming easy. I'm not seeing progress.

That's because
most of the steams are binary streams, and there is no such thing as a
"line" in them (e.g. how often do you need to read a line from a
SocketStream?).

http://www.opengroup.org/onlinepubs/009695399/functions/isatty.html


These are special cases I don't like. There is no such thing in Windows
anyway.

I didn't say I like them. Windows has _isatty: 
http://msdn.microsoft.com/en-us/library/f4s0ddew(v=VS.80).aspx

You need a line when e.g. you parse a HTML header or a email header or
an FTP response. Again, if at a low level the transfer occurs in
blocks, that doesn't mean the API must do the same at all levels.


BSD sockets transmits in blocks. If you need to find a special sequence
in a socket stream, you are forced to fetch a chunk, and manually search
for a needed sequence. My position is that you should do it with an
external predicate (e.g. read until whitespace).

Problem is how you set up interfaces to avoid inefficiencies and 
contortions in the client.

I don't think streams should buffer anything either (what an underlying
OS I/O API caches should suffice), buffered streams adapters can do that
in a stream-independent way (why duplicate code when you can do that as
efficiently with external methods?).

Most OS primitives don't give access to their own internal buffers.
Instead, they ask user code to provide a buffer and transfer data into
it.

Right. This is why Stream may not cache.

This is a big misunderstanding. If the interface is:

size_t read(byte[] buffer);

then *I*, the client, need to provide the buffer. It's in client space. 
This means willing or not I need to do buffering, regardless of whatever 
internal buffering is going on under the wraps.

So clearly buffering on the client side is a must.


I don't see how is it implied from above.

Please implement an abstraction that given this:

interface InputStream
{
    size_t read(ubyte[] buf);
}

defines a line reader.


Andrei