Does the currently proposed streaming (on digitalmars.d) work for such
needs?
Andrei
On 9/27/10 1:34 AM, Shin Fujishiro wrote:
Thus I think we need a buffering layer that exposes a randomly
accessible array to upper layers. ByLine() can be easily and
efficiently implemented with the following primitives defined:
Buffer
{
// The entire buffer.
ubyte[] buffer();
// Slice of buffer() where data is available.
ubyte[] available();
// Moves the beginning of available() by n in buffer().
void bump(sizediff_t n);
// Reads next blob from a source.
bool fetch();
}
Yes, cstdio-esque rawRead() is no good for high-level ByLine. What
high-level I/O entities want is: A randomly accessible buffer. Device
handles may expose block-oriented streaming primitives, but they
must be made "partially random accessible" by the buffering layer.
Shin
Andrei Alexandrescu<[email protected]> wrote:
This is a good point. Allow me to expand it a bit and say that we need a
collection of solid use cases that we want to support. Without them, we
come up with a design that might not work well with certain use patterns.
Example of a requirement: "Given a block-oriented stream, define a
line-oriented range on top of it."
Do we need such a thing? Probably. Can it be implemented efficiently
with the rawRead(ubyte[]) primitive? NO.
struct ByLine(BlockRange) {
private BlockRange _input;
private char[] store;
...
void popFront() {
// read one line from _input
}
}
The problem is that the line reader must sometimes _append_ data to its
buffer (in situations when it has read a long line that doesn't fit in
one buffer). But the input stream does not support appending.
BTW this has been a long source of irritation with C's stdio: the only
ways to read a line has been by reading one character at a time with
fgetc() (pig slow) or by using fgets() (unsafe) or fgetsn() (inefficient
for long lines) or by using getline() (nonportable, see
http://www.gnu.org/s/libc/manual/html_node/Line-Input.html).
Moral of the story? We need to have a number of well-defined scenarios
in mind when defining a streaming interface.
Andrei
On 9/23/10 8:37 CDT, SHOO wrote:
Hmm... Does it mean to have to relay three classes to do I/O processing?
auto handle = FileHandle("file");
scope (exit) handle.close();
auto buf = MemoryBuffer();
auto range = byLine(range);
I think it is slightly complicatedly. What is the reason why it must
come to look like it?
BTW, I don't know well what buffers must do. What is the requirement of
buffers?
(2010/09/21 14:05), Shin Fujishiro wrote:
SHOO<[email protected]> wrote:
I think that there are two problems about I/O operation.
- Location of buffering layers.
- Direction of seeking.
...snip...
It is necessary for the concept to be divided in two at least to realize
them. (Handles and Ranges) Or more(+ Port or Stream).
The opening difficult item appears when I think about this.
How about putting a buffering layer between the two you said? Not
only it just solves the who-does-buffering problem, but also opens a
bit of freedom in the lowermost I/O device layer.
Shin
_______________________________________________
phobos mailing list
[email protected]
http://lists.puremagic.com/mailman/listinfo/phobos
_______________________________________________
phobos mailing list
[email protected]
http://lists.puremagic.com/mailman/listinfo/phobos
Shin
_______________________________________________
phobos mailing list
[email protected]
http://lists.puremagic.com/mailman/listinfo/phobos
_______________________________________________
phobos mailing list
[email protected]
http://lists.puremagic.com/mailman/listinfo/phobos
