On 6/12/18 3:08 AM, Anton Fediushin wrote:
On Sunday, 10 June 2018 at 20:10:31 UTC, Steven Schveighoffer wrote:
iopipe version 0.1.0 has been released.
iopipe is a high-performance pipe processing system that makes it easy
to string together pipelines to process data with as little buffer
copying as possible.
I saw iopipe a while back, but never looked at it closely. Now I did
and... it implements its own kind of pattern and not ranges?
Correct, although it's very similar to ranges.
I guess it
is done for the sake of performance, but how easy it is to use iopipe
with standard range-based methods from std.algorithm for example?
Very simple. You just have to define what is an "element" of a range
that is a sliding window of data.
For example:
https://github.com/schveiguy/iopipe/blob/master/source/iopipe/textpipe.d#L542
Note that asInputRange simply treats the current window as "front", and
"popFront" discards that window and loads the next. It's a crude but
effective tool to convert any iopipe into a range.
The reason iopipe is not based on ranges exactly (the window must be a
random-access range), is because ranges don't handle i/o very
performantly. E.g. I would never use lockingTextReader to process text
data, it would be slow as hell, and too limiting.
File.byLine is fast, but only because of the underlying non-range tricks
it uses to achieve performance. And iopipe still is 2x faster.
As long as it's easy to use with the rest of the phobos - I'd like to
see it in the standard library.
It should be easy to use on its own, and with algorithms from phobos.
I've done so in some of the toy parsers I've written.
I think the one sticking point (which I'm not sure how to reconcile, but
we can probably hash it out) is that for iopipes, char and wchar arrays
are arrays, not auto-decoding ranges.
-Steve
