I want to combine block reads with lazy conversion of utf-8 characters to dchars. Solution I came with is in the program below. This works fine. Has good performance, etc.

Question I have is if there is a better way to do this. For example, a different way to construct the lazy 'decodeUTF8Range' rather than writing it out in this fashion. There is quite a bit of power in the library and I'm still learning it. I'm wondering if I overlooked a useful alternative.

--Jon

Program:
-----------

import std.algorithm: each, joiner, map;
import std.conv;
import std.range;
import std.stdio;
import std.traits;
import std.utf: decodeFront;

auto decodeUTF8Range(Range)(Range charSource)
if (isInputRange!Range && is(Unqual!(ElementType!Range) == char))
{
    static struct Result
    {
        private Range source;
        private dchar next;

        bool empty = false;
        dchar front() @property { return next; }
        void popFront() {
            if (source.empty) {
                empty = true;
                next = dchar.init;
            } else {
                next = source.decodeFront;
            }
        }
    }
    auto r = Result(charSource);
    r.popFront;
    return r;
}

void main(string[] args)
{
if (args.length != 2) { writeln("Provide one file name."); return; }

    ubyte[1024*1024] rawbuf;
    auto inputStream = args[1].File();
    inputStream
        .byChunk(rawbuf)        // Read in blocks
.joiner // Join the blocks into a single input char range .map!(a => to!char(a)) // Cast ubyte to char for decodeFront. Any better ways?
        .decodeUTF8Range        // utf8 to dchar conversion.
        .each;                  // Real work goes here.
    writeln("done");
}

Reply via email to