Simple - don't use array append and don't produce and array. Just produce a lazy forward range that is iterated.At the very least use Appender!(Token[]) it ought to be much faster.
But hasn't each range an array internally? Or how does that work?I try to use Appender, but this lazy forward range sounds interesting.
