On Monday, 30 December 2013 at 21:40:58 UTC, Thomas Gann wrote:
I've written a Markov bot in D, and I have function whose job
it is to take an input string, convert all newline characters
to spaces and all uppercase letters to lowercase, and then
return an array of words that are generated by splitting the
string up by whitespace. Here is the function is question:
string[] split_sentence(string input)
{
string line;
foreach(c; input)
{
if(c == '\n' || c == '\r')
line ~= ' ';
else
line ~= c.toLower();
}
return line.splitter(' ').filter!(a => a.length).array;
}
Obviously, one issue is that because the string is immutable, I
can't modify it directly, and so I actually build an entirely
new string in place. I would have just made a mutable duplicate
of the input and modify that, but then I would get errors
returning, because it expects string[] and not char[][]. Is
there a more elegant way to do what I'm doing?
Not a huge improvement and it could be a lot faster, no doubt:
string[] split_sentence(string input)
{
import std.regex;
auto split_re = ctRegex!(r"[\n\r ]");
return input.split(split_re)
.map!toLower
.filter!(a => !a.empty)
.array();
}
You could return the result of filter instead of an array to make
it lazy and avoid an allocation if the caller is able to use it
in that form. toLower had some really slow and incorrect behavior
but I think that was fixed in 2.064. It still might be ASCII
only though. I believe std.uni has a toLower that is correct for
all of unicode.
