On 12/24/13 2:28 PM, Gordon wrote:
Hello,
I want to load a large text file containing two numeric fields into an
associative array.
The file looks like:
1 40
4 2
42 11
...
And has 11M lines.
My code looks like this:
===
void main()
{
size_t[size_t] unions;
auto f = File("input.txt");
foreach ( line ; f.byLine() ) {
auto fields = line.split();
size_t i = to!size_t(fields[0]);
size_t j = to!size_t(fields[1]);
unions[i] = j; // <-- here be question
}
}
===
This is just a test code to illustrate my question (though general
comments are welcomed - I'm new to D).
Commenting out the highlighted line (not populating the hash), the
program completes in 25 seconds.
Compiling with the highlighted line, the program takes ~3.5 minutes.
Is there a way to speed the loading? perhaps reserving memory in the
hash before populating it? Or another trick?
void main()
{
size_t[size_t] unions;
foreach (e; "input.txt"
.slurp!(size_t, size_t)("%s %s").sort.uniq ) {
unions[e[0]] = e[1];
}
}
Andrei
