I ran the program through Callgrind and I found the performance bottleneck.
The program spends most of its time cloning values that are only used once.
I applied the following patch that reduces the run time from a few minutes
to a fraction of a second.
Jürgen, could you take a look at it and see if the fix is sound? The
performance difference is really remarkable, and if it's correct, it's
really worth applying.
Regards,
Elias
On 25 April 2014 00:21, Elias Mårtenson <loke...@gmail.com> wrote:
> In writing a function that uses lib_file_io to load the content of an
> entire file into an array of strings, I came across a bad performance
> problem that I am having trouble narrowing down.
>
> Here is my current version of the function:
> https://github.com/lokedhs/apl-tools/blob/e3e81816f3ccb4d8c56acc8e4012d53f05de96d6/io.apl#L8
>
> The first version did not do blocked reads and resized the array after
> each row was read. That was terribly slow, so I preallocate a block of 1000
> elements, and resize every 1000 lines, giving the version you can see
> linked above.
>
> I was testing with a text file containing almost 14000 rows, and on my
> laptop it takes many minutes to load the file. One would expect that the
> time taken to load such a small file should not take any noticeable time at
> all.
>
> One interesting aspect of this is that it takes longer and longer to load
> each row as the loading proceeds. I have no explanation as to why that is
> the case. It's not the resizing that takes time, I was measuring the time
> taken to load a block of rows *excluding* the array resize.
>
> Any ideas?
>
> Regards,
> Elias
>
Index: src/Value.cc
===================================================================
--- src/Value.cc (revision 222)
+++ src/Value.cc (working copy)
@@ -1678,6 +1678,10 @@
Value_P
Value::clone(const char * loc) const
{
+ if(is_temp()) {
+ return Value_P(const_cast<Value *>(this));
+ }
+
Value_P ret(new Value(get_shape(), loc));
const Cell * src = &get_ravel(0);
@@ -1686,6 +1690,7 @@
loop(c, count) dst++->init(*src++);
+ ret->SET_temp();
ret->check_value(LOC);
return ret;
}
