Have you tried the profiler, to see where the allocations are happening?
ProfileView.jl makes it relatively simple, because it highlights in red lines
that trigger garbage collection. But be sure to read the "Import points"
section at the end of the README.
--Tim
On Monday, March 10, 2014 01:30:30 PM Keith Campbell wrote:
> Hi all,
>
> I'm trying to minimize memory allocation while doing line-oriented
> processing on a fairly large set of text files. SubString and
> pre-allocated outputs have helped, but I'm still getting memory allocations
> proportional to the size of the input set and looking for new ideas.
>
> The toy example below illustrates how the allocations grow.
> Am I right to suspect that float() is the culprit. Any thoughts for how to
> cut out the remaining allocations?
>
> thanks,
> Keith
>
> julia> function str_with_sub(N)
> mystr = ascii("1.1,2.2")
> fs=Array(Float64,2)
>
> for i in 1:N
> dostr!(mystr, fs)
> end
> end
> str_with_sub (generic function with 1 method)
>
> julia> function dostr!(mystr, fs)
> fs[1] = float(SubString(mystr,1,3))
> fs[2] = float(SubString(mystr,5,7))
> end
> dostr! (generic function with 1 method)
>
> julia> @time str_with_sub(4)
> elapsed time: 0.008214327 seconds (190612 bytes allocated)
>
> julia> @time str_with_sub(4)
> elapsed time: 8.441e-6 seconds (496 bytes allocated)
>
> julia> @time str_with_sub(4)
> elapsed time: 6.493e-6 seconds (496 bytes allocated)
>
> julia> @time str_with_sub(6)
> elapsed time: 7.074e-6 seconds (688 bytes allocated)
>
> julia> @time str_with_sub(8)
> elapsed time: 7.437e-6 seconds (880 bytes allocated)
