Hm I'm not sure. I reran the benchmark with a 100 million size array this
time. I got this number from @allocation
101077550864
That's 101 GB! My machine only has 16GB. Is @allocation reporting the total
amount of memory used by any chance rather than the max allocated?
Activity monitor shows a continuous memory usage of 1.6 GB which is still
much more than the 800MB needed for the array (but maybe that's due to how
the VM allocates memory?)
On Sunday, July 6, 2014 4:28:02 PM UTC-4, Kevin Squire wrote:
>
> It's likely that much of the memory was for temporary allocations that
> were reused.
>
> Cheers,
> Kevin
>
>
> On Sun, Jul 6, 2014 at 10:32 AM, Sid <tmf...@gmail.com <javascript:>>
> wrote:
>
>> Regarding memory allocation -- it seems like it might be a bug in the
>> @allocated macro rather than an actual indication of memory usage.
>>
>> When I run
>>
>> a = rand(100000)
>> @allocated sort(a, by=x->x)
>>
>> I get 768403668 which is 768 MB.
>>
>> However, I was also recording the process's memory usage using %mem. For
>> my 16 GB machine, the usage initially is 0.5% and jumps to 1.3% after
>> running the above two commands which comes to around ~200MB rather than
>> 768MB.
>>
>> While that's still a high jump compared to the array size (which is only
>> ~8MB), that might be because of the way VM allocates more memory. Perhaps
>> someone who's more familiar with the internals can comment.
>>
>> On Sunday, July 6, 2014 7:30:47 AM UTC-4, gentlebeldin wrote:
>>>
>>> No, that wasn't the reason, in this case. I had another, hard look at my
>>> own (rough and ready) implementation of factoring, found a rather stupid
>>> mistake, and now, it runs in 2.2 seconds.
>>> Using List<Integer> or List<Double> instead of int[] or double[] for
>>> numerical computations would be a silly idea, indeed, and not just because
>>> of poor performance. It would also eat up lots of memory. In this case, I
>>> had to sort a big array of integers (100,000,000 elements), with a custom
>>> comparator. That isn't possible with Java's int[], and Integer[] with so
>>> many elements is a bit harsh for a netbook with 2 GB of RAM. Then, I
>>> thought "hey, I can implement the List<Integer> interface with an int[]!"
>>> No such luck, the implementation of Collections.sort calls toArray,
>>> first, ->Integer[] :-(
>>> Then, they introduced streams in Java 8, including IntStream, a stream
>>> of ints, for better performance, and defined the corresponding functional
>>> interface for a custom comparator. That sounds promising, but as we know,
>>> most promises aren't kept. Believe it or not, the implementation of
>>> IntStream.sort collects the whole stream in an Integer[], first. :-(
>>> That's why I put some hope into Julia. Unfortunately, it may allocate
>>> huge amounts of memory in an unpredictable way (for me, at least), too.
>>>
>>> Am Sonntag, 6. Juli 2014 07:12:26 UTC+2 schrieb Sid:
>>>>
>>>> Are you using Integer/Double objects by any chance in Java? I've found
>>>> using them to be a surefire way of destroying Java's performance...
>>>>
>>>> On Saturday, July 5, 2014 3:11:47 AM UTC-4, gentlebeldin wrote:
>>>>>
>>>>> It's hard to tell for sure, I wouldn't know how to check that. I
>>>>> prefer to port a few more solutions. My curiosity was much increased by
>>>>> this one:
>>>>>
>>>>> julia> include("julia/jl/e443gcd.jl")
>>>>> elapsed time: 3.255950529 seconds (929628 bytes allocated)
>>>>>
>>>>>
>>>>> The Java original "runs" much longer, 33 seconds.
>>>>>
>>>>>
>>>>>>
>
