Sorry, I forgot to add:
/>$ perl -MPDL -e 'print $PDL::VERSION."\n"'
2.4.6
/>$ perl --version
This is perl, v5.8.8 built for x86_64-linux-thread-multi
I built PDL on this machine with standard tools (same perl, gcc 4.1.2, no fancy
stuff :-)).
Ben
On 4 Aug 2010, at 01:36, Benjamin Schuster-Boeckler wrote:
> Shockingly, it's true! I just tried the following on a machine with 4 AMD
> Opteron 64bit processors and ~8GB memory:
>
> />$ perl -MPDL -e '$PDL::BIGPDL=1; $x = sequence(float, 23170, 23170); print
> $x->info("%M")."\n"'
> 2047.92Mb
> />$ perl -MPDL -e '$PDL::BIGPDL=1; $x = sequence(float, 23171, 23171); print
> $x->info("%M")."\n"'
> Out of memory!
>
> I chose a matrix of 23170^2 floats because that's 536848900 * 4 bytes, ie
> 1.99GB. Going just a little bit above that limit gives an "Out of memory".
> What's really worrying though is this:
>
> />$ perl -MPDL -e '$PDL::BIGPDL=1; $x = sequence(float, 46350, 46350); print
> $x->info("%M")."\n"'
> -8385331.23Kb
>
> Going substantially above the existing wired memory of the machine returns
> interesting negative numbers for the matrix' size, and clearly fails to warn
> about the memory issue. I think we have a bug here?
>
> Ben
>
> On 4 Aug 2010, at 01:10, Chris Marshall wrote:
>
>> On 8/2/2010 9:13 AM, Albrecht I. Schmid wrote:
>>>
>>> since I use perl for years, I like PDL very much. But during the last
>>> months it is apparent that the 2GB limit on pdls is starting to get
>>> prohibitive.
>>
>> I'm confused about the limit on the size of piddles.
>>
>> Could you give an example? I was not aware that there
>> was any limit besides the OS limits (32bit vs 64bit)
>> and maybe the availability of contiguous memory.
>>
>>> Single datasets exceed this limit and reverting to perl loops sort of
>>> removes the performance gain of threading (maybe), besides making coding
>>> ugly, and adding additional (programming) complexity.
>>
>> If there is such a limit, I would consider it a bug
>> (at least on 64bit OSes). If it is not reported, then
>> it cannot be fixed.
>>
>>> Is there a way to enable PDL to handle bigger pdls? If not, I think I
>>> have to move on to Matlab.
>>
>> Good luck there.
>>
>>> Memory is not the issue, btw., 64 bit systems with 10s of GB RAM.
>>
>> I don't have a 64bit OS to work with. Maybe another
>> user with such an OS could comment. BTW, you might
>> want to add the output of your perl -V and the OS
>> information for your platform.
>>
>> Cheers,
>> Chris
>>
>> _______________________________________________
>> Perldl mailing list
>> [email protected]
>> http://mailman.jach.hawaii.edu/mailman/listinfo/perldl
>
> --
> Benjamin Schuster-Böckler
> Frundsbergstrasse 23a
> 82064 Strasslach
> Deutschland
>
>
>
> _______________________________________________
> Perldl mailing list
> [email protected]
> http://mailman.jach.hawaii.edu/mailman/listinfo/perldl
--
Benjamin Schuster-Böckler
Frundsbergstrasse 23a
82064 Strasslach
Deutschland
_______________________________________________
Perldl mailing list
[email protected]
http://mailman.jach.hawaii.edu/mailman/listinfo/perldl
