Hi,
first of all, thanks for the many replies. I was not aware that this
issue was unknown. I can try debugging the issue, I have access to
enough memory, but little to no knowledge of perls internals.
I ran the following test:
for $i (0..2**27) { $str.='abcdefghijklmnopqrstuvwxyz0123456789';},
that's a bit more than 2**32 (4GB). Took a few seconds to run.
Then
perldl> p length ($str)
4831838244
perldl> p length ($str)/1024/1024/1024
4.50000003352761
So I conclude it is not an underlying perl/string limitation, correct?
Ingo
PS: My machine is unstable gentoo ~amd64, we have ubuntu boxes also.
uname -a
Linux spectre 2.6.33-gentoo-r2 #4 SMP Thu Jul 29 12:26:35 CEST 2010
x86_64 Intel(R) Xeon(R) CPU W3520 @ 2.67GHz GenuineIntel GNU/Linux
12GB RAM
perl -V:
Summary of my perl5 (revision 5 version 12 subversion 1) configuration:
Platform:
osname=linux, osvers=2.6.33-gentoo, archname=x86_64-linux
uname='linux spectre 2.6.33-gentoo #2 smp tue apr 6 10:24:11 cest
2010 x86_64 intel(r) xeon(r) cpu w3520 @ 2.67ghz genuineintel gnulinux '
config_args='-des -Duseshrplib -Darchname=x86_64-linux
-Dcc=x86_64-pc-linux-gnu-gcc -Doptimize=-O2 -pipe -march=core2
-fomit-frame-pointer -msse4 -msse4.1 -msse4.2 -mcx16 -msahf
-Dprefix=/usr -Dsiteprefix=/usr -Dvendorprefix=/usr
-Dprivlib=/usr/lib64/perl5/5.12.1
-Darchlib=/usr/lib64/perl5/5.12.1/x86_64-linux
-Dsitelib=/usr/lib64/perl5/site_perl/5.12.1
-Dsitearch=/usr/lib64/perl5/site_perl/5.12.1/x86_64-linux
-Dvendorlib=/usr/lib64/perl5/vendor_perl/5.12.1
-Dvendorarch=/usr/lib64/perl5/vendor_perl/5.12.1/x86_64-linux
-Dman1dir=/usr/share/man/man1 -Dman3dir=/usr/share/man/man3
-Dsiteman1dir=/usr/share/man/man1 -Dsiteman3dir=/usr/share/man/man3
-Dvendorman1dir=/usr/share/man/man1 -Dvendorman3dir=/usr/share/man/man3
-Dman1ext=1 -Dman3ext=3pm -Dlibperl=libperl.so.5.12.1 -Dlocincpth=
-Duselargefiles -Dd_semctl_semun -Dcf_by=Gentoo -Dmyhostname=localhost
-dperladmin=r...@localhost -Dinstallusrbinperl=n -Ud_csh -Uusenm
-Di_ndbm -Di_gdbm -Di_db -Dinc_version_list=5.12.0 5.12.0/x86_64-linux
-Dusrinc=/usr/include -Dlibpth=/usr/local/lib64 /lib64 /usr/lib64'
hint=recommended, useposix=true, d_sigaction=define
useithreads=undef, usemultiplicity=undef
useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
use64bitint=define, use64bitall=define, uselongdouble=undef
usemymalloc=n, bincompat5005=undef
Compiler:
cc='x86_64-pc-linux-gnu-gcc', ccflags ='-fno-strict-aliasing -pipe
-fstack-protector -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
optimize='-O2 -pipe -march=core2 -fomit-frame-pointer -msse4
-msse4.1 -msse4.2 -mcx16 -msahf',
cppflags='-fno-strict-aliasing -pipe -fstack-protector'
ccversion='', gccversion='4.4.4', gccosandvers=''
intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t',
lseeksize=8
alignbytes=8, prototype=define
Linker and Libraries:
ld='x86_64-pc-linux-gnu-gcc', ldflags =' -fstack-protector'
libpth=/usr/local/lib64 /lib64 /usr/lib64
libs=-lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc -lgdbm_compat
perllibs=-lnsl -ldl -lm -lcrypt -lutil -lc
libc=/lib/libc-2.11.2.so, so=so, useshrplib=true,
libperl=libperl.so.5.12.1
gnulibc_version='2.11.2'
Dynamic Linking:
dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
cccdlflags='-fPIC', lddlflags='-shared -O2 -pipe -march=core2
-fomit-frame-pointer -msse4 -msse4.1 -msse4.2 -mcx16 -msahf
-fstack-protector'
Characteristics of this binary (from libperl):
Compile-time options: PERL_DONT_CREATE_GVSV PERL_MALLOC_WRAP
USE_64_BIT_ALL
USE_64_BIT_INT USE_LARGE_FILES USE_PERLIO
USE_PERL_ATOF
Locally applied patches:
0001-gentoo_MakeMaker-RUNPATH.diff
0002-gentoo_config__over.diff
0003-gentoo_cpan__definstalldirs.diff
0004-gentoo_cpanplus__definstalldirs.diff
0005-gentoo_create-libperl-soname.diff
0006-gentoo_MakeMaker-delete__packlist.diff
0007-fixes_8d66b3f9__h2hp__fix.diff
0008-fixes_ef9df645__glob__crashes__when__File__Glob__is__empty.diff
0009-fixes_e3d01d03__Naif__calls__segfault__T__PRTOBJ__of__the__stock__typemap.diff
Built under linux
Compiled at Jul 26 2010 11:18:49
@INC:
/usr/lib64/perl5/site_perl/5.12.1/x86_64-linux
/usr/lib64/perl5/site_perl/5.12.1
/usr/lib64/perl5/vendor_perl/5.12.1/x86_64-linux
/usr/lib64/perl5/vendor_perl/5.12.1
/usr/lib64/perl5/5.12.1/x86_64-linux
/usr/lib64/perl5/5.12.1
/usr/lib64/perl5/site_perl
/usr/lib64/perl5/vendor_perl
On 08/04/2010 03:59 AM, Chris Marshall wrote:
> On 8/3/2010 9:43 PM, Christian Soeller wrote:
>> Is it possible to change things so that 64 bit sizes
> > can be passed in the two places you identified and see
> > if that works?
>> I appreciate that things could still fall over in various
> > PP autogenerated code pieces if ints are used for offset
> > calculations in slice and other vaffine operations.
>
> It could work. The problem is it needs someone with
> a 64bit OS, lots of memory, and a willingness to
> debug the issue. I don't have any *large* memory
> systems at the moment. Not that I wouldn't like to
> have one. :-)
>
> --Chris
>
>
>> On 4/08/2010, at 12:46 PM, Chris Marshall wrote:
>>
>>> I took a further look at the SvGROW calls in
>>> PDL/Basic/Core routines and the two that I found
>>> both use int type for their sizes. That would
>>> limit a piddle size to<2**31 or about 2GB.
>>>
>>> It looks like 64bit support for PDL may need
>>> to be added to the list for the future. I don't
>>> know the scope of the changes that would be
>>> required to support larger PDL data objects.
>>>
>>> --Chris
>>>
>>> On 8/3/2010 8:39 PM, Chris Marshall wrote:
>>>> On 8/3/2010 8:30 PM, P Kishor wrote:
>>>>> On Tue, Aug 3, 2010 at 7:19 PM, Chris Marshall<[email protected]>
>>>>> wrote:
>>>>>> On 8/3/2010 8:01 PM, P Kishor wrote:
>>>>>>> also on 64-bit Snow Leopard (Mac OS X 10.6.4)
>>>>>>>
>>>>>>> punk...@lucknow ~$ perl -MPDL -e '$PDL::BIGPDL=1; $x = sequence(float,
>>>>>>> 23171, 23171); print $x->info("%M")."\n"'
>>>>>>> perl(85899) malloc: *** mmap(size=18446744071562166272) failed (error
>>>>>>> code=12)
>>>>>>> *** error: can't allocate region
>>>>>>> *** set a breakpoint in malloc_error_break to debug
>>>>>>> Out of memory!
>>>>>>> punk...@lucknow ~$
>>>>>> What is perl -V?
>>>> I looked at the PDL/Basic/Core stuff and it looks like
>>>> if SvGROW can handle a>2GB string then, in principle,
>>>> PDL should be able to handle piddles of that size.
>>>>
>>>> Could you see if you can create a string more than 2GB
>>>> long? It might take a while but it will tell us if the
>>>> limit is perl or PDL. Since the PDL routines for growing
>>>> a new piddle use 4byte ints for their sizes (rather than
>>>> size_t objects), it is pretty clear that there is a bug
>>>> in the PDL allocation stuff if perl can handle the longer
>>>> strings.
>>>>
>>>> --Chris
>>> _______________________________________________
>>> Perldl mailing list
>>> [email protected]
>>> http://mailman.jach.hawaii.edu/mailman/listinfo/perldl
>>
>>
>>
>> No virus found in this incoming message.
>> Checked by AVG - www.avg.com
>> Version: 9.0.851 / Virus Database: 271.1.1/3049 - Release Date: 08/03/10
>> 10:22:00
>>
>
> _______________________________________________
> Perldl mailing list
> [email protected]
> http://mailman.jach.hawaii.edu/mailman/listinfo/perldl
>
_______________________________________________
Perldl mailing list
[email protected]
http://mailman.jach.hawaii.edu/mailman/listinfo/perldl
