On Mon, Jan 4, 2010 at 1:07 AM, Cactus <rieman...@googlemail.com> wrote: > > > On Jan 4, 8:35 am, Cactus <rieman...@googlemail.com> wrote: >> On Jan 4, 3:26 am, Case Vanhorsen <cas...@gmail.com> wrote: >> >> >> >> > On Sun, Jan 3, 2010 at 12:33 PM, Cactus <rieman...@googlemail.com> wrote: >> >> > > On Jan 3, 8:15 pm, Case Vanhorsen <cas...@gmail.com> wrote: >> > >> Hello, >> >> > >> I discovered an interesting memory allocation behavior on Windows vs. >> > >> Linux. I was testing GMPY on 64-bit Windows when I stumbled into this. >> > >> GMPY replaces the native MPIR memory allocation routines with Python's >> > >> memory allocator. If I enable debugging in GMPY, I get a trace of all >> > >> the memory allocation calls. When I ran the following: >> >> > >> python -mtimeit -n 1 -r 1 -s "import >> > >> gmpy;gmpy.mpz(3)**(2**27);gmpy.set_debug(1)" "a=a*a" 2>temp.txt >> >> > >> and look at the output saved in temp.txt, I see that Linux generated >> > >> approximately 34 memory manager calls but Windows generates over >> > >> 100,000 calls. Most of the Windows allocations are for small (<8K) >> > >> chunks of memory while all the Linux requests are for more than 64K. >> > >> The performance between Linux and Windows is similar. Could it be that >> > >> Windows is not using alloca? >> >> > >> I also think I found another memory allocation bug. If I run the above >> > >> multiplication repeatedly: >> >> > >> python -mtimeit -s "import >> > >> gmpy;gmpy.mpz(3)**(2**27);gmpy.set_debug(1)" "a=a*a" 2>temp.txt >> >> > >> it will eventually crash. In looking at the debug output, I see a >> > >> request to allocate 18446744073709498400 bytes of memory. >> >> > >> I tested with both MPIR 1.2.2 and 1.3.0 and get similar behavior. I'm >> > >> using a custom version of GMPY with some fixes for size_t vs. long >> > >> issues that hasn't been committed but I will try to commit those >> > >> changes later today. >> >> > > Hi Case >> >> > > That is _very_ useful information and may explain why Windows >> > > performance lgas that on Linux. >> >> > > A long time ago - in GMP days - I had to turn of the use of alloca as >> > > I kept getting crashes if I used it. >> >> > > I have not tried switching it on in MPIR but I will certainly look at >> > > this again. >> >> > > But _alloca is now deprecated on Windows and its replacement requires >> > > a 'free' procedure that alloca doesn't need. It may hence be quite >> > > difficult to take advantage of this in future. It all depends on how >> > > GMP/MPIR use alloca. >> >> > > Thanks for the debugging! >> >> > > Brian >> >> > Some additional information. >> >> > MPIR 1.2.2 generates approximately 172,000 memory allocator calls. >> > MPIR 1.3.0 generates approximately 221,000 memory allocator calls. >> >> > The count includes both mp_allocate and mp_free. >> >> > Performance numbers: >> > Windows x64, MPIR 1.2.2: 3.35 seconds >> > Windows x64, MPIR 1.3.0, 3.76 seconds >> > Linux, MPIR 1.3.0, 3.05 seconds >> >> > Processor is a Core2. >> >> > Regarding the mp_allocate failure: it looks like it occurs when trying >> > to allocate space for a number larger that 2^32 bits long. >> >> Does this allocation failure only occur on Windows? >> >> The MPIR settings for memory management on Windows are: >> >> #define HAVE_ALLOCA 1 >> #undef HAVE_ALLOCA_H >> #undef WANT_TMP_ALLOCA >> #undef WANT_TMP_DEBUG >> #undef WANT_TMP_NOTREENTRANT >> #define WANT_TMP_REENTRANT 1 >> >> What are the normal settings used on a Linux/GCC build? > > The memory allocation calls in MPIR are of trhe form > > void *memory(pointer, size_t) > > Since size_t is a 32-bit int on Windows, this will fail if an attempt > is made to obtain 2^32 or more _bytes_. On 64-bit Windows, size_t is __int64. "long" is only 32-bits on x64 Windows but "long" is 64-bits on 64-bit Linux. > > But it should be ok for 2^32 bits. It should be, but I tried several tests and they all failed when trying to allocate for more than 2*32 bits. IIRC, an mpz on a 64-bit Linux system has a maximum length of 2^37 bits. Could the logic that enforces that limit fail because it is using 32-bit longs? Perhaps a shift dropping bits off the right?
Case > > Brian > > -- > > You received this message because you are subscribed to the Google Groups > "mpir-devel" group. > To post to this group, send email to mpir-de...@googlegroups.com. > To unsubscribe from this group, send email to > mpir-devel+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/mpir-devel?hl=en. > > > -- You received this message because you are subscribed to the Google Groups "mpir-devel" group. To post to this group, send email to mpir-de...@googlegroups.com. To unsubscribe from this group, send email to mpir-devel+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/mpir-devel?hl=en.
