On Jan 4, 2:07 pm, Case Vanhorsen <cas...@gmail.com> wrote: > On Mon, Jan 4, 2010 at 12:35 AM, Cactus <rieman...@googlemail.com> wrote: > > > On Jan 4, 3:26 am, Case Vanhorsen <cas...@gmail.com> wrote: > >> On Sun, Jan 3, 2010 at 12:33 PM, Cactus <rieman...@googlemail.com> wrote: > > >> > On Jan 3, 8:15 pm, Case Vanhorsen <cas...@gmail.com> wrote: > >> >> Hello, > > >> >> I discovered an interesting memory allocation behavior on Windows vs. > >> >> Linux. I was testing GMPY on 64-bit Windows when I stumbled into this. > >> >> GMPY replaces the native MPIR memory allocation routines with Python's > >> >> memory allocator. If I enable debugging in GMPY, I get a trace of all > >> >> the memory allocation calls. When I ran the following: > > >> >> python -mtimeit -n 1 -r 1 -s "import > >> >> gmpy;gmpy.mpz(3)**(2**27);gmpy.set_debug(1)" "a=a*a" 2>temp.txt > > >> >> and look at the output saved in temp.txt, I see that Linux generated > >> >> approximately 34 memory manager calls but Windows generates over > >> >> 100,000 calls. Most of the Windows allocations are for small (<8K) > >> >> chunks of memory while all the Linux requests are for more than 64K. > >> >> The performance between Linux and Windows is similar. Could it be that > >> >> Windows is not using alloca? > > >> >> I also think I found another memory allocation bug. If I run the above > >> >> multiplication repeatedly: > > >> >> python -mtimeit -s "import > >> >> gmpy;gmpy.mpz(3)**(2**27);gmpy.set_debug(1)" "a=a*a" 2>temp.txt > > >> >> it will eventually crash. In looking at the debug output, I see a > >> >> request to allocate 18446744073709498400 bytes of memory. > > >> >> I tested with both MPIR 1.2.2 and 1.3.0 and get similar behavior. I'm > >> >> using a custom version of GMPY with some fixes for size_t vs. long > >> >> issues that hasn't been committed but I will try to commit those > >> >> changes later today. > > >> > Hi Case > > >> > That is _very_ useful information and may explain why Windows > >> > performance lgas that on Linux. > > >> > A long time ago - in GMP days - I had to turn of the use of alloca as > >> > I kept getting crashes if I used it. > > >> > I have not tried switching it on in MPIR but I will certainly look at > >> > this again. > > >> > But _alloca is now deprecated on Windows and its replacement requires > >> > a 'free' procedure that alloca doesn't need. It may hence be quite > >> > difficult to take advantage of this in future. It all depends on how > >> > GMP/MPIR use alloca. > > >> > Thanks for the debugging! > > >> > Brian > > >> Some additional information. > > >> MPIR 1.2.2 generates approximately 172,000 memory allocator calls. > >> MPIR 1.3.0 generates approximately 221,000 memory allocator calls. > > >> The count includes both mp_allocate and mp_free. > > >> Performance numbers: > >> Windows x64, MPIR 1.2.2: 3.35 seconds > >> Windows x64, MPIR 1.3.0, 3.76 seconds > >> Linux, MPIR 1.3.0, 3.05 seconds > > >> Processor is a Core2. > > >> Regarding the mp_allocate failure: it looks like it occurs when trying > >> to allocate space for a number larger that 2^32 bits long. > > > Does this allocation failure only occur on Windows? > > Yes. If fails with the stock allocator, too. > > > > > The MPIR settings for memory management on Windows are: > > > #define HAVE_ALLOCA 1 > > #undef HAVE_ALLOCA_H > > #undef WANT_TMP_ALLOCA > > #undef WANT_TMP_DEBUG > > #undef WANT_TMP_NOTREENTRANT > > #define WANT_TMP_REENTRANT 1 > > #define HAVE_ALLOCA 1 > #define HAVE_ALLOCA_H 1 > #define WANT_TMP_ALLOCA 1 > /* #undef WANT_TMP_REENTRANT */ > /* #undef WANT_TMP_NOTREENTRANT */ > /* #undef WANT_TMP_DEBUG */
Thanks Case, I can see why it would fail at 2^32 bytes but I can't see why it would fail at 2^32 bits Overcoming the 2^32 bytes failure will be a major exercise since the assummption that size_t is 64-bits long in a 64-bit OS is endemic in MPIR On Windows uintptr_t is 32-bits on win32 and 64-bits on x64 - does this exist on Linux/GCC? Brian -- You received this message because you are subscribed to the Google Groups "mpir-devel" group. To post to this group, send email to mpir-de...@googlegroups.com. To unsubscribe from this group, send email to mpir-devel+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/mpir-devel?hl=en.
