Re: What is the difference between 32 and 64 bit Python on Windows 7 64 bit?
On 11/05/14 08:56, Ross Gayler wrote: It looks to me as though 32 and 64 bit versions of Python on 64 bit Windows are both really 32 bit Python, differing only in how they interact with Windows. No! Pointers are 64 bit, Python integers (on Python 2.x) are 32 bit. Microsoft decided to use a 32 bit long in MSVC for backwards compatiblity, but also because the AMD64 (x86-64) architecture was designed to use a 64 address with a 32 bit offset. (A 64 bit long was originally slightly less efficient.) You can see the value of 64 bit Python e.g. if you allocate a lot of objects or if you try to mmap a huge file. With 32 bit Python you are limited to only 2 GB of virtual memory. In 64 bit Python you can in practice mmap as much as you want. The element size of what you try to index also matters. While a C long and a Python int is 32 bit on Windows, 64-bit Python will use a 64-bit offset internally (Py_ssize_t and Py_intptr_t) even on Windows. The 32 bit Python int just limits how many objects you can index from Python space before Python roll over to using long instead of int. It does not limit the amount of memory a Python int can index. In is only when you index an array of bytes you will see the roll-over from Python int to Python long at the 2 GB limit. Typically, object will be much larger than one byte. Here are two examples: - A one-dimensional NumPy array with dtype np.float64 can keep 16 GB of data before a 32 bit index is too small and Python starts to use long. A two-dimensional NumPy array with dtype np.float64 can keep 256 GB of data before a 32 bit index is too small. - A Python list stores internally an array of pointers, each of which is 64 bit. So just indexing those goes up to 16 GB of pointer data before the int rolls over. Then each of these pointers point to a Python object. A Python float on my computer (not Windows) is 24 bytes, which I got from sys.getsizeof(1.) So 2**32 of those are another 383 GB. So if I indexed a list of Python floats on this computer, Python could handle an almost 400 GB data structure with a 32 bit int as indexer without rolling over to long. This is obviously way beyond anything the 2 GB limit on 32 bit Python allows. Sturla -- https://mail.python.org/mailman/listinfo/python-list
Re: What is the difference between 32 and 64 bit Python on Windows 7 64 bit?
On 12/05/14 15:42, Sturla Molden wrote: - A one-dimensional NumPy array with dtype np.float64 can keep 16 GB of data before a 32 bit index is too small and Python starts to use long. A two-dimensional NumPy array with dtype np.float64 can keep 256 GB of data before a 32 bit index is too small. Oops, the latter should be 34359738336 GB (that is, 32767 pentabytes) :) Sturla -- https://mail.python.org/mailman/listinfo/python-list
Re: What is the difference between 32 and 64 bit Python on Windows 7 64 bit?
On 11/05/14 08:56, Ross Gayler wrote: Is that true?I have spent a couple of hours searching for a definitive description of the difference between the 32 and 64 bit versions of Python for Windows and haven't found anything. Why do you care if a Python int object uses 32 or 64 bits internally? Python 2.x will automatically switch to long when needed. The size of the Python integer is an internal implementation detail you will not notice. Python knows when to use a long instead of an int. Python 3.x does not even have a fixed-size integer. 64 bit Python is 64 bit Python, even on Windows. The difference between 32 bit and 64 bit Python is what you would expect: The size of a C pointer is 64 bits, and the virtual address space is much larger (in general not 2**63-1 bytes, but some OS dependent value). Sturla -- https://mail.python.org/mailman/listinfo/python-list
Re: What is the difference between 32 and 64 bit Python on Windows 7 64 bit?
On 2014-05-13 00:41, Sturla Molden wrote: On 12/05/14 15:42, Sturla Molden wrote: - A one-dimensional NumPy array with dtype np.float64 can keep 16 GB of data before a 32 bit index is too small and Python starts to use long. A two-dimensional NumPy array with dtype np.float64 can keep 256 GB of data before a 32 bit index is too small. Oops, the latter should be 34359738336 GB (that is, 32767 pentabytes) :) Double oops, Sturla, it's petabyte (well, pebibyte actually). :-) -- https://mail.python.org/mailman/listinfo/python-list
Re: What is the difference between 32 and 64 bit Python on Windows 7 64 bit?
On Tue, May 13, 2014 at 10:05 AM, Sturla Molden sturla.mol...@gmail.com wrote: On 11/05/14 08:56, Ross Gayler wrote: Is that true?I have spent a couple of hours searching for a definitive description of the difference between the 32 and 64 bit versions of Python for Windows and haven't found anything. Why do you care if a Python int object uses 32 or 64 bits internally? Python 2.x will automatically switch to long when needed. The size of the Python integer is an internal implementation detail you will not notice. Python knows when to use a long instead of an int. Python 3.x does not even have a fixed-size integer. Sometimes you just want to confirm. :) Or maybe you want your program to be able to detect which it's on. There are ways of doing both, but sys.maxint isn't one of them, as it's specific to the int-long promotion of Py2. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: What is the difference between 32 and 64 bit Python on Windows 7 64 bit?
On 13/05/14 02:09, Chris Angelico wrote: Sometimes you just want to confirm. :) Or maybe you want your program to be able to detect which it's on. There are ways of doing both, but sys.maxint isn't one of them, as it's specific to the int-long promotion of Py2. The OPs main mistake, I guess, was to assume that sys.maxint is the biggest integer value Python 2.x can use. Sturla -- https://mail.python.org/mailman/listinfo/python-list
Re: What is the difference between 32 and 64 bit Python on Windows 7 64 bit?
On 5/11/2014 2:56 AM, Ross Gayler wrote: Hi, I want to install Python on a PC with 16GB of RAM and the 64 bit version of Windows 7. I want Python to be able to use as much as possible of the RAM. When I install the 64 bit version of Python I find that sys.maxint == 2**31 - 1 Since sys.maxint is gone in 3.x, you must be using some version of 2.x. Do yourself a favor and install 3.4 unless you absolutely need 2.x. With 3.4: a = [None]*10 sys.getsizeof(a) 800064 That is 10 8-byte pointers, as I expected. -- Terry Jan Reedy -- https://mail.python.org/mailman/listinfo/python-list
Re: What is the difference between 32 and 64 bit Python on Windows 7 64 bit?
On Sat, May 10, 2014 at 11:56 PM, Ross Gayler r.gay...@gmail.com wrote: Hi, I want to install Python on a PC with 16GB of RAM and the 64 bit version of Windows 7. I want Python to be able to use as much as possible of the RAM. When I install the 64 bit version of Python I find that sys.maxint == 2**31 - 1 Whereas the Pythpon installed on my 64 bit linux system returns sys.maxint == 2**63 - 1. That comes from the underlying C implementation. 64-bit MSVC still has long int as 32-bit. You need to specify long long int to get a 64-bit number even on a 64-bit compiler. Microsoft is a little nuts on the backwards compatiblity. It looks to me as though 32 and 64 bit versions of Python on 64 bit Windows are both really 32 bit Python, differing only in how they interact with Windows. So I wouldn't expect 64 bit Python running on 64 bit Windows to allow the large data struictures I could have with 64 bit Python running on 64 bit linux. Is that true?I have spent a couple of hours searching for a definitive description of the difference between the 32 and 64 bit versions of Python for Windows and haven't found anything. long int (the size of an integer) != size_t (the size of an object). 64-bit Python still uses 64-bit pointers so it can still address more than 4GB of memory. It just rolls over into longs after 32-bit int max instead of after 64-bit int max. -- https://mail.python.org/mailman/listinfo/python-list
