> > Hey Dan. Now, that you mention you are using OS X, I'm fairly > confident that the problem is that you are using a 32-bit version of > Python (i.e. you are not running in full 64-bit mode and so the 4GB > limit applies). > > The most common Python on OS X is 32-bit python. I think a few people > in the SAGE project have successfully built Python in 64-bit mode on OSX > (but I don't think they have released anything yet). You would have to > use a 64-bit version of Python to compile NumPy if you want to access > large memory. > > -Travis > > Travis, thanks for the message. I think you're probably right -- I didn't build python myself but instead downloaded the universal OSX binary from the python download page -- and that surely wasn't built for 64-bit system. So I guess I'll have to figure out to do the 64-bit build.
Which leaves a question that I think Chuck brought up in a way: > In [1]: s = numpy.random.binomial(1,.5,(20000,100)) > In [2]: inner(s,s) > Out[2]: > array([[45, 22, 17, ..., 20, 26, 23], > [22, 52, 26, ..., 23, 33, 24], > [17, 26, 52, ..., 27, 27, 19], > ..., > [20, 23, 27, ..., 46, 26, 22], > [26, 33, 27, ..., 26, 54, 25], > [23, 24, 19, ..., 22, 25, 44]]) >This on 32 bit fedora 8 with 2GiB of actual memory. It was slow and a couple of hundred megs of something went into swap, but it >did complete. So this looks to me like an OS X problem. Are there any limitations on the user memory sizes? There might be some >system setting accounting for this. Chuck, is this another way of asking: why is my OS X system not paging memory the way you'd expect a system to respond to the malloc command? Is python somehow overloaded the malloc command so that when the OS says a swap would have to occur, somehow instead of just the swap, that an error message involving mmap is somehow triggered? (Sorry if this makes no sense.) I should add that the tried the same code on a 32-bit windows machine and got the same error as on OS X. Maybe the Linux python builds manage this stuff better.
_______________________________________________ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
