Peter Otten <[EMAIL PROTECTED]> wrote: > Chi Yin Cheung wrote: > > > Is there a way in python to output binary files? I need to python to > > write out a stream of 5 million floating point numbers, separated by > > some separator, but it seems that all python supports natively is string > > information output, which is extremely space inefficient. > > > > I'd tried using the pickle module, but it crashed whenever I tried using > > it due to the large amount of data involved. > > A minimalistic alternative is array.tofile()/fromfile(), but pickle should > handle a list, say, of 5 million floating point numbers just fine. What > exactly are you doing to provoke a crash, and what does it look like? > Please give minimal code and the traceback.
cPickle worked fine when I tried it... >>> L=map(float, range(5000000)) >>> import cPickle >>> out=file("z", "wb") >>> cPickle.dump(L, out, -1) >>> out.close() >>> inp=file("z", "rb") >>> K=cPickle.load(inp) >>> inp.close() >>> import os >>> os.system("ls -l z") -rw-r--r-- 1 ncw ncw 45010006 Apr 19 18:43 z 0 >>> Indicating each float took 9 bytes to store, which is 1 byte more than a 64 bit float would normally take. The pickle dump / load each took about 2 seconds. -- Nick Craig-Wood <[EMAIL PROTECTED]> -- http://www.craig-wood.com/nick -- http://mail.python.org/mailman/listinfo/python-list