Hi, Eric Jonas wrote: > I assume most of us are using Cython because normal python is too slow > for our particular operation. I've been writing a compiler in python > which generates large graphs with ~600k nodes, and I'd like to > serialize/checkpoint these to disk for later stages of the compiler > pipeline to use. However, at the moment, it takes ~60 seconds to > serialize the resulting graph to disk with cPickle.
You can implement your own pickle mechanism for a specific object. We do that in lxml.objectify, where you can pickle an XML element into a serialised XML byte stream. If you can find a way to efficiently walk your graph at the C level and serialise that by hand step by step, you may end up being much faster than pickle's introspection machinery. Stefan _______________________________________________ Cython-dev mailing list [email protected] http://codespeak.net/mailman/listinfo/cython-dev
