Hi, I noticed that there is a PEP (3154) and a GSoC proposal about improving Pickle. Given the recent discussion on this list about using Cython for the import module, I wonder if it wouldn't make even more sense to switch from a C (accelerator) implementation to Cython for _pickle.
The rationale is that C code that deals a lot with object operations tends to be rather verbose, and _pickle specifically looks very verbose in many places. Some of this is optimised I/O, ok, but most of it seems to take its complexity from code specialisations for builtin types and a lot of error handling code. A Cython reimplementation would take a lot of weight out of this. Note that the approach won't be as simple as compiling pickle.py. _pickle uses a lot of optimisations that only work at the C level, at least efficiently. So the idea would be to rewrite _pickle in Cython instead. It's currently about 6500 lines of C. Even if we divide that only by a rather conservative factor of 3, we'd end up with some 2000 lines of Cython code, all extracted straight from the existing C code. That sounds like less than two weeks of work, maybe even if we add the marshal module to it. In less than a month of GSoC time, this could easily reach a point where it's "close to the speed of what we have" and "fast enough", but a lot more accessible and maintainable, thus also making it easier to add the extensions described in the PEP. What do you think? Stefan _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
