Yep. That is the problem. Figured it out from both sides of the equation. As of Python 2.7.1 and 3.2, the BZ2File object type does not support reading multi-stream bz2 files, like those generated by pbzip2.
I'm going to have to look into another solution. -Eric -=--=---=----=----=---=--=-=--=---=----=---=--=-=- Eric B. Wolf 720-334-7734 On Tue, Mar 1, 2011 at 3:06 PM, Christian Vetter <[email protected]>wrote: > On Tue, Mar 1, 2011 at 10:54 PM, Eric Wolf <[email protected]> wrote: > > Has anyone tried reading the bz2 compressed planet file? > > I'm currently trying to figure out if there is something wrong in > Python's > > bz2 module, zlib, or the planet files themselves. > > The python BZ2file module lets you read and write bz2 files like normal > > files without decompressing the whole thing. This is key, especially when > > dealing with full-planet which is 500GB+ uncompressed. However, I've > found a > > few instances where different people have had issues with the OSM planet > > files and the bz2 module quitting after reading the first 900,000 bytes. > Has > > anyone had success with this? > > FYI: The script works perfectly with a 165MB compressed file from > Cloudmade. > > -Eric > > -=--=---=----=----=---=--=-=--=---=----=---=--=-=- > > Eric B. Wolf 720-334-7734 > > > > > > > > > > _______________________________________________ > > dev mailing list > > [email protected] > > http://lists.openstreetmap.org/listinfo/dev > > > > > > Most likely the module does not handle multi-stream bz2 files on > itself. These files are created by parallel packing programs and if > the module does not handle them correctly it will stop after the end > of the first stream. I would take a look into the documentation > whether it says anything about unpacking multi-stream archives and how > you have to initialize the next stream. > > Greetings, > > Christian Vetter >
_______________________________________________ dev mailing list [email protected] http://lists.openstreetmap.org/listinfo/dev
