Re: compressed serialization module

2008-11-19 Thread Tino Wildenhain
Hi, greg wrote: Mark wrote: Thanks guys. This is for serializing to disk. I was hoping to not have to use too many intermediate steps You should be able to use a gzip.GzipFile or bz2.BZ2File and pickle straight into it. also: import codecs out=codecs.open("picklefile.bz2",mode="wb",enc

Re: compressed serialization module

2008-11-19 Thread Nick Craig-Wood
greg <[EMAIL PROTECTED]> wrote: > Nick Craig-Wood wrote: > > (Note that basic pickle protocol is likely to be more compressible > > than the binary version!) > > Although the binary version may be more compact to > start with. It would be interesting to compare the > two and see which one wins

Re: compressed serialization module

2008-11-18 Thread greg
Nick Craig-Wood wrote: (Note that basic pickle protocol is likely to be more compressible than the binary version!) Although the binary version may be more compact to start with. It would be interesting to compare the two and see which one wins. -- Greg -- http://mail.python.org/mailman/listi

Re: compressed serialization module

2008-11-18 Thread Nick Craig-Wood
greg <[EMAIL PROTECTED]> wrote: > Mark wrote: > > Thanks guys. This is for serializing to disk. I was hoping to not > > have to use too many intermediate steps > > You should be able to use a gzip.GzipFile > or bz2.BZ2File and pickle straight into it. Good idea - that will be much more memor

Re: compressed serialization module

2008-11-17 Thread greg
Mark wrote: Thanks guys. This is for serializing to disk. I was hoping to not have to use too many intermediate steps You should be able to use a gzip.GzipFile or bz2.BZ2File and pickle straight into it. -- Greg -- http://mail.python.org/mailman/listinfo/python-list

Re: compressed serialization module

2008-11-17 Thread Mark
On Nov 17, 3:08 pm, [EMAIL PROTECTED] wrote: >     Mark> def saveOjb(self, dataObj): >     Mark>     fName = self.version + '_' + self.modname + '.dat' >     Mark>     f = open(fName, 'w') >     Mark>     dStr = pickle.dumps(dataObj) >     Mark>     c = dStr.encode("bz2") >     Mark>     pickle.dum

Re: compressed serialization module

2008-11-17 Thread skip
Mark> def saveOjb(self, dataObj): Mark> fName = self.version + '_' + self.modname + '.dat' Mark> f = open(fName, 'w') Mark> dStr = pickle.dumps(dataObj) Mark> c = dStr.encode("bz2") Mark> pickle.dump(c, f, pickle.HIGHEST_PROTOCOL) Mark> f.close()

Re: compressed serialization module

2008-11-17 Thread Mark
Thanks guys. This is for serializing to disk. I was hoping to not have to use too many intermediate steps, but I couldn't figure out how to pickle data into zipfile without using either intermediate string or file. That's cool here's what I'll probably settle on (tested) - now just need to reve

Re: compressed serialization module

2008-11-17 Thread Nick Craig-Wood
[EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: > > >> I used pickle and found the file was saved in text format. I wonder > >> whether anyone is familiar with a good compact off-the-shelf module > >> available that will save in compressed format... or maybe an opinion > >> on a

Re: compressed serialization module

2008-11-17 Thread skip
>> I used pickle and found the file was saved in text format. I wonder >> whether anyone is familiar with a good compact off-the-shelf module >> available that will save in compressed format... or maybe an opinion >> on a smart approach for making a custom one? Joe> Well, her

Re: compressed serialization module

2008-11-17 Thread Joe Strout
On Nov 17, 2008, at 10:47 AM, Mark wrote: I used pickle and found the file was saved in text format. I wonder whether anyone is familiar with a good compact off-the-shelf module available that will save in compressed format... or maybe an opinion on a smart approach for making a custom one? W

compressed serialization module

2008-11-17 Thread Mark
I used pickle and found the file was saved in text format. I wonder whether anyone is familiar with a good compact off-the-shelf module available that will save in compressed format... or maybe an opinion on a smart approach for making a custom one? Appreciate it! I'm a bit of a n00b but have be