Re: writing large dictionaries to file using cPickle

2009-04-26 Thread repi
ble.com/writing-large-dictionaries-to-file-using-cPickle-tp21708938p23246693.html Sent from the Python - python-list mailing list archive at Nabble.com. -- http://mail.python.org/mailman/listinfo/python-list

Re: Using cPickle

2009-02-25 Thread mmcclaf
Thanks! All fixed! -- http://mail.python.org/mailman/listinfo/python-list

Re: Using cPickle

2009-02-06 Thread MRAB
mmcclaf wrote: On Feb 6, 3:09 pm, MRAB wrote: mmcclaf wrote: On Feb 6, 10:25 am, Steve Holden wrote: mmcclaf wrote: Hi there, I have to make a small database using cPickle. I'm having troubles trying to read in the information if it's more than one line. I'm pretty sure i

Re: Using cPickle

2009-02-06 Thread mmcclaf
On Feb 6, 3:09 pm, MRAB wrote: > mmcclaf wrote: > > On Feb 6, 10:25 am, Steve Holden wrote: > >> mmcclaf wrote: > >>> Hi there, > >>> I have to make a small database using cPickle. I'm having troubles > >>> trying to read in the informa

Re: Using cPickle

2009-02-06 Thread MRAB
mmcclaf wrote: On Feb 6, 10:25 am, Steve Holden wrote: mmcclaf wrote: Hi there, I have to make a small database using cPickle. I'm having troubles trying to read in the information if it's more than one line. I'm pretty sure it's in the line "for line in stuff

Re: Using cPickle

2009-02-06 Thread mmcclaf
On Feb 6, 10:25 am, Steve Holden wrote: > mmcclaf wrote: > > Hi there, > > > I have to make a small database using cPickle. I'm having troubles > > trying to read in the information if it's more than one line. I'm > > pretty sure it's in the li

Re: Using cPickle

2009-02-06 Thread Steve Holden
mmcclaf wrote: > Hi there, > > I have to make a small database using cPickle. I'm having troubles > trying to read in the information if it's more than one line. I'm > pretty sure it's in the line "for line in stuff:" Can anyone help me > out

Using cPickle

2009-02-06 Thread mmcclaf
Hi there, I have to make a small database using cPickle. I'm having troubles trying to read in the information if it's more than one line. I'm pretty sure it's in the line "for line in stuff:" Can anyone help me out? Basically the end result is wanting it to look som

Re: writing large dictionaries to file using cPickle

2009-01-30 Thread Aaron Brady
On Jan 30, 2:44 pm, perfr...@gmail.com wrote: > On Jan 28, 6:08 pm, Aaron Brady wrote: > > > > > On Jan 28, 4:43 pm, perfr...@gmail.com wrote: > > > > On Jan 28, 5:14 pm, John Machin wrote: > > > > > On Jan 29, 3:13 am, perfr...@gmail.com wrote: > > > > > > hello all, > > > > > > i have a large d

Re: writing large dictionaries to file using cPickle

2009-01-30 Thread perfreem
On Jan 28, 6:08 pm, Aaron Brady wrote: > On Jan 28, 4:43 pm, perfr...@gmail.com wrote: > > > On Jan 28, 5:14 pm, John Machin wrote: > > > > On Jan 29, 3:13 am, perfr...@gmail.com wrote: > > > > > hello all, > > > > > i have a large dictionary which contains about 10 keys, each key has a > > > > v

Re: writing large dictionaries to file using cPickle

2009-01-28 Thread Gabriel Genellina
En Wed, 28 Jan 2009 14:13:10 -0200, escribió: i have a large dictionary which contains about 10 keys, each key has a value which is a list containing about 1 to 5 million (small) dictionaries. for example, mydict = {key1: [{'a': 1, 'b': 2, 'c': 'hello'}, {'d', 3, 'e': 4, 'f': 'world'}, ...],

Re: writing large dictionaries to file using cPickle

2009-01-28 Thread John Machin
On Jan 29, 9:43 am, perfr...@gmail.com wrote: > On Jan 28, 5:14 pm, John Machin wrote: > > > > > On Jan 29, 3:13 am, perfr...@gmail.com wrote: > > > > hello all, > > > > i have a large dictionary which contains about 10 keys, each key has a > > > value which is a list containing about 1 to 5 milli

Re: writing large dictionaries to file using cPickle

2009-01-28 Thread Aaron Brady
On Jan 28, 4:43 pm, perfr...@gmail.com wrote: > On Jan 28, 5:14 pm, John Machin wrote: > > > > > On Jan 29, 3:13 am, perfr...@gmail.com wrote: > > > > hello all, > > > > i have a large dictionary which contains about 10 keys, each key has a > > > value which is a list containing about 1 to 5 milli

Re: writing large dictionaries to file using cPickle

2009-01-28 Thread perfreem
On Jan 28, 5:14 pm, John Machin wrote: > On Jan 29, 3:13 am, perfr...@gmail.com wrote: > > > > > hello all, > > > i have a large dictionary which contains about 10 keys, each key has a > > value which is a list containing about 1 to 5 million (small) > > dictionaries. for example, > > > mydict = {

Re: writing large dictionaries to file using cPickle

2009-01-28 Thread John Machin
On Jan 29, 3:13 am, perfr...@gmail.com wrote: > hello all, > > i have a large dictionary which contains about 10 keys, each key has a > value which is a list containing about 1 to 5 million (small) > dictionaries. for example, > > mydict = {key1: [{'a': 1, 'b': 2, 'c': 'hello'}, {'d', 3, 'e': 4, 'f

Re: writing large dictionaries to file using cPickle

2009-01-28 Thread Aaron Brady
On Jan 28, 10:13 am, perfr...@gmail.com wrote: > hello all, > > i have a large dictionary which contains about 10 keys, each key has a > value which is a list containing about 1 to 5 million (small) > dictionaries. for example, snip > but this takes just as long... any ideas ? is there a different

Re: writing large dictionaries to file using cPickle

2009-01-28 Thread Christian Heimes
perfr...@gmail.com schrieb: > but this takes just as long... any ideas ? is there a different module > i could use that's more suitable for large dictionaries ? > thank you very much. Have a look at ZODB. -- http://mail.python.org/mailman/listinfo/python-list

Re: writing large dictionaries to file using cPickle

2009-01-28 Thread perfreem
On Jan 28, 11:32 am, pyt...@bdurham.com wrote: > Hi, > > Change: > > pickle.dump(mydict, pfile) > > to: > > pickle.dump(mydict, pfile, -1 ) > > I think you will see a big difference in performance and also a much > smaller file on disk. > > BTW: What type of application are you developing that crea

Re: writing large dictionaries to file using cPickle

2009-01-28 Thread python
Hi, Change: pickle.dump(mydict, pfile) to: pickle.dump(mydict, pfile, -1 ) I think you will see a big difference in performance and also a much smaller file on disk. BTW: What type of application are you developing that creates so many dictionaries? Sounds interesting. Malcolm -- http://mail

writing large dictionaries to file using cPickle

2009-01-28 Thread perfreem
hello all, i have a large dictionary which contains about 10 keys, each key has a value which is a list containing about 1 to 5 million (small) dictionaries. for example, mydict = {key1: [{'a': 1, 'b': 2, 'c': 'hello'}, {'d', 3, 'e': 4, 'f': 'world'}, ...], key2: [...]} in total

Re: Strange KeyError using cPickle

2005-06-01 Thread Tim Peters
[Tim Peters] >> What is "XWwz"? Assuming it's a bizarre typo for "open", change the >> 'w' there to 'wb'. Pickles are binary data, and files holding pickles >> must be opened in binary mode, especially since: >> >>> ... >>> (on WinXP, CPython 2.4.1) [Rune Strand] > Thanks Tim. The bizarre 'typo

Re: Strange KeyError using cPickle

2005-06-01 Thread Rune Strand
[Tim Peters] > What is "XWwz"? Assuming it's a bizarre typo for "open", change the > 'w' there to 'wb'. Pickles are binary data, and files holding pickles > must be opened in binary mode, especially since: > > > ... > > (on WinXP, CPython 2.4.1) Thanks Tim. The bizarre 'typo' appears to be cause

Re: Strange KeyError using cPickle

2005-06-01 Thread Tim Peters
[Rune Strand] > I'm experiencing strange errors both with pickle and cPickle in the > below code: > > > import cPickle as pickle > #import pickle > from string import ascii_uppercase > from string import ascii_lowercase > > def createData(): >d1 = list("Something's rotten") >d2 = tuple('in

Strange KeyError using cPickle

2005-06-01 Thread Rune Strand
I'm experiencing strange errors both with pickle and cPickle in the below code: import cPickle as pickle #import pickle from string import ascii_uppercase from string import ascii_lowercase def createData(): d1 = list("Something's rotten") d2 = tuple('in the state of Denmark') d3 =