Re: Load a list subset with pickle?
En Thu, 15 Oct 2009 16:28:11 -0300, Peng Yu pengyu...@gmail.com escribió: On Thu, Oct 15, 2009 at 12:01 PM, Gabriel Genellina gagsl-...@yahoo.com.ar wrote: En Thu, 15 Oct 2009 13:05:18 -0300, Peng Yu pengyu...@gmail.com escribió: How do I determine if I have loaded all the elements? I use the following code. I'm wondering if there is any better solution than this. try: while 1: e = pickle.load(input) print e except EOFError: pass Pickle the list length before its contents. Suppose that the list length was not in 'serialize_list.output/serialize_list.pkl'. Is try-except block the best solution? I don't know if it's the best, but it's the one I'd probably use. -- Gabriel Genellina -- http://mail.python.org/mailman/listinfo/python-list
Re: Load a list subset with pickle?
On Tue, Oct 13, 2009 at 1:23 PM, Robert Kern robert.k...@gmail.com wrote: On 2009-10-13 13:00 PM, Peng Yu wrote: I use pickle to dump a long list. But when I load it, I only want to load the first a few elements in the list. I am wondering if there is a easy way to do so? Thank you! Not by pickling the list. However, you can concatenate pickles, so you could just pickle each item from the list in order to the same file and only unpickle the first few. In [1]: import cPickle In [2]: from cStringIO import StringIO In [3]: very_long_list = range(10) In [4]: f = StringIO() In [5]: for item in very_long_list: ...: cPickle.dump(item, f) ...: ...: In [6]: f.seek(0,0) In [7]: cPickle.load(f) Out[7]: 0 In [8]: cPickle.load(f) Out[8]: 1 In [9]: cPickle.load(f) Out[9]: 2 In [10]: cPickle.load(f) Out[10]: 3 How do I determine if I have loaded all the elements? I use the following code. I'm wondering if there is any better solution than this. ### import pickle alist = [1, 2.0, 3, 4+6j] output=open('serialize_list.output/serialize_list.pkl', 'wb') for e in alist: pickle.dump(e, output) output.close() input=open('serialize_list.output/serialize_list.pkl', 'rb') try: while 1: e = pickle.load(input) print e except EOFError: pass -- http://mail.python.org/mailman/listinfo/python-list
Re: Load a list subset with pickle?
On 2009-10-15 11:05 AM, Peng Yu wrote: On Tue, Oct 13, 2009 at 1:23 PM, Robert Kernrobert.k...@gmail.com wrote: On 2009-10-13 13:00 PM, Peng Yu wrote: I use pickle to dump a long list. But when I load it, I only want to load the first a few elements in the list. I am wondering if there is a easy way to do so? Thank you! Not by pickling the list. However, you can concatenate pickles, so you could just pickle each item from the list in order to the same file and only unpickle the first few. In [1]: import cPickle In [2]: from cStringIO import StringIO In [3]: very_long_list = range(10) In [4]: f = StringIO() In [5]: for item in very_long_list: ...: cPickle.dump(item, f) ...: ...: In [6]: f.seek(0,0) In [7]: cPickle.load(f) Out[7]: 0 In [8]: cPickle.load(f) Out[8]: 1 In [9]: cPickle.load(f) Out[9]: 2 In [10]: cPickle.load(f) Out[10]: 3 How do I determine if I have loaded all the elements? I use the following code. I'm wondering if there is any better solution than this. ### import pickle alist = [1, 2.0, 3, 4+6j] output=open('serialize_list.output/serialize_list.pkl', 'wb') for e in alist: pickle.dump(e, output) output.close() input=open('serialize_list.output/serialize_list.pkl', 'rb') try: while 1: e = pickle.load(input) print e except EOFError: pass You could write out an integer with the number of expected elements at the very beginning. In [1]: import cPickle In [2]: alist = [1, 2.0, 3, 4+6j] In [3]: output = open('foo.pkl', 'wb') In [4]: cPickle.dump(len(alist), output) In [5]: for item in alist: ...: cPickle.dump(item, output) ...: ...: In [6]: output.close() In [7]: input = open('foo.pkl', 'rb') In [8]: n = cPickle.load(input) In [9]: n Out[9]: 4 In [10]: for i in range(n): : print cPickle.load(input) : : 1 2.0 3 (4+6j) In [11]: assert input.read(1) == '' -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: Load a list subset with pickle?
En Thu, 15 Oct 2009 13:05:18 -0300, Peng Yu pengyu...@gmail.com escribió: How do I determine if I have loaded all the elements? I use the following code. I'm wondering if there is any better solution than this. ### import pickle alist = [1, 2.0, 3, 4+6j] output=open('serialize_list.output/serialize_list.pkl', 'wb') for e in alist: pickle.dump(e, output) output.close() input=open('serialize_list.output/serialize_list.pkl', 'rb') try: while 1: e = pickle.load(input) print e except EOFError: pass Pickle the list length before its contents. -- Gabriel Genellina -- http://mail.python.org/mailman/listinfo/python-list
Re: Load a list subset with pickle?
En Thu, 15 Oct 2009 13:05:18 -0300, Peng Yu pengyu...@gmail.com escribió: How do I determine if I have loaded all the elements? I use the following code. I'm wondering if there is any better solution than this. ### import pickle alist = [1, 2.0, 3, 4+6j] output=open('serialize_list.output/serialize_list.pkl', 'wb') for e in alist: pickle.dump(e, output) output.close() input=open('serialize_list.output/serialize_list.pkl', 'rb') try: while 1: e = pickle.load(input) print e except EOFError: pass Pickle the list length before its contents. -- Gabriel Genellina -- http://mail.python.org/mailman/listinfo/python-list
Re: Load a list subset with pickle?
On Thu, Oct 15, 2009 at 12:01 PM, Gabriel Genellina gagsl-...@yahoo.com.ar wrote: En Thu, 15 Oct 2009 13:05:18 -0300, Peng Yu pengyu...@gmail.com escribió: How do I determine if I have loaded all the elements? I use the following code. I'm wondering if there is any better solution than this. ### import pickle alist = [1, 2.0, 3, 4+6j] output=open('serialize_list.output/serialize_list.pkl', 'wb') for e in alist: pickle.dump(e, output) output.close() input=open('serialize_list.output/serialize_list.pkl', 'rb') try: while 1: e = pickle.load(input) print e except EOFError: pass Pickle the list length before its contents. Suppose that the list length was not in 'serialize_list.output/serialize_list.pkl'. Is try-except block the best solution? -- http://mail.python.org/mailman/listinfo/python-list
Load a list subset with pickle?
I use pickle to dump a long list. But when I load it, I only want to load the first a few elements in the list. I am wondering if there is a easy way to do so? Thank you! -- http://mail.python.org/mailman/listinfo/python-list
Re: Load a list subset with pickle?
On 2009-10-13 13:00 PM, Peng Yu wrote: I use pickle to dump a long list. But when I load it, I only want to load the first a few elements in the list. I am wondering if there is a easy way to do so? Thank you! Not by pickling the list. However, you can concatenate pickles, so you could just pickle each item from the list in order to the same file and only unpickle the first few. In [1]: import cPickle In [2]: from cStringIO import StringIO In [3]: very_long_list = range(10) In [4]: f = StringIO() In [5]: for item in very_long_list: ...: cPickle.dump(item, f) ...: ...: In [6]: f.seek(0,0) In [7]: cPickle.load(f) Out[7]: 0 In [8]: cPickle.load(f) Out[8]: 1 In [9]: cPickle.load(f) Out[9]: 2 In [10]: cPickle.load(f) Out[10]: 3 -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list