The multiprocessing queue uses pickle to serialize/deserialize the objects before/after sending them between processes.
Since the proto message contains a C extension class, it can't be serialized or deserialized with pickle. And it doesn't need to, just use the protobuf serialization/deserialization routines. So serialize and do Q.put(serizlied), then retrieve using Q.get() and deserialize. On Tue, Apr 26, 2011 at 7:37 PM, vish <vishi...@gmail.com> wrote: > So far what I've found is python protocol buffer objects are not > serializable using Pickle. > > On Apr 25, 3:55 pm, vish <vishi...@gmail.com> wrote: > > Environment: CentOS 64bit > > Python 2.6 > > Protocol Buffers 2.4.0.a > > Python client with Native C++ support > > python protocol buffers installed to ~/protobuf > > LD_LIBRARY_PATH=~/protobuf/lib:~/protobuf/google/protobuf/src/.libs > > PYTHONPATH=:~/protobuf/:.:~/protobuf/google/protobuf/src/.libs:~/ > > protobuf/lib > > > > Hi, > > In a multiprocessing environment, my code is trying to add decoded > > messages to a shared Queue, and then pop them off at a later stage. > > > > Inserting messages in the Queue is working fine. But while fetching > > those messages, am getting this error: > > > > Traceback (most recent call last): > > File "/usr/lib64/python2.6/multiprocessing/queues.py", line 242, in > > _feed > > send(obj) > > PicklingError: Can't pickle <type > > > 'google3.net.google.protobuf.python.internal._net_proto2___python.CMessage' > >: > > import of module > > google3.net.google.protobuf.python.internal._net_proto2___python > > failed > > > > Verified LD_LIBRARY_PATH, LD_RUN_PATH, PYTHONPATH and they all seem to > > be valid. Anyway help in solving this problem is appreciated. > > > > Inserting messages to Queue: > > > > while not self.stopped(): > > el = self.rcvr.next_message() > > if el is not None: > > print "%s is the event"%el > > Q.put(el) > > > > Popping messages off Queue: > > > > def next_message(self): > > """Returns next event (if available) from Queue""" > > return Q.get() > > > > where Q is multiprocessing.Queue > > -- > You received this message because you are subscribed to the Google Groups > "Protocol Buffers" group. > To post to this group, send email to protobuf@googlegroups.com. > To unsubscribe from this group, send email to > protobuf+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/protobuf?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to protobuf@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
