David Hirschfield wrote: > I have a pair of programs which trade python data back and forth by > pickling up lists of objects on one side (using > pickle.HIGHEST_PROTOCOL), and sending that data over a TCP socket > connection to the receiver, who unpickles the data and uses it. > > So far this has been working fine, but I now need a way of separating > multiple chunks of pickled binary data in the stream being sent back and > forth. > > Questions: > > Is it safe to do what I'm doing? I didn't think there was anything > fundamentally wrong with sending binary pickled data, especially in the > closed, safe environment these programs operate under...but maybe I'm > making a poor assumption? > > I was going to separate the chunks of pickled data with some well-formed > string, but couldn't that string potentially randomly appear in the > pickled data? Do I just pick an extremely > unlikely-to-be-randomly-generated string as the separator? Is there some > string that will definitely NEVER show up in pickled binary data? > > I thought about base64 encoding the data, and then decoding on the > opposite side (like what xmlrpclib does), but that turns out to be a > very expensive operation, which I want to avoid, speed is of the essence > in this situation. > > Is there a reliable way to determine the byte count of some pickled > binary data? Can I rely on len(<pickled data>) == bytes? > Instead of communicating directly with the TCP socket, you could talk to it via an object which precedes each chunk with a byte count, and if you're working with multiple streams of picked data, then each chunk could also have an identifier which specified which stream it belonged to.
-- http://mail.python.org/mailman/listinfo/python-list
