On Oct 20, 2010, at 6:30 AM, Wayne wrote: > I am not sure how many bytes,
Then I don't think your performance numbers really mean anything substantial. Deserialization time is inevitably going to go up with the amount of data present, so unless you know how much data you actually have, there's no way to know if the time being taken is reasonable or not. > but we do convert the cassandra object that is returned in 3s into a > dictionary in ~1s and then again into a custom python object in about ~1.5s. > Expectations are based on this timing. If we can convert what thrift returns > into a completely new python object in 1s why does thrift need 3s to give it > to us? (de)serialization, even of binary protocols, is a good deal more complex than simply copying around data you already have on the heap (or maybe even in CPU cache, if it's small enough) in native format. Worse, unless you're explicitly doing deep copies (which is usually the wrong thing to do), you're probably _not_ constructing a "completely new python object". You're constructing an object that has references to bits of data already deserialized onto the heap by Thrift. This makes the comparison even less meaningful. -NK
