[
https://issues.apache.org/jira/browse/THRIFT-110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12639483#action_12639483
]
Chad Walters commented on THRIFT-110:
-------------------------------------
bq. Containers - I'm not sure that requiring all of the elements to be of a
single type is that big of a deal. If the options are storing an additional
byte per element to indicate type encoding vs just choosing the best compromise
type for all of them, I think the compromise is probably best. The only time
where it really becomes a problem is with ints, and that's easily resolvable.
If the numbers are all positive and less than a 64bits long, then we can use
positive varints. If they're all negative, we can use negative varints. If
there's a mix and the values are no bigger than the threshold, we can use
zigzag encoding. If none of those compromises yields the best size, then we can
always go for fixed size.
Bryan, I think it is difficult to implement this unless you are willing to
buffer a lot of state in the protocol, since the type code gets written out
before you have seen the values being written. This was one of the main reasons
I was proposing the type modifier extensions to the IDL -- passing the buck
along to the application writer.
> A more compact format
> ----------------------
>
> Key: THRIFT-110
> URL: https://issues.apache.org/jira/browse/THRIFT-110
> Project: Thrift
> Issue Type: Improvement
> Reporter: Noble Paul
> Attachments: compact_proto_spec.txt, compact_proto_spec.txt
>
>
> Thrift is not very compact in writing out data as (say protobuf) . It does
> not have the concept of variable length integers and various other
> optimizations possible . In Solr we use a lot of such optimizations to make a
> very compact payload. Thrift has a lot common with that format.
> It is all done in a single class
> http://svn.apache.org/viewvc/lucene/solr/trunk/src/java/org/apache/solr/common/util/NamedListCodec.java?revision=685640&view=markup
> The other optimizations include writing type/value in same byte, very fast
> writes of Strings, externalizable strings etc
> We could use a thrift format for non-java clients and I would like to see it
> as compact as the current java version
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
