[
https://issues.apache.org/jira/browse/THRIFT-110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12624560#action_12624560
]
David Reiss commented on THRIFT-110:
------------------------------------
The type information is necessary for two reasons. The first is that if
someone changes the type of a field (which they shouldn't, but they might), we
want to just skip that field rather than desyncing the entire stream. The
second reason is more important. If one side of the connection sends a field
that the other side doesn't understand, we need the type information embedded
in the stream in order to know how to properly skip it.
The most obvious downside of using a bitfield (aside from a more complicated
implementation) is that it is worse for very sparse structures. If you only
have one field set but that field happens to be number 600, you're going to be
wasting a lot of space. It would definitely save space in (what we have found
to be) the more common cases, though.
> A more compact format
> ----------------------
>
> Key: THRIFT-110
> URL: https://issues.apache.org/jira/browse/THRIFT-110
> Project: Thrift
> Issue Type: Improvement
> Reporter: Noble Paul
>
> Thrift is not very compact in writing out data as (say protobuf) . It does
> not have the concept of variable length integers and various other
> optimizations possible . In Solr we use a lot of such optimizations to make a
> very compact payload. Thrift has a lot common with that format.
> It is all done in a single class
> http://svn.apache.org/viewvc/lucene/solr/trunk/src/java/org/apache/solr/common/util/NamedListCodec.java?revision=685640&view=markup
> The other optimizations include writing type/value in same byte, very fast
> writes of Strings, externalizable strings etc
> We could use a thrift format for non-java clients and I would like to see it
> as compact as the current java version
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
