[
https://issues.apache.org/jira/browse/THRIFT-110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bryan Duxbury updated THRIFT-110:
---------------------------------
Attachment: compact_proto_spec.txt
I've created a grammar-like spec for the compact format and attached it here.
Some general notes on features:
* Heavy use of variable length integers to keep size down.
* Wherever possible, use the 4 MSB in the type field as a modifier to store
type-specific info.
* Booleans are encoded as constants.
* String externalization is supported on RPC calls, but not direct
serialization.
Some known issues:
* Autogenerated negative field ids will be represented poorly as varints. Need
to think of a strategy that doesn't overly punish those who use specified field
ids, but doesn't immediately use 3-5 bytes for autogenerated values.
* Containers must contain values all encoded the same way, so the protocol will
have to choose the most effective encoding for all the values.
* Doubles can possibly be varint compressed as well... need to do more research.
I'd much appreciate people weighing in on the ideas in this spec and the open
issues I've listed above.
> A more compact format
> ----------------------
>
> Key: THRIFT-110
> URL: https://issues.apache.org/jira/browse/THRIFT-110
> Project: Thrift
> Issue Type: Improvement
> Reporter: Noble Paul
> Attachments: compact_proto_spec.txt
>
>
> Thrift is not very compact in writing out data as (say protobuf) . It does
> not have the concept of variable length integers and various other
> optimizations possible . In Solr we use a lot of such optimizations to make a
> very compact payload. Thrift has a lot common with that format.
> It is all done in a single class
> http://svn.apache.org/viewvc/lucene/solr/trunk/src/java/org/apache/solr/common/util/NamedListCodec.java?revision=685640&view=markup
> The other optimizations include writing type/value in same byte, very fast
> writes of Strings, externalizable strings etc
> We could use a thrift format for non-java clients and I would like to see it
> as compact as the current java version
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
