Looking at things from the standpoint of efficiency also argues for a limited set of modifiers, since we need to actually pass this information on the wire.
In THRIFT-110, I suggest that we reserve 2 bits at the top of the type byte for modifiers. This allows for 3 different modifiers for any given primary type (plus a default). For integer types, I suggested that we add "fixed" and "zipper"/"zigzag". The default assumption would be that a variable encoding could be used. "fixed" would mean that variable encoding is unlikely to be helpful, and "zipper"/"zigzag" would suggest a variable encoding suitable for fields likely to contain a mix of negative and positive values. For string types, I suggested that we add "extern". Strings marked with this modifier could be stored in some separate indexed table and referenced by index when passed through the protocol. Does this sound like a good starting point? Are there other modifiers we'd like to add? Chad On 8/21/08 4:41 PM, "Mark Slee" <[EMAIL PROTECTED]> wrote: I'd propose starting with a fixed set of type hints that we agree upon. We could certainly implement a dynamic any-string goes type of thing. But here's my concern with that: 1/ If we agree upon a fixed set of modifiers, we're more likely to get them implemented across more languages since deciding on the set will involve getting some basic agreement amongst the Thrift community. We're likely to retain more overall compatibility. 2/ If the modifiers are dynamic, it implies that people would make their own local changes, and write their own custom Protocol implementations. We're more likely to duplicate this work or disagree on things. This will make custom protocol submissions back into the trunk more difficult, and the core project becomes more likely to lose quality work. So, while a dynamic implementation is arguably more elegant/flexible, I think having the community agree upon a reasonable set of type annotations is most healthy for the coherence of Thrift in the long run. Cheers, Mark -----Original Message----- From: Bryan Duxbury [mailto:[EMAIL PROTECTED] Sent: Thursday, August 21, 2008 3:40 PM To: [email protected] Subject: Re: Adding a "vint" type to Thrift I'm not opposed to using a type modifier approach instead of a separate type. Will we really be adding "arbitrary" annotations to the objects, or will they be of a specific nature? That is, will we have modifiers like "fixed", "variable", "extern", etc. be compiler tokens, or will you just be allowed to add any string you want as a modifier before the type? The latter would certainly be more flexible, but would be more complicated as well. -Bryan On Aug 21, 2008, at 2:07 PM, David Reiss wrote: > First, I don't understand why it would be limited to 64-bit. What if > I have an integer that I want represented as 32 bits in memory but > serialized with a variable-length encoding. > > Second, I wonder if creating a new type is the best solution. I've > been thinking for a while that it might be useful to add arbitrary > annotations to types. We already have a few special cases of this > (cpp_type in C++, slist in Erlang), and vint would be a third. I > think it might be time to bite the bullet and implement annotations so > we can stop adding new syntactic constructs for each minor extension > to the data model. > > --David > > Bryan Duxbury wrote: >> All, >> >> We've been following the discussion on THRIFT-110. I think as a first >> step we should add a "vint" (variable integer) to Thrift. >> >> My general proposal is that we add vint as a new type. Protocols that >> don't do variable length integers can treat vints as i64s. Protocols >> that do implement variable length integers can choose encodings that >> make sense for them. >> >> We'd have to change the compiler and all the generators and libraries >> to recognize vint, if for no other reason than to treat it as an i64. >> However, this would make implementing other protocols that included >> variable length integers very easy. >> >> Are there any objections to this proposal? If not, I'll make an >> issue, and Rapleaf is willing to commit time to this enhancement. >> >> -Bryan >>
