Protocol Buffers is yet another RPC scheme that requires compilation of the data types. If on the other hand you define simple data types this can be much simpler and finessed, and including dealing with such RPC issues as endian-ness. ....wondering if is there any sort of compression of XML that is done solely in utf8 encoding (rather than binary)? Also, I'm not wanting a solution that involves paging the data, since the data I'm after is the entire list. Breaking up the data does nothing for me except take longer and increase bandwidth needs. I want the entire chunk with the least cost to Twitter and myself of time and cpu/bandwidth... so smaller data is better. Paging is also bad because of it will cost me as a rate-limiting penalty... unless the page sizes are really large... 8-) Speaking of which, BarackObama is the worst by far with 600k+ lists... but is the trend only going to get worse? When/if Twitter really gets popular will we be seeing users with 1mil users lists? At some point, even with a binary RPC scheme and data compression this can get expensive... (me thinks caching on my side is unavoidable)
On Tue, Mar 3, 2009 at 1:13 PM, Alex Payne <[email protected]> wrote: > > That would definitely require us to weigh our current knowledge of > Thrift vs Protocol Buffers. I'll think about it. > > On Tue, Mar 3, 2009 at 12:42, Dossy Shiobara <[email protected]> wrote: > > > > On 3/3/09 3:07 PM, Alex Payne wrote: > >> > >> We're fully aware of what Protocol Buffers are their intended use. We > >> use Thrift, Facebook's clone of Protocol Buffers. You might note the > >> use of the world "internal" in the material you quoted. > > > > Quoting from http://code.google.com/apis/protocolbuffers/docs/faq.html: > > > > "We would like to provide public APIs that accept protocol buffers as > > well as XML, both because it is more efficient and because we're just > going > > to convert that XML to protocol buffers on our end anyway." > > > > Their use of the word "internal" simply clarifies where they _currently_ > use > > it, not its limitation. > > > > Could Twitter be the first service to offer protocol buffers? Sure. I > > guess you're saying it's not going to happen, though. > > > > -- > > Dossy Shiobara | [email protected] | http://dossy.org/ > > Panoptic Computer Network | http://panoptic.com/ > > "He realized the fastest way to change is to laugh at your own > > folly -- then you can let go and quickly move on." (p. 70) > > > > > > -- > Alex Payne - API Lead, Twitter, Inc. > http://twitter.com/al3x >
