On 10/20/10 8:12 AM, M. Edward (Ed) Borasky wrote: >> 6) Why not restrict IDs to 53bits? >> >> A Snowflake ID is composed: >> * 41bits for millisecond precision time (69 years) >> * 10bits for a configured machine identity (1024 machines) >> * 12bits for a sequence number (4096 per machine) >> >> The factor influencing the length of the ID is the time. For a 53bit >> ID this >> would mean only 31bits are available for the time. 31bits is only >> enough for >> 24 days (2147483648/(1000*60*60*24)) of time. >> >> Reducing the resolution of the timestamp would prevent a K-sorted >> resolution >> of 1 second or less. >> >> Reducing the configured machine identity or sequence number by 1bit would >> mean we couldn’t scale Twitter, or operate our infrastructure in an >> uncoordinated high-available way. > > Interesting ... so you have the theoretical capacity to scale to 2**22 > (about 4 million) tweets per millisecond? Even 4 million tweets a second > seems unrealistic, as does a single "machine" only being able to > generate 4096 IDs. I think if you're really expecting this kind of > volume, the FPGA vendors probably can help you out. We are talking > clocks and counters, here, right, not Javascript interpreters or robust > linear regressions? ;-) > > Ah, well, I'll check back on you guys in 69 years to see how you're > holding up. ;-) >
I'd say that you could remove a maximum of 2 bits from the time - this would divide the 69 years by 4, making it a max of 17 years. By then, I'd assume that we are past the 53-bit limit. You could remove 3 bits from the machine ID and 5 bits from the sequence number. It would mean that there could be only 128 ID servers with 128 IDs per millisecond per machine -> 16 million tweets per second. In total you would have removed 10 bits from a number that had only 63 bits -> 53 bits. The question is: do you want that? I don't think you do. I really prefer the current solution. Tom -- Twitter developer documentation and resources: http://dev.twitter.com/doc API updates via Twitter: http://twitter.com/twitterapi Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list Change your membership to this group: http://groups.google.com/group/twitter-development-talk
