[
https://issues.apache.org/jira/browse/PHOENIX-1422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14240420#comment-14240420
]
Andrew Purtell edited comment on PHOENIX-1422 at 12/10/14 12:48 AM:
--------------------------------------------------------------------
Hybrid Logical Clock
(http://muratbuffalo.blogspot.com/2014/07/hybrid-logical-clocks.html) might be
a good approach for generating stateless sequences. It looks like a HLC logical
timestamp is monotonic, uses NTP synchronized physical clocks but smooths over
hiccups, and fits in 64 bits:
{quote}
We can have a compact representation of HLC timestamps using l and c. NTP uses
64-bit timestamps which consist of a 32-bit part for seconds and a 32-bit part
for fractional second. We restrict l to track only the most significant 48 bits
of pt. Rounding up pt values to 48 bits l values still gives us microsecond
granularity tracking of pt. 16 bits remain for c and allows it room to grow up
to 65536. (In our experiments c mostly stayed in single digits.)
{quote}
was (Author: apurtell):
Hybrid Logical Clock
(http://muratbuffalo.blogspot.com/2014/07/hybrid-logical-clocks.html) might be
a good approach for generating stateless sequences. It looks like a HLC logical
timestamps monotonic and fit in 64 bits:
{quote}
We can have a compact representation of HLC timestamps using l and c. NTP uses
64-bit timestamps which consist of a 32-bit part for seconds and a 32-bit part
for fractional second. We restrict l to track only the most significant 48 bits
of pt. Rounding up pt values to 48 bits l values still gives us microsecond
granularity tracking of pt. 16 bits remain for c and allows it room to grow up
to 65536. (In our experiments c mostly stayed in single digits.)
{quote}
> Stateless Sequences
> -------------------
>
> Key: PHOENIX-1422
> URL: https://issues.apache.org/jira/browse/PHOENIX-1422
> Project: Phoenix
> Issue Type: Improvement
> Reporter: Lars Hofhansl
>
> [~apurtell] and I were chatting yesterday.
> It would be good if Phoenix had stateless sequences, i.e. sequences that give
> out unique ids with (very) high probability.
> We can do that by starting with a timestamp, shifting it left 16 or 24 bits
> left and fill in the new bits with a random number.
> So we're guaranteed to get a new id for each millisecond and within a
> millisecond we break the tie with a random number. If we can make the
> likelihood of duplicate numbers lower than (say) a data center failure, we're
> OK. I would test this with a upsert into x ... select from x ... type query
> inserting 100's of millions of rows.
> Need to think of a syntax too.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
