[ https://issues.apache.org/jira/browse/HBASE-14070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16142478#comment-16142478 ]
Amit Patel edited comment on HBASE-14070 at 8/26/17 12:15 AM: -------------------------------------------------------------- Just as a status update, I'll be finishing my internship so I will be much less active in the future (I still hope to maintain some degree of involvement with the work). I am extremely grateful for the help and mentorship that I have received from the community. At the moment there are still a few outstanding issues that should be addressed before merging the work into master: * Performance of HLC (Clock#now, Clock#update are expensive, not as big of an issue if HLC is only used for meta but certainly is for user tables) * Huge number of tests timing out in in the pre-commit/[public branch|https://builds.apache.org/job/HBASE-14070.HLC/] builds (but haven't been able to replicate the test time-out and they pass just fine if ran locally) In terms of enabling HLC on user tables, remaining issues would include: * Some tests that are still explicitly setting the timestamp and need to be converted to instead manipulate the current time via mocking the clocks * Bulk load does not update the HLC Some additional brainstorming/discussions includes: * [Deprecate setting of timestamp in client for HLC|https://issues.apache.org/jira/browse/HBASE-18642] * [Add 'Transaction ID' to Result for HLC|https://issues.apache.org/jira/browse/HBASE-18643] was (Author: amit.patel): Just as a status update, I'll be finishing my internship so I will be much less active in the future. I am extremely grateful for the help and support that I have received from the community. At the moment there are still a few outstanding issues that should be addressed before merging the work into master: * Performance of HLC (Clock#now, Clock#update are expensive, not as big of an issue if HLC is only used for meta but certainly is for user tables) * Huge number of tests timing out in in the pre-commit/[public branch|https://builds.apache.org/job/HBASE-14070.HLC/] builds (but haven't been able to replicate the test time-out and they pass just fine if ran locally) In terms of enabling HLC on user tables, remaining issues would include: * Some tests that are still explicitly setting the timestamp and need to be converted to instead manipulate the current time via mocking the clocks * Bulk load does not update the HLC Some additional brainstorming/discussions includes: * [Deprecate setting of timestamp in client for HLC|https://issues.apache.org/jira/browse/HBASE-18642] * [Add 'Transaction ID' to Result for HLC|https://issues.apache.org/jira/browse/HBASE-18643] > Hybrid Logical Clocks for HBase > ------------------------------- > > Key: HBASE-14070 > URL: https://issues.apache.org/jira/browse/HBASE-14070 > Project: HBase > Issue Type: New Feature > Reporter: Enis Soztutar > Assignee: Amit Patel > Attachments: HBASE-14070.master.001.patch, > HybridLogicalClocksforHBaseandPhoenix.docx, > HybridLogicalClocksforHBaseandPhoenix.pdf > > > HBase and Phoenix uses systems physical clock (PT) to give timestamps to > events (read and writes). This works mostly when the system clock is strictly > monotonically increasing and there is no cross-dependency between servers > clocks. However we know that leap seconds, general clock skew and clock drift > are in fact real. > This jira proposes using Hybrid Logical Clocks (HLC) as an implementation of > hybrid physical clock + a logical clock. HLC is best of both worlds where it > keeps causality relationship similar to logical clocks, but still is > compatible with NTP based physical system clock. HLC can be represented in > 64bits. > A design document is attached and also can be found here: > https://docs.google.com/document/d/1LL2GAodiYi0waBz5ODGL4LDT4e_bXy8P9h6kWC05Bhw/edit# -- This message was sent by Atlassian JIRA (v6.4.14#64029)