[
https://issues.apache.org/jira/browse/HBASE-20003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16413621#comment-16413621
]
Anoop Sam John commented on HBASE-20003:
----------------------------------------
[~chancelq] Yes in the case you mentioned the 2 replicas are out of sync with
latest mutation. The secondary is not yet in bad state considering what got
committed in primary. So yes it is eligible to become the primary. And on the
case that we may see the latest data (Which is not committed to user) if
tertiary was selected to become new primary or not see it is secondary became
new primary. I agree this is strange and we can discuss this how/whether we
can really avoid. But the point is we even now have this issue. Single region
only (no replica) and a write comes and data is synced to WAL and then node
went down. The writer did not get reply in time and see it as write fail. But
on WAL replay , the data will come back. If the fail was before the WAL
append, it won't come. IMHO this situation is very similar to what you said.
WDYT?
Duo - It is not HDFS less.. The Hfiles are still with HDFS and we need its
data availability maintain mechanism like the re replication for under
replication etc. Ya it is WAL been avoided and its replication is done as
region memstore data replication.
> WALLess HBase on Persistent Memory
> ----------------------------------
>
> Key: HBASE-20003
> URL: https://issues.apache.org/jira/browse/HBASE-20003
> Project: HBase
> Issue Type: New Feature
> Reporter: Anoop Sam John
> Assignee: Anoop Sam John
> Priority: Major
>
> This JIRA aims to make use of persistent memory (pmem) technologies in HBase.
> One such usage is to make the Memstore to reside on pmem. Making a persistent
> memstore would remove the need for WAL and paves way for a WALLess HBase.
> The existing region replica feature could be used here and ensure the data
> written to memstores are synchronously replicated to the replicas and ensure
> strong consistency of the data. (pipeline model)
> Advantages :
> - Data Availability : Since the data across replicas are consistent
> (synchronously written) our data is always 100 % available.
> - Lower MTTR : It becomes easier/faster to switch over to the replicas on a
> primary region failure as there is no WAL replay involved. Building the
> memstore map data also is much faster than reading the WAL and replaying the
> WAL.
> - Possibility of bigger memstores : These pmems are designed to have more
> memory than DRAMs so it would also enable us to have bigger sized memstores
> which leads to lesser flushes/compaction IO.
> - Removes the dependency of HDFS on the write path
> Initial PoC has been designed and developed. Testing is underway and we would
> publish the PoC results along with the design doc sooner. The PoC doc will
> talk about the design decisions, the libraries considered to work with these
> pmem devices, pros and cons of those libraries and the performance results.
> Note : Next gen memory technologies using 3DXPoint gives persistent memory
> feature. Such memory DIMMs are soon to appear in the market. The PoC is done
> around Intel's ApachePass (AEP)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
