[ https://issues.apache.org/jira/browse/HBASE-20952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16615326#comment-16615326 ]
Sergey Soldatov commented on HBASE-20952: ----------------------------------------- A few of my personal thoughts that are based on previous experience of implementing such kind of functionality in HBase (C5, non-stop hbase, hydrabase). It would be better to perform atomic meaningful changes that address a particular problem without breaking the existing functionality and without adding an additional complexity. We need a 'pluggable' WALProvider that will be agnostic to file system structure. Fine. Let's make it configurable and remove the dependency on file systems (introducing WALIdentity or whatever we call it). It doesn't work with the current implementation of replication? Not good, but we may mention that it in the Limitation section. All we need is an understanding that it's possible to implement even if it would require a lot of efforts. Possible the replication itself requires a new Jira 'Revisit Replication approach' especially if we are talking about different kind of wal providers. Possible someone would love to have a setup with replication between on-prem cluster that is using regular WAL and another in AWS using Bookeeper wal. And that would require another layer of abstraction for replication itself. Should it be done under this jira? Honestly speaking I don't think so. > Re-visit the WAL API > -------------------- > > Key: HBASE-20952 > URL: https://issues.apache.org/jira/browse/HBASE-20952 > Project: HBase > Issue Type: Sub-task > Components: wal > Reporter: Josh Elser > Priority: Major > Attachments: 20952.v1.txt > > > Take a step back from the current WAL implementations and think about what an > HBase WAL API should look like. What are the primitive calls that we require > to guarantee durability of writes with a high degree of performance? > The API needs to take the current implementations into consideration. We > should also have a mind for what is happening in the Ratis LogService (but > the LogService should not dictate what HBase's WAL API looks like RATIS-272). > Other "systems" inside of HBase that use WALs are replication and > backup&restore. Replication has the use-case for "tail"'ing the WAL which we > should provide via our new API. B&R doesn't do anything fancy (IIRC). We > should make sure all consumers are generally going to be OK with the API we > create. > The API may be "OK" (or OK in a part). We need to also consider other methods > which were "bolted" on such as {{AbstractFSWAL}} and > {{WALFileLengthProvider}}. Other corners of "WAL use" (like the > {{WALSplitter}} should also be looked at to use WAL-APIs only). > We also need to make sure that adequate interface audience and stability > annotations are chosen. -- This message was sent by Atlassian JIRA (v7.6.3#76005)