[
https://issues.apache.org/jira/browse/HBASE-20952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16615267#comment-16615267
]
Josh Elser commented on HBASE-20952:
------------------------------------
{quote}The current implementation has a bunch of dependencies on file
operations such as temporary files, list of corrupted files, etc. From HBase
perspective, it would be much easier to keep it as is and make log splitter an
interface that should take log and create list of recovery logs.
{quote}
{quote}In other hands for the 1st iteration, this may be hidden by the schema
where 3rd party wal may not use the splitter at all and recovery would be
reading a stream of records provided by the WALProvider for a particular region
{quote}
Thanks for your assessment, Sergey. Your take makes sense to me (e.g. let WAL
implementation decide if it needs splitting). This concern and assessment is
something that should be captured in the design doc, [~yuzhih...@gmail.com].
> Re-visit the WAL API
> --------------------
>
> Key: HBASE-20952
> URL: https://issues.apache.org/jira/browse/HBASE-20952
> Project: HBase
> Issue Type: Sub-task
> Components: wal
> Reporter: Josh Elser
> Priority: Major
> Attachments: 20952.v1.txt
>
>
> Take a step back from the current WAL implementations and think about what an
> HBase WAL API should look like. What are the primitive calls that we require
> to guarantee durability of writes with a high degree of performance?
> The API needs to take the current implementations into consideration. We
> should also have a mind for what is happening in the Ratis LogService (but
> the LogService should not dictate what HBase's WAL API looks like RATIS-272).
> Other "systems" inside of HBase that use WALs are replication and
> backup&restore. Replication has the use-case for "tail"'ing the WAL which we
> should provide via our new API. B&R doesn't do anything fancy (IIRC). We
> should make sure all consumers are generally going to be OK with the API we
> create.
> The API may be "OK" (or OK in a part). We need to also consider other methods
> which were "bolted" on such as {{AbstractFSWAL}} and
> {{WALFileLengthProvider}}. Other corners of "WAL use" (like the
> {{WALSplitter}} should also be looked at to use WAL-APIs only).
> We also need to make sure that adequate interface audience and stability
> annotations are chosen.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
