[
https://issues.apache.org/jira/browse/HADOOP-5188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12676669#action_12676669
]
Flavio Paiva Junqueira commented on HADOOP-5188:
------------------------------------------------
Konstantin says that if we don't synchronize on logSync, then edits can be
written multiple times and the edit log would be corrupted. Although this
statement sounds correct for the current implementation of FSEditLog, it is not
mandatory for every possible implementation of an EditLog, and that's why Luca
is proposing to change it. One example is the implementation in the patch. With
this implementation, there is no shared buffer, and each thread has its own
buffer and writes that buffer to a BookKeeper ledger, which obviates the need
for synchronization on logSync.
I understand that there might be a concern about the semantics of the file
system, but in the way I see there is no such a problem. Having a buffer for
each thread and writing it to BookKeeper does not violate the file system
semantics because BookKeeper respects the order of calls to add edits to a
ledger, and calls back to the Namenode using the same order. That is, the
namenode will receive callbacks in the same order that have been submitted, so
if the namenode processes return values using the same order of callbacks, then
the execution order will be respected. Note that there is no change with
respect to the order of callback processing, and if the Namenode processes them
correctly today, then I don't see why it wouldn't with the proposed solution.
Back to the logSync issue, in my interpretation, Konstantin is trying to
enforce the semantics defined for the file system by not allowing logSync to be
overridden. This is a valid argument, but the lack of flexibility precludes
solutions that do not require logSync implemented in the way it is currently,
so I'd rather have a more flexible implementation.
> Modifications to enable multiple types of logging
> --------------------------------------------------
>
> Key: HADOOP-5188
> URL: https://issues.apache.org/jira/browse/HADOOP-5188
> Project: Hadoop Core
> Issue Type: Improvement
> Components: dfs
> Affects Versions: 0.19.0
> Reporter: Luca Telloli
> Attachments: HADOOP-5188.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
