[
https://issues.apache.org/jira/browse/HADOOP-5188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12672588#action_12672588
]
Luca Telloli commented on HADOOP-5188:
--------------------------------------
Raghu, in reply to your comment
https://issues.apache.org/jira/browse/HADOOP-5189?focusedCommentId=12672454#action_12672454
it's hard for me to tell if the current interface is "inadequate", but I can
definitely tell was insufficient for our needs, and this brings back to the
logSync() issue, that I'll detail here for the sake of completeness.
In the current implementation, during sync/flush a process among others gets
elected to be the "flusher" while the others wait for it to complete; the fact
that the NN writes multiple entries for each flush due to the interleaving of
threads is a very well written piece of code, but it is not good for our
purposes.
What we do with BookKeeper is to allow multiple parallel ordered asynchronous
writes to be pipelined on the BookKeeper side rather than the NN side, and to
achieve this we need to hack the logSync() method to avoid unnecessary
synchronizations. While the NN uses multiple synchronized threads, BookKeeper
uses queues, which we believe should help the overall performance.
Shortly: Bk does implement the two interfaces + hacks the logSync(), which is
not part of the interfaces. So I went the "abstract class" way, do you think
that's reasonable?
> Modifications to enable multiple types of logging
> --------------------------------------------------
>
> Key: HADOOP-5188
> URL: https://issues.apache.org/jira/browse/HADOOP-5188
> Project: Hadoop Core
> Issue Type: Improvement
> Components: dfs
> Affects Versions: 0.19.0
> Reporter: Luca Telloli
> Attachments: HADOOP-5188.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
