[
https://issues.apache.org/jira/browse/HADOOP-5188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12675143#action_12675143
]
Konstantin Shvachko commented on HADOOP-5188:
---------------------------------------------
Sorry it took me a little more than a couple of days.
_Luca> will your patch support multiple logging system at a time?_
Yes HADOOP-4539 supports multiple logging systems. Namely, it supports
traditional file streams and (what I call) backup streams. In my view you
should be able to extend EditLogOutputStream and it will work with BookKeeper.
_Luca> 0.19 didn't allow me to modify the logSync() method. Will the patch
you'll post fix the issue?_
No I don't think logSync() method should be modified at least not the way you
did in your patch HADOOP-5189.
Synchronization (locking) in the logSync() method prevents different handler
threads to write the same information (edits) multiple times into the same
edits file. Suppose you have 1 file stream (one edits file) and 2 threads call
logSync() at the same time. If you remove locking as you did in your patch then
both threads may write the same edits transactions into the same file twice.
This will simply corrupt the edits file.
logSync() may be modified to write in parallel into _different streams_. If you
have 2 file streams then it makes sense to write into 2 files in parallel. But
the information should be written into each file only once and in the right
order. And your patch breaks that, afaiu.
My main objection to your approach though, is abstracting the whole EditsLog
class rather than just the {{EditLog*Streams}}. This leads to replication of a
lot of code. Say, both classes {{EditLog}} and {{BKEditLogThreadBuf}} have the
{{loadFSEdits()}} methods, which are literally identical, right? We should
avoid that at all cost.
> Modifications to enable multiple types of logging
> --------------------------------------------------
>
> Key: HADOOP-5188
> URL: https://issues.apache.org/jira/browse/HADOOP-5188
> Project: Hadoop Core
> Issue Type: Improvement
> Components: dfs
> Affects Versions: 0.19.0
> Reporter: Luca Telloli
> Attachments: HADOOP-5188.patch
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
