[
https://issues.apache.org/jira/browse/HDFS-3900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13450304#comment-13450304
]
Todd Lipcon commented on HDFS-3900:
-----------------------------------
should note: I wasn't able to easily write a unit test for this, but I verified
that the log rolls no longer validated the logs whereas edits synchronization
did. This also sped up the log roll timing significantly.
> QJM: avoid validating log segments on log rolls
> -----------------------------------------------
>
> Key: HDFS-3900
> URL: https://issues.apache.org/jira/browse/HDFS-3900
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Affects Versions: QuorumJournalManager (HDFS-3077)
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Attachments: hdfs-3900.txt
>
>
> Currently, we are paranoid and validate every log segment when it is
> finalized. For the a log segment that has been written entirely by one
> writer, with no recovery in between, this is overly paranoid (we don't do
> this for local journals). It also causes log rolls to be slow and take time
> linear in the size of the segment. Instead, we should optimize this path to
> simply trust that the segment is correct so long as the txids match up as
> expected.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
