[
https://issues.apache.org/jira/browse/HDFS-3900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13451001#comment-13451001
]
Todd Lipcon commented on HDFS-3900:
-----------------------------------
bq. One question: note that the refactor into the abortCurSegment() method also
causes the code in newEpoch to now call "curSegment.abort()" instead of
"curSegment.close()". Is there any semantic change there that we should be
concerned about?
I noticed this as well, but I determined it doesn't really matter. The only
difference is that abort() leaves the preallocated padding at the end of the
file, whereas close() truncates. Really it might make sense to change all of
the abort() calls to close() but I didn't want to change that in this JIRA. But
semantically, we never use the byte-wise length, just the valid-txn-length, so
it doesn't really matter.
> QJM: avoid validating log segments on log rolls
> -----------------------------------------------
>
> Key: HDFS-3900
> URL: https://issues.apache.org/jira/browse/HDFS-3900
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Affects Versions: QuorumJournalManager (HDFS-3077)
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Attachments: hdfs-3900.txt
>
>
> Currently, we are paranoid and validate every log segment when it is
> finalized. For the a log segment that has been written entirely by one
> writer, with no recovery in between, this is overly paranoid (we don't do
> this for local journals). It also causes log rolls to be slow and take time
> linear in the size of the segment. Instead, we should optimize this path to
> simply trust that the segment is correct so long as the txids match up as
> expected.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
