[
https://issues.apache.org/jira/browse/HDFS-2018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13059988#comment-13059988
]
Ivan Kelly commented on HDFS-2018:
----------------------------------
I've removed a lot of tests from TestFSImageStorageInspector. The functionality
is now tested by the following:
public void testCurrentStorageInspector() throws IOException {
- Simply removed editlog stuff
public void testPlanWithGaps() throws IOException {
- TestEditLog#testLoadingWithGaps()
public void testPlanWithInProgressInMiddle() throws IOException {
- Scenario is not possible. Inprogress files will be finalized before
creating a new segment, however this _will_ lead to the case of
InconsistentEndTxIds (see below).
public void testLogGroupRecoveryNoop() throws IOException {
- TestFileJournalManager#testNormalOperation
public void testLogGroupRecoveryMixed() throws IOException {
- TestFileJournalManager#testInprogressRecovery
- TestFileJournalManager#testInprogressRecoveryMixed
public void testLogGroupRecoveryInconsistentEndTxIds() throws IOException {
- TestFileJournalManager#testManyLogsWithGapWhereInprogressInMiddle
This isn't a failure case. If log A has 1-2 and log B has 1-5, then that
means A failed after transaction 2 and wasn't reenabled until the next roll if
at all. This means that A has a gap, from 3-5, so on loading B will be selected
for that segment.
public void testLogGroupRecoveryInProgress() throws IOException {
- TestFileJournalManager#testInprogressRecoveryAll
public void testCurrentSplitEditsAndImage() throws IOException {
- No longer relevant whith storage inspector only handling image files
public void testLogManifest() throws IOException {
- TestEditLog#testEditLogManifest
public void testLogManifestInProgressComesFirst() throws IOException {
- TestEditLog#testEditLogInprogressComesFirst
> Move all journal stream management code into one place
> ------------------------------------------------------
>
> Key: HDFS-2018
> URL: https://issues.apache.org/jira/browse/HDFS-2018
> Project: Hadoop HDFS
> Issue Type: Improvement
> Reporter: Ivan Kelly
> Assignee: Ivan Kelly
> Fix For: Edit log branch (HDFS-1073)
>
> Attachments: HDFS-2018.diff, HDFS-2018.diff, HDFS-2018.diff,
> HDFS-2018.diff, HDFS-2018.diff, HDFS-2018.diff, HDFS-2018.diff, HDFS-2018.diff
>
>
> Currently in the HDFS-1073 branch, the code for creating output streams is in
> FileJournalManager and the code for input streams is in the inspectors. This
> change does a number of things.
> - Input and Output streams are now created by the JournalManager.
> - FSImageStorageInspectors now deals with URIs when referring to edit logs
> - Recovery of inprogress logs is performed by counting the number of
> transactions instead of looking at the length of the file.
> The patch for this applies on top of the HDFS-1073 branch + HDFS-2003 patch.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
