[ https://issues.apache.org/jira/browse/HDFS-14631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16887634#comment-16887634 ]
He Xiaoqiao commented on HDFS-14631: ------------------------------------ [~LiJinglun], [^HDFS-14631.002.patch] looks almost good to me. Some minor comments. 1. The constant block id number is not clean at all. Please check #testLocalReplicaUpdateWithReplica. {code:java} long blkId = 7600037L; {code} 2. Maybe it is not necessary to set {{BASE_PATH}}, we could replace it with the following test {{basedir}}. {code:java} File basedir = new File(GenericTestUtils.getRandomizedTempPath()); {code} 3. Do not understand the following {{assert}}, some comments may be helpful. {code:java} + assertEquals(BASE_PATH + SEP + subdir1 + SEP + "subdir15", LocalReplica + .parseBaseDir(new File(BASE_PATH + SEP + subdir1 + SEP + "subdir15"), + blkId).baseDirPath); {code} I will give my +1 after update. Ping [~ayushtkn], do you mind to take another review? Thanks again. > The DirectoryScanner doesn't fix the wrongly placed replica. > ------------------------------------------------------------ > > Key: HDFS-14631 > URL: https://issues.apache.org/jira/browse/HDFS-14631 > Project: Hadoop HDFS > Issue Type: Bug > Reporter: Jinglun > Assignee: Jinglun > Priority: Major > Attachments: HDFS-14631.001.patch, HDFS-14631.002.patch > > > When DirectoryScanner scans block files, if the block refers to the block > file does not exist the DirectoryScanner will update the block based on the > replica file found on the disk. See FsDatasetImpl#checkAndUpdate. > > {code:java} > /* > * Block exists in volumeMap and the block file exists on the disk > */ > // Compare block files > if (memBlockInfo.blockDataExists()) { > ... > } else { > // Block refers to a block file that does not exist. > // Update the block with the file found on the disk. Since the block > // file and metadata file are found as a pair on the disk, update > // the block based on the metadata file found on the disk > LOG.warn("Block file in replica " > + memBlockInfo.getBlockURI() > + " does not exist. Updating it to the file found during scan " > + diskFile.getAbsolutePath()); > memBlockInfo.updateWithReplica( > StorageLocation.parse(diskFile.toString())); > LOG.warn("Updating generation stamp for block " + blockId > + " from " + memBlockInfo.getGenerationStamp() + " to " + diskGS); > memBlockInfo.setGenerationStamp(diskGS); > } > {code} > But the DirectoryScanner doesn't really fix it because in > LocalReplica#parseBaseDir() the 'subdir' are ignored. > > -- This message was sent by Atlassian JIRA (v7.6.14#76016) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org