[jira] [Assigned] (HDFS-3119) Overreplicated block is not deleted even after the replication factor is reduced after sync follwed by closing that file
[ https://issues.apache.org/jira/browse/HDFS-3119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Li reassigned HDFS-3119: Assignee: Brandon Li Overreplicated block is not deleted even after the replication factor is reduced after sync follwed by closing that file Key: HDFS-3119 URL: https://issues.apache.org/jira/browse/HDFS-3119 Project: Hadoop HDFS Issue Type: Bug Components: name-node Affects Versions: 0.24.0 Reporter: J.Andreina Assignee: Brandon Li Priority: Minor Fix For: 0.24.0, 0.23.2 cluster setup: -- 1NN,2 DN,replication factor 2,block report interval 3sec ,block size-256MB step1: write a file filewrite.txt of size 90bytes with sync(not closed) step2: change the replication factor to 1 using the command: ./hdfs dfs -setrep 1 /filewrite.txt step3: close the file * At the NN side the file Decreasing replication from 2 to 1 for /filewrite.txt , logs has occured but the overreplicated blocks are not deleted even after the block report is sent from DN * while listing the file in the console using ./hdfs dfs -ls the replication factor for that file is mentioned as 1 * In fsck report for that files displays that the file is replicated to 2 datanodes -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HDFS-2902) HA: Allow new (shared) edits log dir to be configured while NN is running
[ https://issues.apache.org/jira/browse/HDFS-2902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Li reassigned HDFS-2902: Assignee: Brandon Li (was: Bikas Saha) HA: Allow new (shared) edits log dir to be configured while NN is running - Key: HDFS-2902 URL: https://issues.apache.org/jira/browse/HDFS-2902 Project: Hadoop HDFS Issue Type: New Feature Components: ha, name-node Affects Versions: 0.24.0 Reporter: Bikas Saha Assignee: Brandon Li -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HDFS-2937) TestDFSHAAdmin needs tests with MiniDFSCluster
[ https://issues.apache.org/jira/browse/HDFS-2937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Li reassigned HDFS-2937: Assignee: Brandon Li (was: Suresh Srinivas) TestDFSHAAdmin needs tests with MiniDFSCluster -- Key: HDFS-2937 URL: https://issues.apache.org/jira/browse/HDFS-2937 Project: Hadoop HDFS Issue Type: Sub-task Components: ha Affects Versions: HA branch (HDFS-1623) Reporter: Suresh Srinivas Assignee: Brandon Li TestDFSHAAdmin currently works with Mockito base HAServiceProtocol. Tests are needed with real namenodes using MiniDFSCluster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HDFS-2908) Add apache license header for StorageReport.java
[ https://issues.apache.org/jira/browse/HDFS-2908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Li reassigned HDFS-2908: Assignee: Brandon Li Add apache license header for StorageReport.java Key: HDFS-2908 URL: https://issues.apache.org/jira/browse/HDFS-2908 Project: Hadoop HDFS Issue Type: Sub-task Affects Versions: 0.24.0 Reporter: Suresh Srinivas Assignee: Brandon Li Priority: Minor Fix For: 0.24.0 StorageReport.java added in HDFS-2899 is missing Apache license header. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HDFS-2830) HA: Improvements for SBN web UI
[ https://issues.apache.org/jira/browse/HDFS-2830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Li reassigned HDFS-2830: Assignee: (was: Brandon Li) The first item will be fixed by HDFS-2901. HA: Improvements for SBN web UI --- Key: HDFS-2830 URL: https://issues.apache.org/jira/browse/HDFS-2830 Project: Hadoop HDFS Issue Type: Sub-task Components: ha, name-node Affects Versions: HA branch (HDFS-1623) Reporter: Todd Lipcon A few improvements should be done to the NN web UI while it is in standby mode: - Since the SBN doesn't compute replication queues, we shouldn't show under-replicated/missing blocks or corrupt files - In an edits dir that is open for read only, we should probably mark it as such - We should include the latest txid reflected in the namespace, as well as expose some of the HA-related metrics (eg queued block reports) - We should include a link to the other NNs in the cluster, as well as the NN's nameservice ID and namenode ID when available. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HDFS-2525) Race between BlockPoolSliceScanner and append
[ https://issues.apache.org/jira/browse/HDFS-2525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Li reassigned HDFS-2525: Assignee: Brandon Li Race between BlockPoolSliceScanner and append - Key: HDFS-2525 URL: https://issues.apache.org/jira/browse/HDFS-2525 Project: Hadoop HDFS Issue Type: Bug Components: data-node Affects Versions: 0.23.0 Reporter: Todd Lipcon Assignee: Brandon Li Priority: Critical I wrote a test which runs append() in a loop on a single file with a single replica, appending 0~100 bytes each time. If this races with the BlockPoolSliceScanner, I observe the BlockPoolSliceScanner getting FNFE, then reporting the block as bad to the NN. This causes the writer thread to loop forever on completeFile() since it doesn't see a valid replica. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HDFS-2830) HA: Improvements for SBN web UI
[ https://issues.apache.org/jira/browse/HDFS-2830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Li reassigned HDFS-2830: Assignee: Brandon Li HA: Improvements for SBN web UI --- Key: HDFS-2830 URL: https://issues.apache.org/jira/browse/HDFS-2830 Project: Hadoop HDFS Issue Type: Sub-task Components: ha, name-node Affects Versions: HA branch (HDFS-1623) Reporter: Todd Lipcon Assignee: Brandon Li A few improvements should be done to the NN web UI while it is in standby mode: - Since the SBN doesn't compute replication queues, we shouldn't show under-replicated/missing blocks or corrupt files - In an edits dir that is open for read only, we should probably mark it as such - We should include the latest txid reflected in the namespace, as well as expose some of the HA-related metrics (eg queued block reports) - We should include a link to the other NNs in the cluster, as well as the NN's nameservice ID and namenode ID when available. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HDFS-2811) HA: Client should fail if a failover occurs which switches block pool ID
[ https://issues.apache.org/jira/browse/HDFS-2811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Li reassigned HDFS-2811: Assignee: Brandon Li HA: Client should fail if a failover occurs which switches block pool ID Key: HDFS-2811 URL: https://issues.apache.org/jira/browse/HDFS-2811 Project: Hadoop HDFS Issue Type: Sub-task Components: ha, hdfs client, name-node Affects Versions: HA branch (HDFS-1623) Reporter: Aaron T. Myers Assignee: Brandon Li Making sure that the client is talking to an NN with the same block pool ID as the one it was previously talking to seems like a good sanity check. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HDFS-2805) HA: Add a test for a federated cluster with HA NNs
[ https://issues.apache.org/jira/browse/HDFS-2805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Li reassigned HDFS-2805: Assignee: Brandon Li HA: Add a test for a federated cluster with HA NNs -- Key: HDFS-2805 URL: https://issues.apache.org/jira/browse/HDFS-2805 Project: Hadoop HDFS Issue Type: Sub-task Components: ha, name-node Affects Versions: HA branch (HDFS-1623) Reporter: Aaron T. Myers Assignee: Brandon Li Add a test for configuring/interacting with a federated cluster wherein each name service is itself HA. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HDFS-2655) BlockReaderLocal#skip performs unnecessary IO
[ https://issues.apache.org/jira/browse/HDFS-2655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Li reassigned HDFS-2655: Assignee: Brandon Li BlockReaderLocal#skip performs unnecessary IO - Key: HDFS-2655 URL: https://issues.apache.org/jira/browse/HDFS-2655 Project: Hadoop HDFS Issue Type: Improvement Components: data-node Affects Versions: 0.23.1 Reporter: Eli Collins Assignee: Brandon Li Per HDFS-2654 BlockReaderLocal#skip performs the skip by reading the data so we stay in sync with checksums. This could be implemented more efficiently in the future to skip to the beginning of the appropriate checksum chunk and then only read to the middle of that chunk. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HDFS-2776) Missing interface annotation on JournalSet
[ https://issues.apache.org/jira/browse/HDFS-2776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Li reassigned HDFS-2776: Assignee: Brandon Li Missing interface annotation on JournalSet -- Key: HDFS-2776 URL: https://issues.apache.org/jira/browse/HDFS-2776 Project: Hadoop HDFS Issue Type: Task Components: name-node Affects Versions: 0.24.0 Reporter: Todd Lipcon Assignee: Brandon Li Priority: Trivial Labels: newbie This public class is missing an annotation that it is for private usage. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
