[jira] [Assigned] (HDFS-3119) Overreplicated block is not deleted even after the replication factor is reduced after sync follwed by closing that file
[ https://issues.apache.org/jira/browse/HDFS-3119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Li reassigned HDFS-3119: Assignee: Brandon Li > Overreplicated block is not deleted even after the replication factor is > reduced after sync follwed by closing that file > > > Key: HDFS-3119 > URL: https://issues.apache.org/jira/browse/HDFS-3119 > Project: Hadoop HDFS > Issue Type: Bug > Components: name-node >Affects Versions: 0.24.0 >Reporter: J.Andreina >Assignee: Brandon Li >Priority: Minor > Fix For: 0.24.0, 0.23.2 > > > cluster setup: > -- > 1NN,2 DN,replication factor 2,block report interval 3sec ,block size-256MB > step1: write a file "filewrite.txt" of size 90bytes with sync(not closed) > step2: change the replication factor to 1 using the command: "./hdfs dfs > -setrep 1 /filewrite.txt" > step3: close the file > * At the NN side the file "Decreasing replication from 2 to 1 for > /filewrite.txt" , logs has occured but the overreplicated blocks are not > deleted even after the block report is sent from DN > * while listing the file in the console using "./hdfs dfs -ls " the > replication factor for that file is mentioned as 1 > * In fsck report for that files displays that the file is replicated to 2 > datanodes -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HDFS-2902) HA: Allow new (shared) edits log dir to be configured while NN is running
[ https://issues.apache.org/jira/browse/HDFS-2902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Li reassigned HDFS-2902: Assignee: Brandon Li (was: Bikas Saha) > HA: Allow new (shared) edits log dir to be configured while NN is running > - > > Key: HDFS-2902 > URL: https://issues.apache.org/jira/browse/HDFS-2902 > Project: Hadoop HDFS > Issue Type: New Feature > Components: ha, name-node >Affects Versions: 0.24.0 >Reporter: Bikas Saha >Assignee: Brandon Li > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HDFS-2937) TestDFSHAAdmin needs tests with MiniDFSCluster
[ https://issues.apache.org/jira/browse/HDFS-2937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Li reassigned HDFS-2937: Assignee: Brandon Li (was: Suresh Srinivas) > TestDFSHAAdmin needs tests with MiniDFSCluster > -- > > Key: HDFS-2937 > URL: https://issues.apache.org/jira/browse/HDFS-2937 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ha >Affects Versions: HA branch (HDFS-1623) >Reporter: Suresh Srinivas >Assignee: Brandon Li > > TestDFSHAAdmin currently works with Mockito base HAServiceProtocol. Tests are > needed with real namenodes using MiniDFSCluster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HDFS-2908) Add apache license header for StorageReport.java
[ https://issues.apache.org/jira/browse/HDFS-2908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Li reassigned HDFS-2908: Assignee: Brandon Li > Add apache license header for StorageReport.java > > > Key: HDFS-2908 > URL: https://issues.apache.org/jira/browse/HDFS-2908 > Project: Hadoop HDFS > Issue Type: Sub-task >Affects Versions: 0.24.0 >Reporter: Suresh Srinivas >Assignee: Brandon Li >Priority: Minor > Fix For: 0.24.0 > > > StorageReport.java added in HDFS-2899 is missing Apache license header. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HDFS-2830) HA: Improvements for SBN web UI
[ https://issues.apache.org/jira/browse/HDFS-2830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Li reassigned HDFS-2830: Assignee: (was: Brandon Li) The first item will be fixed by HDFS-2901. > HA: Improvements for SBN web UI > --- > > Key: HDFS-2830 > URL: https://issues.apache.org/jira/browse/HDFS-2830 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ha, name-node >Affects Versions: HA branch (HDFS-1623) >Reporter: Todd Lipcon > > A few improvements should be done to the NN web UI while it is in standby > mode: > - Since the SBN doesn't compute replication queues, we shouldn't show > under-replicated/missing blocks or corrupt files > - In an edits dir that is open for read only, we should probably mark it as > such > - We should include the latest txid reflected in the namespace, as well as > expose some of the HA-related metrics (eg queued block reports) > - We should include a link to the other NNs in the cluster, as well as the > NN's nameservice ID and namenode ID when available. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HDFS-2830) HA: Improvements for SBN web UI
[ https://issues.apache.org/jira/browse/HDFS-2830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Li reassigned HDFS-2830: Assignee: Brandon Li > HA: Improvements for SBN web UI > --- > > Key: HDFS-2830 > URL: https://issues.apache.org/jira/browse/HDFS-2830 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ha, name-node >Affects Versions: HA branch (HDFS-1623) >Reporter: Todd Lipcon >Assignee: Brandon Li > > A few improvements should be done to the NN web UI while it is in standby > mode: > - Since the SBN doesn't compute replication queues, we shouldn't show > under-replicated/missing blocks or corrupt files > - In an edits dir that is open for read only, we should probably mark it as > such > - We should include the latest txid reflected in the namespace, as well as > expose some of the HA-related metrics (eg queued block reports) > - We should include a link to the other NNs in the cluster, as well as the > NN's nameservice ID and namenode ID when available. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HDFS-2525) Race between BlockPoolSliceScanner and append
[ https://issues.apache.org/jira/browse/HDFS-2525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Li reassigned HDFS-2525: Assignee: Brandon Li > Race between BlockPoolSliceScanner and append > - > > Key: HDFS-2525 > URL: https://issues.apache.org/jira/browse/HDFS-2525 > Project: Hadoop HDFS > Issue Type: Bug > Components: data-node >Affects Versions: 0.23.0 >Reporter: Todd Lipcon >Assignee: Brandon Li >Priority: Critical > > I wrote a test which runs append() in a loop on a single file with a single > replica, appending 0~100 bytes each time. If this races with the > BlockPoolSliceScanner, I observe the BlockPoolSliceScanner getting FNFE, then > reporting the block as bad to the NN. This causes the writer thread to loop > forever on completeFile() since it doesn't see a valid replica. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HDFS-2811) HA: Client should fail if a failover occurs which switches block pool ID
[ https://issues.apache.org/jira/browse/HDFS-2811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Li reassigned HDFS-2811: Assignee: Brandon Li > HA: Client should fail if a failover occurs which switches block pool ID > > > Key: HDFS-2811 > URL: https://issues.apache.org/jira/browse/HDFS-2811 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ha, hdfs client, name-node >Affects Versions: HA branch (HDFS-1623) >Reporter: Aaron T. Myers >Assignee: Brandon Li > > Making sure that the client is talking to an NN with the same block pool ID > as the one it was previously talking to seems like a good sanity check. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HDFS-2805) HA: Add a test for a federated cluster with HA NNs
[ https://issues.apache.org/jira/browse/HDFS-2805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Li reassigned HDFS-2805: Assignee: Brandon Li > HA: Add a test for a federated cluster with HA NNs > -- > > Key: HDFS-2805 > URL: https://issues.apache.org/jira/browse/HDFS-2805 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ha, name-node >Affects Versions: HA branch (HDFS-1623) >Reporter: Aaron T. Myers >Assignee: Brandon Li > > Add a test for configuring/interacting with a federated cluster wherein each > name service is itself HA. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HDFS-2655) BlockReaderLocal#skip performs unnecessary IO
[ https://issues.apache.org/jira/browse/HDFS-2655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Li reassigned HDFS-2655: Assignee: Brandon Li > BlockReaderLocal#skip performs unnecessary IO > - > > Key: HDFS-2655 > URL: https://issues.apache.org/jira/browse/HDFS-2655 > Project: Hadoop HDFS > Issue Type: Improvement > Components: data-node >Affects Versions: 0.23.1 >Reporter: Eli Collins >Assignee: Brandon Li > > Per HDFS-2654 BlockReaderLocal#skip performs the skip by reading the data so > we stay in sync with checksums. This could be implemented more efficiently in > the future to skip to the beginning of the appropriate checksum chunk and > then only read to the middle of that chunk. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HDFS-2776) Missing interface annotation on JournalSet
[ https://issues.apache.org/jira/browse/HDFS-2776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Li reassigned HDFS-2776: Assignee: Brandon Li > Missing interface annotation on JournalSet > -- > > Key: HDFS-2776 > URL: https://issues.apache.org/jira/browse/HDFS-2776 > Project: Hadoop HDFS > Issue Type: Task > Components: name-node >Affects Versions: 0.24.0 >Reporter: Todd Lipcon >Assignee: Brandon Li >Priority: Trivial > Labels: newbie > > This public class is missing an annotation that it is for private usage. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
