[jira] [Created] (HDFS-3370) HDFS hardlink

Hairong Kuang created HDFS-3370: --- Summary: HDFS hardlink Key: HDFS-3370 URL: https://issues.apache.org/jira/browse/HDFS-3370 Project: Hadoop HDFS Issue Type: New Feature Reporter:

[jira] [Created] (HDFS-2362) More Improvements on NameNode Scalability

More Improvements on NameNode Scalability - Key: HDFS-2362 URL: https://issues.apache.org/jira/browse/HDFS-2362 Project: Hadoop HDFS Issue Type: Improvement Components: name-node

[jira] [Resolved] (HDFS-1904) Secondary Namenode dies when a mkdir on a non-existent parent directory is run

[ https://issues.apache.org/jira/browse/HDFS-1904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang resolved HDFS-1904. - Resolution: Duplicate Secondary Namenode dies when a mkdir on a non-existent parent directory is

[jira] [Created] (HDFS-1826) NameNode should save image to name directories in parallel during upgrade

NameNode should save image to name directories in parallel during upgrade - Key: HDFS-1826 URL: https://issues.apache.org/jira/browse/HDFS-1826 Project: Hadoop HDFS

[jira] Created: (HDFS-1765) Block Replication should respect under-replication block priority

Block Replication should respect under-replication block priority - Key: HDFS-1765 URL: https://issues.apache.org/jira/browse/HDFS-1765 Project: Hadoop HDFS Issue Type:

[jira] Created: (HDFS-1766) Datanode is marked dead, but datanode process is alive and verifying blocks

Datanode is marked dead, but datanode process is alive and verifying blocks --- Key: HDFS-1766 URL: https://issues.apache.org/jira/browse/HDFS-1766 Project: Hadoop HDFS

[jira] Created: (HDFS-1658) A less expensive way to figure out directory size

A less expensive way to figure out directory size - Key: HDFS-1658 URL: https://issues.apache.org/jira/browse/HDFS-1658 Project: Hadoop HDFS Issue Type: Improvement Reporter:

[jira] Created: (HDFS-1627) Fix NullPointerException in Secondary NameNode

Fix NullPointerException in Secondary NameNode -- Key: HDFS-1627 URL: https://issues.apache.org/jira/browse/HDFS-1627 Project: Hadoop HDFS Issue Type: Bug Components: name-node

[jira] Created: (HDFS-1622) Update TestDFSUpgradeFromImage to test batch hardlink improvment

Update TestDFSUpgradeFromImage to test batch hardlink improvment Key: HDFS-1622 URL: https://issues.apache.org/jira/browse/HDFS-1622 Project: Hadoop HDFS Issue Type:

[jira] Resolved: (HDFS-1335) HDFS side of HADOOP-6904: first step towards inter-version communications between dfs client and NameNode

[ https://issues.apache.org/jira/browse/HDFS-1335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang resolved HDFS-1335. - Resolution: Fixed Fix Version/s: 0.23.0 Hadoop Flags: [Reviewed] HDFS side of

[jira] Created: (HDFS-1578) First step towards data transter protocol compatibility: support DatanodeProtocol#getDataTransferProtocolVersion

First step towards data transter protocol compatibility: support DatanodeProtocol#getDataTransferProtocolVersion Key: HDFS-1578 URL:

[jira] Created: (HDFS-1577) Fall back to a random datanode when bestNode fails

Fall back to a random datanode when bestNode fails -- Key: HDFS-1577 URL: https://issues.apache.org/jira/browse/HDFS-1577 Project: Hadoop HDFS Issue Type: Bug Components: name-node

[jira] Resolved: (HDFS-1555) HDFS 20 append: Disallow pipeline recovery if a file is already being lease recovered

[ https://issues.apache.org/jira/browse/HDFS-1555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang resolved HDFS-1555. - Resolution: Fixed Hadoop Flags: [Reviewed] I just committed this! HDFS 20 append:

[jira] Resolved: (HDFS-110) Failed to execute fsck with -move option

[ https://issues.apache.org/jira/browse/HDFS-110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang resolved HDFS-110. Resolution: Won't Fix Failed to execute fsck with -move option

[jira] Resolved: (HDFS-937) Port HDFS-101 to branch 0.20

[ https://issues.apache.org/jira/browse/HDFS-937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang resolved HDFS-937. Resolution: Not A Problem Port HDFS-101 to branch 0.20

[jira] Created: (HDFS-1553) Hftp file read should retry a different datanode if the chosen best datanode fails to connect to NameNode

Hftp file read should retry a different datanode if the chosen best datanode fails to connect to NameNode - Key: HDFS-1553 URL:

[jira] Created: (HDFS-1554) New semantics for recoverLease

New semantics for recoverLease -- Key: HDFS-1554 URL: https://issues.apache.org/jira/browse/HDFS-1554 Project: Hadoop HDFS Issue Type: Improvement Reporter: Hairong Kuang Assignee: Hairong

[jira] Created: (HDFS-1555) HDFS 20 append: Disallow pipeline recovery if a file is already being lease recovered

HDFS 20 append: Disallow pipeline recovery if a file is already being lease recovered - Key: HDFS-1555 URL: https://issues.apache.org/jira/browse/HDFS-1555 Project:

[jira] Created: (HDFS-1550) NPE when listing a file with no location

NPE when listing a file with no location Key: HDFS-1550 URL: https://issues.apache.org/jira/browse/HDFS-1550 Project: Hadoop HDFS Issue Type: Bug Components: hdfs client Affects

[jira] Created: (HDFS-1541) Not marking datanodes dead When namenode in safemode

Not marking datanodes dead When namenode in safemode Key: HDFS-1541 URL: https://issues.apache.org/jira/browse/HDFS-1541 Project: Hadoop HDFS Issue Type: Improvement Components:

[jira] Created: (HDFS-1537) Add a metrics for tracking the number of reported corrupt replicas

Add a metrics for tracking the number of reported corrupt replicas -- Key: HDFS-1537 URL: https://issues.apache.org/jira/browse/HDFS-1537 Project: Hadoop HDFS Issue Type:

[jira] Resolved: (HDFS-1482) Add listCorruptFileBlocks to DistributedFileSystem (and ClientProtocol)

[ https://issues.apache.org/jira/browse/HDFS-1482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang resolved HDFS-1482. - Resolution: Fixed Release Note: This patch bumps the version number of ClientProtocol to

[jira] Resolved: (HDFS-1467) Append pipeline never succeeds with more than one replica

[ https://issues.apache.org/jira/browse/HDFS-1467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang resolved HDFS-1467. - Resolution: Fixed Hadoop Flags: [Reviewed] I just committed this. Thanks Todd! Append

[jira] Resolved: (HDFS-1483) DFSClient.getBlockLocations returns BlockLocations with no indication that the corresponding blocks are corrupt

[ https://issues.apache.org/jira/browse/HDFS-1483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang resolved HDFS-1483. - Resolution: Fixed Fix Version/s: 0.22.0 Hadoop Flags: [Reviewed] I've just committed

[jira] Resolved: (HDFS-903) NN should verify images and edit logs on startup

[ https://issues.apache.org/jira/browse/HDFS-903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang resolved HDFS-903. Resolution: Fixed Release Note: Store fsimage MD5 checksum in VERSION file. Validate checksum

[jira] Resolved: (HDFS-40) Unit test org.apache.hadoop.dfs.TestBalancer fails on Solaris with a timeout

[ https://issues.apache.org/jira/browse/HDFS-40?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang resolved HDFS-40. --- Resolution: Won't Fix Unit test org.apache.hadoop.dfs.TestBalancer fails on Solaris with a timeout

[jira] Resolved: (HDFS-290) Additional performance improvement to chooseTarget

[ https://issues.apache.org/jira/browse/HDFS-290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang resolved HDFS-290. Resolution: Later Additional performance improvement to chooseTarget

[jira] Created: (HDFS-1481) NameNode should validate fsimage before rolling

NameNode should validate fsimage before rolling --- Key: HDFS-1481 URL: https://issues.apache.org/jira/browse/HDFS-1481 Project: Hadoop HDFS Issue Type: Improvement Components: name-node

[jira] Created: (HDFS-1436) LeaseRenew RPC does not need to grab fsnamesystem write lock

LeaseRenew RPC does not need to grab fsnamesystem write lock Key: HDFS-1436 URL: https://issues.apache.org/jira/browse/HDFS-1436 Project: Hadoop HDFS Issue Type: Improvement

[jira] Created: (HDFS-1429) Make lease expiration limit configurable

Make lease expiration limit configurable Key: HDFS-1429 URL: https://issues.apache.org/jira/browse/HDFS-1429 Project: Hadoop HDFS Issue Type: Improvement Components: name-node Affects

[jira] Created: (HDFS-1425) Improve NameNode responsiveness by speeding up serving refreshNodes RPC

Improve NameNode responsiveness by speeding up serving refreshNodes RPC --- Key: HDFS-1425 URL: https://issues.apache.org/jira/browse/HDFS-1425 Project: Hadoop HDFS Issue

[jira] Resolved: (HDFS-1346) DFSClient receives out of order packet ack

[ https://issues.apache.org/jira/browse/HDFS-1346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang resolved HDFS-1346. - Hadoop Flags: [Reviewed] Resolution: Fixed I've committed this. DFSClient receives out of

[jira] Created: (HDFS-1368) Add a block counter to DatanodeDescriptor

Add a block counter to DatanodeDescriptor - Key: HDFS-1368 URL: https://issues.apache.org/jira/browse/HDFS-1368 Project: Hadoop HDFS Issue Type: Improvement Components: name-node

[jira] Resolved: (HDFS-86) Corrupted blocks get deleted but not replicated

[ https://issues.apache.org/jira/browse/HDFS-86?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang resolved HDFS-86. --- Resolution: Invalid Corrupted blocks get deleted but not replicated

[jira] Created: (HDFS-1348) DecommissionManager holds fsnamesystem lock during the whole process of checking if decomissioning DataNodes are finished or not

DecommissionManager holds fsnamesystem lock during the whole process of checking if decomissioning DataNodes are finished or not Key: HDFS-1348

[jira] Created: (HDFS-1346) DFSClient receives out of order packet ack

DFSClient receives out of order packet ack -- Key: HDFS-1346 URL: https://issues.apache.org/jira/browse/HDFS-1346 Project: Hadoop HDFS Issue Type: Bug Components: data-node, hdfs client

[jira] Created: (HDFS-1335) HDFS side of HADOOP-6904: first step towards inter-version communications between dfs client and NameNode

HDFS side of HADOOP-6904: first step towards inter-version communications between dfs client and NameNode - Key: HDFS-1335 URL:

[jira] Created: (HDFS-1329) A baby step towards inter-version communications between dfs client and NameNode

A baby step towards inter-version communications between dfs client and NameNode Key: HDFS-1329 URL: https://issues.apache.org/jira/browse/HDFS-1329 Project: Hadoop HDFS

[jira] Created: (HDFS-1165) createSymlink should not hold the fsnamesytem lock when sync its edit log to disk

createSymlink should not hold the fsnamesytem lock when sync its edit log to disk - Key: HDFS-1165 URL: https://issues.apache.org/jira/browse/HDFS-1165 Project: Hadoop

[jira] Created: (HDFS-1104) Fsck triggers full GC on NameNode

Fsck triggers full GC on NameNode - Key: HDFS-1104 URL: https://issues.apache.org/jira/browse/HDFS-1104 Project: Hadoop HDFS Issue Type: Bug Reporter: Hairong Kuang Fix For: 0.20.3

[jira] Resolved: (HDFS-522) Create stress tests for append feature

[ https://issues.apache.org/jira/browse/HDFS-522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang resolved HDFS-522. Resolution: Duplicate Create stress tests for append feature

[jira] Resolved: (HDFS-265) Revisit append

[ https://issues.apache.org/jira/browse/HDFS-265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang resolved HDFS-265. Resolution: Fixed A few recent updates to the jira: 1. Block recovery tests have been committed to

[jira] Resolved: (HDFS-686) NullPointerException is thrown while merging edit log and image

[ https://issues.apache.org/jira/browse/HDFS-686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang resolved HDFS-686. Resolution: Fixed NullPointerException is thrown while merging edit log and image

[jira] Created: (HDFS-985) HDFS should issue multiple RPCs for listing a large directory

HDFS should issue multiple RPCs for listing a large directory - Key: HDFS-985 URL: https://issues.apache.org/jira/browse/HDFS-985 Project: Hadoop HDFS Issue Type: New Feature

[jira] Created: (HDFS-946) NameNode should not return full path name when lisitng a diretory or getting the status of a file

NameNode should not return full path name when lisitng a diretory or getting the status of a file - Key: HDFS-946 URL: https://issues.apache.org/jira/browse/HDFS-946

[jira] Resolved: (HDFS-872) DFSClient 0.20.1 is incompatible with HDFS 0.20.2

[ https://issues.apache.org/jira/browse/HDFS-872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang resolved HDFS-872. Resolution: Fixed Hadoop Flags: [Reviewed] I resolve this for now. If we ever want to port

[jira] Reopened: (HDFS-520) Create new tests for block recovery

[ https://issues.apache.org/jira/browse/HDFS-520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang reopened HDFS-520: I am reopening this. Block recovery leads to the truncation of data thus has the risk of losing data.

[jira] Resolved: (HDFS-849) TestFiDataTransferProtocol2#pipeline_Fi_18 sometimes fails

[ https://issues.apache.org/jira/browse/HDFS-849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang resolved HDFS-849. Resolution: Fixed I've committed this! TestFiDataTransferProtocol2#pipeline_Fi_18 sometimes fails

[jira] Resolved: (HDFS-483) Data transfer (aka pipeline) implementation cannot tolerate exceptions

[ https://issues.apache.org/jira/browse/HDFS-483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang resolved HDFS-483. Resolution: Fixed Fix Version/s: 0.22.0 0.21.0 Data transfer (aka

[jira] Created: (HDFS-849) .TestFiDataTransferProtocol2#pipeline_Fi_18 sometimes fails

.TestFiDataTransferProtocol2#pipeline_Fi_18 sometimes fails --- Key: HDFS-849 URL: https://issues.apache.org/jira/browse/HDFS-849 Project: Hadoop HDFS Issue Type: Bug

[jira] Created: (HDFS-800) The last block of a file under construction may change to the COMPLETE state in response to getAdditionalBlock or completeFileInternal

The last block of a file under construction may change to the COMPLETE state in response to getAdditionalBlock or completeFileInternal -- Key:

[jira] Resolved: (HDFS-731) Support new Syncable interface in HDFS

[ https://issues.apache.org/jira/browse/HDFS-731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang resolved HDFS-731. Resolution: Fixed Hadoop Flags: [Reviewed] I've just committed this. Support new Syncable

[jira] Created: (HDFS-734) TestDatanodeBlockScanner times out in branch 0.21

TestDatanodeBlockScanner times out in branch 0.21 - Key: HDFS-734 URL: https://issues.apache.org/jira/browse/HDFS-734 Project: Hadoop HDFS Issue Type: Bug Components: test

[jira] Created: (HDFS-728) Creat a comprehensive functional test for append

Creat a comprehensive functional test for append Key: HDFS-728 URL: https://issues.apache.org/jira/browse/HDFS-728 Project: Hadoop HDFS Issue Type: New Feature Affects Versions: 0.21.0

[jira] Created: (HDFS-723) Deadlock in DFSClient#DFSOutputStream

Deadlock in DFSClient#DFSOutputStream - Key: HDFS-723 URL: https://issues.apache.org/jira/browse/HDFS-723 Project: Hadoop HDFS Issue Type: Bug Reporter: Hairong Kuang Assignee:

[jira] Created: (HDFS-694) Add a test to make sure that node decomission does not get blocked by underreplicated blocks in an unclosed file

Add a test to make sure that node decomission does not get blocked by underreplicated blocks in an unclosed file Key: HDFS-694 URL:

[jira] Created: (HDFS-690) TestAppend2#testComplexAppend failed on Too many open files

TestAppend2#testComplexAppend failed on Too many open files - Key: HDFS-690 URL: https://issues.apache.org/jira/browse/HDFS-690 Project: Hadoop HDFS Issue Type: Bug

[jira] Created: (HDFS-686) NullPointerException is thrown while merging edit log and image

NullPointerException is thrown while merging edit log and image --- Key: HDFS-686 URL: https://issues.apache.org/jira/browse/HDFS-686 Project: Hadoop HDFS Issue Type: Bug

[jira] Resolved: (HDFS-637) DataNode sends a Success ack when block write fails

[ https://issues.apache.org/jira/browse/HDFS-637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang resolved HDFS-637. Resolution: Fixed I've just committed this. DataNode sends a Success ack when block write fails

[jira] Created: (HDFS-637) DataNode sends an Success ack when block write fails

DataNode sends an Success ack when block write fails Key: HDFS-637 URL: https://issues.apache.org/jira/browse/HDFS-637 Project: Hadoop HDFS Issue Type: Bug Components: data-node

[jira] Created: (HDFS-624) Client support pipeline recovery

Client support pipeline recovery Key: HDFS-624 URL: https://issues.apache.org/jira/browse/HDFS-624 Project: Hadoop HDFS Issue Type: Sub-task Affects Versions: Append Branch Reporter: Hairong

[jira] Created: (HDFS-609) Create a file with the append flag does not work in HDFS

Create a file with the append flag does not work in HDFS Key: HDFS-609 URL: https://issues.apache.org/jira/browse/HDFS-609 Project: Hadoop HDFS Issue Type: Bug Affects Versions:

[jira] Created: (HDFS-586) TestBlocksWithNotEnoughRacks fails

TestBlocksWithNotEnoughRacks fails -- Key: HDFS-586 URL: https://issues.apache.org/jira/browse/HDFS-586 Project: Hadoop HDFS Issue Type: Bug Components: test Affects Versions: 0.21.0

[jira] Resolved: (HDFS-537) DataNode uses ReplicaBeingWritten to support dfs writes/hflush

[ https://issues.apache.org/jira/browse/HDFS-537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang resolved HDFS-537. Resolution: Fixed Hadoop Flags: [Reviewed] I've committed this! DataNode uses

[jira] Created: (HDFS-553) BlockSender reports wrong failed position in ChecksumException

BlockSender reports wrong failed position in ChecksumException -- Key: HDFS-553 URL: https://issues.apache.org/jira/browse/HDFS-553 Project: Hadoop HDFS Issue Type: Bug

[jira] Created: (HDFS-550) DataNode restarts may introduce corrupt/duplicated/lost replicas when handling detached replicas

DataNode restarts may introduce corrupt/duplicated/lost replicas when handling detached replicas Key: HDFS-550 URL: https://issues.apache.org/jira/browse/HDFS-550

[jira] Created: (HDFS-544) Add a rbw sub directory to DataNode data directory

Add a rbw sub directory to DataNode data directory Key: HDFS-544 URL: https://issues.apache.org/jira/browse/HDFS-544 Project: Hadoop HDFS Issue Type: Sub-task Components:

[jira] Created: (HDFS-510) Rename DatanodeBlockInfo to be ReplicaInfo

Rename DatanodeBlockInfo to be ReplicaInfo -- Key: HDFS-510 URL: https://issues.apache.org/jira/browse/HDFS-510 Project: Hadoop HDFS Issue Type: Improvement Components: data-node Affects