[jira] [Created] (HDFS-16875) Erasure Coding: data access proxy to allow old clients to read EC data

2022-12-19 Thread Jing Zhao (Jira)
Jing Zhao created HDFS-16875: Summary: Erasure Coding: data access proxy to allow old clients to read EC data Key: HDFS-16875 URL: https://issues.apache.org/jira/browse/HDFS-16875 Project: Hadoop HDFS

[jira] [Created] (HDFS-16874) Improve DataNode decommission for Erasure Coding

2022-12-19 Thread Jing Zhao (Jira)
Jing Zhao created HDFS-16874: Summary: Improve DataNode decommission for Erasure Coding Key: HDFS-16874 URL: https://issues.apache.org/jira/browse/HDFS-16874 Project: Hadoop HDFS Issue Type

[jira] [Resolved] (HDFS-16268) Balancer stuck when moving striped blocks due to NPE

2021-10-13 Thread Jing Zhao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao resolved HDFS-16268. -- Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Balancer stuck when mov

[jira] [Resolved] (HDFS-10648) Expose Balancer metrics through Metrics2

2021-09-21 Thread Jing Zhao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-10648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao resolved HDFS-10648. -- Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Resolved Thanks for the work

[jira] [Resolved] (HDFS-16224) testBalancerWithObserverWithFailedNode times out

2021-09-17 Thread Jing Zhao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao resolved HDFS-16224. -- Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed I've committed th

[jira] [Resolved] (HDFS-15842) HDFS mover to emit metrics

2021-06-19 Thread Jing Zhao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao resolved HDFS-15842. -- Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed +1. Thank you for the

[jira] [Resolved] (HDFS-15781) Add metrics for how blocks are moved in replaceBlock

2021-02-23 Thread Jing Zhao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao resolved HDFS-15781. -- Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed I've committed the

[jira] [Resolved] (HDFS-15683) Allow configuring DISK/ARCHIVE capacity for individual volumes

2021-02-08 Thread Jing Zhao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao resolved HDFS-15683. -- Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Allow configuring D

[jira] [Resolved] (HDFS-15549) Use Hardlink to move replica between DISK and ARCHIVE storage if on same filesystem mount

2021-01-15 Thread Jing Zhao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao resolved HDFS-15549. -- Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed +1. Thank you for the

[jira] [Resolved] (HDFS-14904) Add Option to let Balancer prefer highly utilized nodes in each iteration

2020-12-02 Thread Jing Zhao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao resolved HDFS-14904. -- Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Add Option to let Balan

[jira] [Resolved] (HDFS-15548) Allow configuring DISK/ARCHIVE storage types on same device mount

2020-11-09 Thread Jing Zhao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao resolved HDFS-15548. -- Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed I've committed the c

[jira] [Reopened] (HDFS-8498) Blocks can be committed with wrong size

2017-02-16 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao reopened HDFS-8498: - Reopen for the branch-2 patch. > Blocks can be committed with wrong s

[jira] [Reopened] (HDFS-8498) Blocks can be committed with wrong size

2017-01-10 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao reopened HDFS-8498: - > Blocks can be committed with wrong size > --- > >

[jira] [Created] (HDFS-10968) BlockManager#isNewRack should consider decommissioning nodes

2016-10-05 Thread Jing Zhao (JIRA)
Jing Zhao created HDFS-10968: Summary: BlockManager#isNewRack should consider decommissioning nodes Key: HDFS-10968 URL: https://issues.apache.org/jira/browse/HDFS-10968 Project: Hadoop HDFS

[jira] [Created] (HDFS-10858) FBR processing may generate incorrect reportedBlock-blockGroup mapping

2016-09-12 Thread Jing Zhao (JIRA)
Jing Zhao created HDFS-10858: Summary: FBR processing may generate incorrect reportedBlock-blockGroup mapping Key: HDFS-10858 URL: https://issues.apache.org/jira/browse/HDFS-10858 Project: Hadoop HDFS

[jira] [Created] (HDFS-10688) BPServiceActor may run into a tight loop for sending block report when hitting IOException

2016-07-25 Thread Jing Zhao (JIRA)
Jing Zhao created HDFS-10688: Summary: BPServiceActor may run into a tight loop for sending block report when hitting IOException Key: HDFS-10688 URL: https://issues.apache.org/jira/browse/HDFS-10688

[jira] [Created] (HDFS-10508) DFSInputStream should set thread's interrupt status after catching InterruptException from sleep

2016-06-08 Thread Jing Zhao (JIRA)
Jing Zhao created HDFS-10508: Summary: DFSInputStream should set thread's interrupt status after catching InterruptException from sleep Key: HDFS-10508 URL: https://issues.apache.org/jira/browse/HDFS-

[jira] [Created] (HDFS-10419) Building HDFS on top of Ozone's storage containers

2016-05-17 Thread Jing Zhao (JIRA)
Jing Zhao created HDFS-10419: Summary: Building HDFS on top of Ozone's storage containers Key: HDFS-10419 URL: https://issues.apache.org/jira/browse/HDFS-10419 Project: Hadoop HDFS Issue

[jira] [Created] (HDFS-9876) shouldProcessOverReplicated should not count number of pending replicas

2016-02-29 Thread Jing Zhao (JIRA)
Jing Zhao created HDFS-9876: --- Summary: shouldProcessOverReplicated should not count number of pending replicas Key: HDFS-9876 URL: https://issues.apache.org/jira/browse/HDFS-9876 Project: Hadoop HDFS

[jira] [Created] (HDFS-9866) BlockManager#chooseExcessReplicasStriped may weaken rack fault tolerance

2016-02-26 Thread Jing Zhao (JIRA)
Jing Zhao created HDFS-9866: --- Summary: BlockManager#chooseExcessReplicasStriped may weaken rack fault tolerance Key: HDFS-9866 URL: https://issues.apache.org/jira/browse/HDFS-9866 Project: Hadoop HDFS

[jira] [Created] (HDFS-9837) BlockManager#countNodes should be able to detect duplicated internal blocks

2016-02-19 Thread Jing Zhao (JIRA)
Jing Zhao created HDFS-9837: --- Summary: BlockManager#countNodes should be able to detect duplicated internal blocks Key: HDFS-9837 URL: https://issues.apache.org/jira/browse/HDFS-9837 Project: Hadoop HDFS

[jira] [Created] (HDFS-9818) Correctly handle EC reconstruction work caused by not enough racks

2016-02-16 Thread Jing Zhao (JIRA)
Jing Zhao created HDFS-9818: --- Summary: Correctly handle EC reconstruction work caused by not enough racks Key: HDFS-9818 URL: https://issues.apache.org/jira/browse/HDFS-9818 Project: Hadoop HDFS

[jira] [Created] (HDFS-9794) Streamer threads may leak if failure happens when closing the striped outputstream

2016-02-11 Thread Jing Zhao (JIRA)
Jing Zhao created HDFS-9794: --- Summary: Streamer threads may leak if failure happens when closing the striped outputstream Key: HDFS-9794 URL: https://issues.apache.org/jira/browse/HDFS-9794 Project: Hadoop

[jira] [Created] (HDFS-9789) Correctly update DataNode's scheduled block size when writing small EC file

2016-02-10 Thread Jing Zhao (JIRA)
Jing Zhao created HDFS-9789: --- Summary: Correctly update DataNode's scheduled block size when writing small EC file Key: HDFS-9789 URL: https://issues.apache.org/jira/browse/HDFS-9789 Project: Hadoop

[jira] [Created] (HDFS-9754) Avoid unnecessary getBlockCollection calls in BlockManager

2016-02-03 Thread Jing Zhao (JIRA)
Jing Zhao created HDFS-9754: --- Summary: Avoid unnecessary getBlockCollection calls in BlockManager Key: HDFS-9754 URL: https://issues.apache.org/jira/browse/HDFS-9754 Project: Hadoop HDFS Issue

[jira] [Created] (HDFS-9677) Rename generationStampV1/generationStampV2 to legacyGenerationStamp/generationStamp

2016-01-21 Thread Jing Zhao (JIRA)
Jing Zhao created HDFS-9677: --- Summary: Rename generationStampV1/generationStampV2 to legacyGenerationStamp/generationStamp Key: HDFS-9677 URL: https://issues.apache.org/jira/browse/HDFS-9677 Project

[jira] [Created] (HDFS-9646) ErasureCodingWorker may fail when recovering data blocks with length less than the first internal block

2016-01-13 Thread Jing Zhao (JIRA)
Jing Zhao created HDFS-9646: --- Summary: ErasureCodingWorker may fail when recovering data blocks with length less than the first internal block Key: HDFS-9646 URL: https://issues.apache.org/jira/browse/HDFS-9646

[jira] [Created] (HDFS-9621) {{getListing}} wrongly associates Erasure Coding policy to pre-existing replicated files under an EC directory

2016-01-06 Thread Jing Zhao (JIRA)
Jing Zhao created HDFS-9621: --- Summary: {{getListing}} wrongly associates Erasure Coding policy to pre-existing replicated files under an EC directory Key: HDFS-9621 URL: https://issues.apache.org/jira/browse/HDFS

[jira] [Created] (HDFS-9575) Use byte array for internal block indices in a striped block

2015-12-18 Thread Jing Zhao (JIRA)
Jing Zhao created HDFS-9575: --- Summary: Use byte array for internal block indices in a striped block Key: HDFS-9575 URL: https://issues.apache.org/jira/browse/HDFS-9575 Project: Hadoop HDFS Issue

Re: [Important] Lots of 2.8.0 commits are in branch-2 only

2015-12-16 Thread Jing Zhao
Thanks for reporting the issue, Wangda. Recreating a new branch-2.8 based on the current branch-2 makes sense to me. Thanks, -Jing On Wed, Dec 16, 2015 at 5:38 PM, Wangda Tan wrote: > Hi folks, > > I found there're lots of commits are in branch-2 only. > Ran "git log branch-2.8..branch-2". > >

[jira] [Created] (HDFS-9542) Move BlockIdManager from FSNamesystem to BlockManager

2015-12-10 Thread Jing Zhao (JIRA)
Jing Zhao created HDFS-9542: --- Summary: Move BlockIdManager from FSNamesystem to BlockManager Key: HDFS-9542 URL: https://issues.apache.org/jira/browse/HDFS-9542 Project: Hadoop HDFS Issue Type

[jira] [Created] (HDFS-9535) Fix TestReplication#testNoExtraReplicationWhenBlockReceivedIsLate

2015-12-09 Thread Jing Zhao (JIRA)
Jing Zhao created HDFS-9535: --- Summary: Fix TestReplication#testNoExtraReplicationWhenBlockReceivedIsLate Key: HDFS-9535 URL: https://issues.apache.org/jira/browse/HDFS-9535 Project: Hadoop HDFS

[jira] [Created] (HDFS-9371) Code cleanup for DatanodeManager

2015-11-03 Thread Jing Zhao (JIRA)
Jing Zhao created HDFS-9371: --- Summary: Code cleanup for DatanodeManager Key: HDFS-9371 URL: https://issues.apache.org/jira/browse/HDFS-9371 Project: Hadoop HDFS Issue Type: Sub-task

Re: Erasure coding in branch-2 [Was Re: [VOTE] Merge HDFS-7285 (erasure coding) branch to trunk]

2015-11-02 Thread Jing Zhao
trunk. I'll update on this thread when the merge is > > >>> done. > > >>>>>>>> > > >>>>>>>> --- > > >>>>>>>> Zhe Zhang > > >>>>>>>> > > >>>>>>>> O

Re: Erasure coding in branch-2 [Was Re: [VOTE] Merge HDFS-7285 (erasure coding) branch to trunk]

2015-11-02 Thread Jing Zhao
ange it to binding.) > > >>>>>>>> > > >>>>>>>> +1 > > >>>>>>>> I have been involved in the development and code review on the > > >>>>>> feature > > >>>>>>>

[jira] [Created] (HDFS-9319) Make DatanodeInfo thread safe

2015-10-27 Thread Jing Zhao (JIRA)
Jing Zhao created HDFS-9319: --- Summary: Make DatanodeInfo thread safe Key: HDFS-9319 URL: https://issues.apache.org/jira/browse/HDFS-9319 Project: Hadoop HDFS Issue Type: Sub-task

Re: [VOTE] Release Apache Hadoop 2.6.2

2015-10-26 Thread Jing Zhao
+1 (binding) Download the tarball and run some HDFS tests and MR examples in a pseudo-distributed cluster. Thanks, -Jing On Mon, Oct 26, 2015 at 2:06 PM, Vinod Vavilapalli wrote: > I was helping Sangjin offline with the release. > > We briefly discussed the KEYS problem before, but it missed m

[jira] [Created] (HDFS-9223) Code cleanup for DatanodeDescriptor and HeartbeatManager

2015-10-09 Thread Jing Zhao (JIRA)
Jing Zhao created HDFS-9223: --- Summary: Code cleanup for DatanodeDescriptor and HeartbeatManager Key: HDFS-9223 URL: https://issues.apache.org/jira/browse/HDFS-9223 Project: Hadoop HDFS Issue Type

[jira] [Created] (HDFS-9204) DatanodeDescriptor#PendingReplicationWithoutTargets is wrongly calculated

2015-10-06 Thread Jing Zhao (JIRA)
Jing Zhao created HDFS-9204: --- Summary: DatanodeDescriptor#PendingReplicationWithoutTargets is wrongly calculated Key: HDFS-9204 URL: https://issues.apache.org/jira/browse/HDFS-9204 Project: Hadoop HDFS

[jira] [Created] (HDFS-9180) Update excluded DataNodes in DFSStripedOutputStream based on failures in data streamers

2015-09-30 Thread Jing Zhao (JIRA)
Jing Zhao created HDFS-9180: --- Summary: Update excluded DataNodes in DFSStripedOutputStream based on failures in data streamers Key: HDFS-9180 URL: https://issues.apache.org/jira/browse/HDFS-9180 Project

[jira] [Created] (HDFS-9145) Tracking methods that hold FSNamesytemLock for too long

2015-09-25 Thread Jing Zhao (JIRA)
Jing Zhao created HDFS-9145: --- Summary: Tracking methods that hold FSNamesytemLock for too long Key: HDFS-9145 URL: https://issues.apache.org/jira/browse/HDFS-9145 Project: Hadoop HDFS Issue Type

Re: [VOTE] Merge HDFS-7285 (erasure coding) branch to trunk

2015-09-23 Thread Jing Zhao
S, snapshot, append, truncate, hflush, hsync, > >and so forth. > > > >Development of this feature has been a collaboration across many > >companies and institutions. I'd like to thank J. Andreina, Takanobu > >Asanuma, Vinayakumar B, Li Bo, Takuya Fukudome, Uma Maheswar

[jira] [Created] (HDFS-9063) Correctly handle snapshot path for getContentSummary

2015-09-11 Thread Jing Zhao (JIRA)
Jing Zhao created HDFS-9063: --- Summary: Correctly handle snapshot path for getContentSummary Key: HDFS-9063 URL: https://issues.apache.org/jira/browse/HDFS-9063 Project: Hadoop HDFS Issue Type: Bug

[jira] [Created] (HDFS-9011) Support splitting BlockReport of a storage into multiple RPC

2015-09-02 Thread Jing Zhao (JIRA)
Jing Zhao created HDFS-9011: --- Summary: Support splitting BlockReport of a storage into multiple RPC Key: HDFS-9011 URL: https://issues.apache.org/jira/browse/HDFS-9011 Project: Hadoop HDFS Issue

[jira] [Created] (HDFS-8980) Remove unnecessary block replacement in INodeFile

2015-08-27 Thread Jing Zhao (JIRA)
Jing Zhao created HDFS-8980: --- Summary: Remove unnecessary block replacement in INodeFile Key: HDFS-8980 URL: https://issues.apache.org/jira/browse/HDFS-8980 Project: Hadoop HDFS Issue Type: Sub

[jira] [Created] (HDFS-8930) Block report lease may leak if the 2nd full block report comes when NN is still in safemode

2015-08-20 Thread Jing Zhao (JIRA)
Jing Zhao created HDFS-8930: --- Summary: Block report lease may leak if the 2nd full block report comes when NN is still in safemode Key: HDFS-8930 URL: https://issues.apache.org/jira/browse/HDFS-8930

[jira] [Resolved] (HDFS-8877) Allow bypassing some minor exceptions while loading editlog

2015-08-07 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao resolved HDFS-8877. - Resolution: Not A Problem > Allow bypassing some minor exceptions while loading edit

[jira] [Created] (HDFS-8877) Allow bypassing some minor exceptions while loading editlog

2015-08-07 Thread Jing Zhao (JIRA)
Jing Zhao created HDFS-8877: --- Summary: Allow bypassing some minor exceptions while loading editlog Key: HDFS-8877 URL: https://issues.apache.org/jira/browse/HDFS-8877 Project: Hadoop HDFS Issue

[jira] [Reopened] (HDFS-8623) Refactor NameNode handling of invalid, corrupt, and under-recovery blocks

2015-08-06 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao reopened HDFS-8623: - > Refactor NameNode handling of invalid, corrupt, and under-recovery blo

[jira] [Resolved] (HDFS-8623) Refactor NameNode handling of invalid, corrupt, and under-recovery blocks

2015-08-06 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao resolved HDFS-8623. - Resolution: Later Fix Version/s: (was: 2.8.0) The patch was reverted from trunk and branch-2

[jira] [Reopened] (HDFS-8499) Refactor BlockInfo class hierarchy with static helper class

2015-08-05 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao reopened HDFS-8499: - > Refactor BlockInfo class hierarchy with static helper cl

[jira] [Resolved] (HDFS-8804) Erasure Coding: use DirectBufferPool in DFSStripedInputStream for buffer allocation

2015-08-03 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao resolved HDFS-8804. - Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: HDFS-7285 I've committed this t

[jira] [Created] (HDFS-8804) Erasure Coding: use DirectBufferPool in DFSStripedInputStream for buffer allocation

2015-07-21 Thread Jing Zhao (JIRA)
Jing Zhao created HDFS-8804: --- Summary: Erasure Coding: use DirectBufferPool in DFSStripedInputStream for buffer allocation Key: HDFS-8804 URL: https://issues.apache.org/jira/browse/HDFS-8804 Project

[jira] [Created] (HDFS-8797) WebHdfsFileSystem creates too many connections for pread

2015-07-17 Thread Jing Zhao (JIRA)
Jing Zhao created HDFS-8797: --- Summary: WebHdfsFileSystem creates too many connections for pread Key: HDFS-8797 URL: https://issues.apache.org/jira/browse/HDFS-8797 Project: Hadoop HDFS Issue Type

[jira] [Created] (HDFS-8777) Erasure Coding: add tests for taking snapshots on EC files

2015-07-14 Thread Jing Zhao (JIRA)
Jing Zhao created HDFS-8777: --- Summary: Erasure Coding: add tests for taking snapshots on EC files Key: HDFS-8777 URL: https://issues.apache.org/jira/browse/HDFS-8777 Project: Hadoop HDFS Issue

[jira] [Resolved] (HDFS-8652) Track BlockInfo instead of Block in CorruptReplicasMap

2015-07-13 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao resolved HDFS-8652. - Resolution: Invalid Hadoop Flags: (was: Reviewed) > Track BlockInfo instead of Block

[jira] [Reopened] (HDFS-8652) Track BlockInfo instead of Block in CorruptReplicasMap

2015-07-13 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao reopened HDFS-8652: - > Track BlockInfo instead of Block in CorruptReplicas

[jira] [Created] (HDFS-8760) Erasure Coding: reuse BlockReader when reading the same block in pread

2015-07-10 Thread Jing Zhao (JIRA)
Jing Zhao created HDFS-8760: --- Summary: Erasure Coding: reuse BlockReader when reading the same block in pread Key: HDFS-8760 URL: https://issues.apache.org/jira/browse/HDFS-8760 Project: Hadoop HDFS

[jira] [Resolved] (HDFS-8484) Erasure coding: Two contiguous blocks occupy IDs belong to same striped group

2015-07-09 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao resolved HDFS-8484. - Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: HDFS-7285 I've committed this t

[jira] [Created] (HDFS-8669) Erasure Coding: handle missing internal block locations in DFSStripedInputStream

2015-06-25 Thread Jing Zhao (JIRA)
Jing Zhao created HDFS-8669: --- Summary: Erasure Coding: handle missing internal block locations in DFSStripedInputStream Key: HDFS-8669 URL: https://issues.apache.org/jira/browse/HDFS-8669 Project: Hadoop

[jira] [Created] (HDFS-8652) Track BlockInfo instead of Block in CorruptReplicasMap

2015-06-23 Thread Jing Zhao (JIRA)
Jing Zhao created HDFS-8652: --- Summary: Track BlockInfo instead of Block in CorruptReplicasMap Key: HDFS-8652 URL: https://issues.apache.org/jira/browse/HDFS-8652 Project: Hadoop HDFS Issue Type

[jira] [Resolved] (HDFS-8602) Erasure Coding: Client can't read(decode) the EC files which have corrupt blocks.

2015-06-19 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao resolved HDFS-8602. - Resolution: Fixed Hadoop Flags: Reviewed Thanks for the confirmation and updating the patch

[jira] [Resolved] (HDFS-8254) Avoid assigning a leading streamer in StripedDataStreamer to tolerate datanode failure

2015-06-19 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao resolved HDFS-8254. - Resolution: Fixed Fix Version/s: HDFS-7285 Hadoop Flags: Reviewed +1 as well. I've comm

[jira] [Created] (HDFS-8619) Erasure Coding: revisit replica counting for striped blocks

2015-06-17 Thread Jing Zhao (JIRA)
Jing Zhao created HDFS-8619: --- Summary: Erasure Coding: revisit replica counting for striped blocks Key: HDFS-8619 URL: https://issues.apache.org/jira/browse/HDFS-8619 Project: Hadoop HDFS Issue

[jira] [Created] (HDFS-8559) Erasure Coding: fix non-protobuf fsimage for striped blocks

2015-06-08 Thread Jing Zhao (JIRA)
Jing Zhao created HDFS-8559: --- Summary: Erasure Coding: fix non-protobuf fsimage for striped blocks Key: HDFS-8559 URL: https://issues.apache.org/jira/browse/HDFS-8559 Project: Hadoop HDFS Issue

[jira] [Resolved] (HDFS-8323) Bump GenerationStamp for write faliure in DFSStripedOutputStream

2015-05-19 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao resolved HDFS-8323. - Resolution: Fixed Fix Version/s: HDFS-7285 Hadoop Flags: Reviewed +1 on the latest patch. I&#x

[jira] [Resolved] (HDFS-4942) Add retry cache support in Namenode

2015-05-19 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao resolved HDFS-4942. - Resolution: Fixed Fix Version/s: 2.1.0-beta > Add retry cache support in Namen

[jira] [Resolved] (HDFS-8372) Erasure coding: compute storage type quotas for striped files, to be consistent with HDFS-8327

2015-05-12 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao resolved HDFS-8372. - Resolution: Fixed > Erasure coding: compute storage type quotas for striped files, to be > cons

[jira] [Reopened] (HDFS-8372) Erasure coding: compute storage type quotas for striped files, to be consistent with HDFS-8327

2015-05-12 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao reopened HDFS-8372: - > Erasure coding: compute storage type quotas for striped files, to be > consistent with HDF

[jira] [Resolved] (HDFS-8355) Erasure Coding: Refactor BlockInfo and BlockInfoUnderConstruction

2015-05-08 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao resolved HDFS-8355. - Resolution: Fixed Fix Version/s: HDFS-7285 Hadoop Flags: Reviewed I've committed

[jira] [Resolved] (HDFS-8289) Erasure Coding: add ECSchema to HdfsFileStatus

2015-05-07 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao resolved HDFS-8289. - Resolution: Fixed Fix Version/s: HDFS-7285 Hadoop Flags: Reviewed I've committed this t

[jira] [Resolved] (HDFS-8203) Erasure Coding: Seek and other Ops in DFSStripedInputStream.

2015-05-07 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao resolved HDFS-8203. - Resolution: Fixed Fix Version/s: HDFS-7285 Hadoop Flags: Reviewed I've committed this t

[jira] [Created] (HDFS-8319) Erasure Coding: support decoding for stateful read

2015-05-04 Thread Jing Zhao (JIRA)
Jing Zhao created HDFS-8319: --- Summary: Erasure Coding: support decoding for stateful read Key: HDFS-8319 URL: https://issues.apache.org/jira/browse/HDFS-8319 Project: Hadoop HDFS Issue Type: Sub

[jira] [Resolved] (HDFS-8281) Erasure Coding: implement parallel stateful reading for striped layout

2015-05-04 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao resolved HDFS-8281. - Resolution: Fixed Fix Version/s: HDFS-7285 Thanks again for the review, Zhe and Nicholas! I&#x

[jira] [Resolved] (HDFS-8316) Erasure coding: refactor EC constants to be consistent with HDFS-8249

2015-05-04 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao resolved HDFS-8316. - Resolution: Fixed Fix Version/s: HDFS-7285 Hadoop Flags: Reviewed I've committed this t

[jira] [Resolved] (HDFS-8308) Erasure Coding: NameNode may get blocked in waitForLoadingFSImage() when loading editlog

2015-04-30 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao resolved HDFS-8308. - Resolution: Fixed Fix Version/s: HDFS-7285 Thanks for the review, Nicholas! I've committed

[jira] [Created] (HDFS-8308) Erasure Coding: NameNode may get blocked in waitForLoadingFSImage() when loading editlog

2015-04-30 Thread Jing Zhao (JIRA)
Jing Zhao created HDFS-8308: --- Summary: Erasure Coding: NameNode may get blocked in waitForLoadingFSImage() when loading editlog Key: HDFS-8308 URL: https://issues.apache.org/jira/browse/HDFS-8308 Project

[jira] [Created] (HDFS-8300) Fix unit test failures and findbugs warning caused by HDFS-8283

2015-04-30 Thread Jing Zhao (JIRA)
Jing Zhao created HDFS-8300: --- Summary: Fix unit test failures and findbugs warning caused by HDFS-8283 Key: HDFS-8300 URL: https://issues.apache.org/jira/browse/HDFS-8300 Project: Hadoop HDFS

[jira] [Created] (HDFS-8293) Erasure Coding: test the retry logic of DFSStripedInputStream

2015-04-29 Thread Jing Zhao (JIRA)
Jing Zhao created HDFS-8293: --- Summary: Erasure Coding: test the retry logic of DFSStripedInputStream Key: HDFS-8293 URL: https://issues.apache.org/jira/browse/HDFS-8293 Project: Hadoop HDFS Issue

[jira] [Created] (HDFS-8281) Erasure Coding: implement parallel stateful reading for striped layout

2015-04-28 Thread Jing Zhao (JIRA)
Jing Zhao created HDFS-8281: --- Summary: Erasure Coding: implement parallel stateful reading for striped layout Key: HDFS-8281 URL: https://issues.apache.org/jira/browse/HDFS-8281 Project: Hadoop HDFS

[jira] [Created] (HDFS-8280) Code Cleanup in DFSInputStream

2015-04-28 Thread Jing Zhao (JIRA)
Jing Zhao created HDFS-8280: --- Summary: Code Cleanup in DFSInputStream Key: HDFS-8280 URL: https://issues.apache.org/jira/browse/HDFS-8280 Project: Hadoop HDFS Issue Type: Improvement

[jira] [Resolved] (HDFS-8235) Erasure Coding: Create DFSStripedInputStream in DFSClient#open

2015-04-28 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao resolved HDFS-8235. - Resolution: Fixed Fix Version/s: HDFS-7285 Hadoop Flags: Reviewed The .3 patch looks good to

[jira] [Created] (HDFS-8273) logSync() is called inside of write lock for delete op

2015-04-27 Thread Jing Zhao (JIRA)
Jing Zhao created HDFS-8273: --- Summary: logSync() is called inside of write lock for delete op Key: HDFS-8273 URL: https://issues.apache.org/jira/browse/HDFS-8273 Project: Hadoop HDFS Issue Type

[jira] [Created] (HDFS-8272) Erasure Coding: simplify the retry logic in DFSStripedInputStream

2015-04-27 Thread Jing Zhao (JIRA)
Jing Zhao created HDFS-8272: --- Summary: Erasure Coding: simplify the retry logic in DFSStripedInputStream Key: HDFS-8272 URL: https://issues.apache.org/jira/browse/HDFS-8272 Project: Hadoop HDFS

[jira] [Resolved] (HDFS-8223) Should calculate checksum for parity blocks in DFSStripedOutputStream

2015-04-23 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao resolved HDFS-8223. - Resolution: Fixed Fix Version/s: HDFS-7285 Hadoop Flags: Reviewed I've committed this t

[jira] [Resolved] (HDFS-8233) Fix DFSStripedOutputStream#getCurrentBlockGroupBytes when the last stripe is at the block group boundary

2015-04-23 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao resolved HDFS-8233. - Resolution: Fixed Fix Version/s: HDFS-7285 Thanks for the review, Nicholas! I've committed

[jira] [Created] (HDFS-8235) Create DFSStripedInputStream in DFSClient#open

2015-04-23 Thread Jing Zhao (JIRA)
Jing Zhao created HDFS-8235: --- Summary: Create DFSStripedInputStream in DFSClient#open Key: HDFS-8235 URL: https://issues.apache.org/jira/browse/HDFS-8235 Project: Hadoop HDFS Issue Type: Sub-task

[jira] [Created] (HDFS-8233) Fix DFSStripedOutputStream#getCurrentBlockGroupBytes when the last stripe is at the block group boundary

2015-04-23 Thread Jing Zhao (JIRA)
Jing Zhao created HDFS-8233: --- Summary: Fix DFSStripedOutputStream#getCurrentBlockGroupBytes when the last stripe is at the block group boundary Key: HDFS-8233 URL: https://issues.apache.org/jira/browse/HDFS-8233

[jira] [Resolved] (HDFS-8145) Fix the editlog corruption exposed by failed TestAddStripedBlocks

2015-04-17 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao resolved HDFS-8145. - Resolution: Fixed Fix Version/s: HDFS-7285 I've committed this to the feature branch. Thank

[jira] [Resolved] (HDFS-7937) Erasure Coding: INodeFile quota computation unit tests

2015-04-17 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao resolved HDFS-7937. - Resolution: Fixed Fix Version/s: HDFS-7285 Hadoop Flags: Reviewed I've committed this t

[jira] [Resolved] (HDFS-8166) DFSStripedOutputStream should not create empty blocks

2015-04-17 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao resolved HDFS-8166. - Resolution: Fixed Fix Version/s: HDFS-7285 Thanks Nicholas for the review! I've committed th

[jira] [Reopened] (HDFS-8166) DFSStripedOutputStream should not create empty blocks

2015-04-17 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao reopened HDFS-8166: - Assignee: Jing Zhao (was: Tsz Wo Nicholas Sze) Reopen the jira. We will separate out the fix for

[jira] [Resolved] (HDFS-8166) DFSStripedOutputStream should not create empty blocks

2015-04-16 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao resolved HDFS-8166. - Resolution: Duplicate I confirmed with Nicholas. Closed this as dup. > DFSStripedOutputStream should

[jira] [Resolved] (HDFS-7952) On starting Standby with "rollback" option, lastPromisedEpoch gets updated and Active Namenode is shutting down.

2015-04-15 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao resolved HDFS-7952. - Resolution: Duplicate Resolve as duplicate after HDFS-7934 got committed. > On starting Standby w

[jira] [Resolved] (HDFS-8120) Erasure coding: created util class to analyze striped block groups

2015-04-15 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao resolved HDFS-8120. - Resolution: Fixed Fix Version/s: HDFS-7285 Hadoop Flags: Reviewed I've committed

[jira] [Created] (HDFS-8151) Always use snapshot path as source when invalid snapshot names are used for diff based distcp

2015-04-15 Thread Jing Zhao (JIRA)
Jing Zhao created HDFS-8151: --- Summary: Always use snapshot path as source when invalid snapshot names are used for diff based distcp Key: HDFS-8151 URL: https://issues.apache.org/jira/browse/HDFS-8151

[jira] [Created] (HDFS-8145) Fix the editlog corruption exposed by failed TestAddStripedBlocks

2015-04-14 Thread Jing Zhao (JIRA)
Jing Zhao created HDFS-8145: --- Summary: Fix the editlog corruption exposed by failed TestAddStripedBlocks Key: HDFS-8145 URL: https://issues.apache.org/jira/browse/HDFS-8145 Project: Hadoop HDFS

Re: [VOTE] Release Apache Hadoop 2.7.0 RC0

2015-04-14 Thread Jing Zhao
+1 (binding) downloaded the source tar ball, built it, and ran some examples and distcp job. Thanks, -Jing On Tue, Apr 14, 2015 at 11:06 AM, Robert Kanter wrote: > +1 (non-binding) > > + verified checksum > + Deployed binary tarball, ran some examples, clicked around in the Web UIs > > thanks

[jira] [Resolved] (HDFS-8106) Fix test failure in TestAddStripedBlocks

2015-04-08 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao resolved HDFS-8106. - Resolution: Duplicate > Fix test failure in TestAddStripedBlo

[jira] [Resolved] (HDFS-8107) Fix stripping related test failures in TestFSImage

2015-04-08 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao resolved HDFS-8107. - Resolution: Duplicate duplicated with HDFS-8077. Besides, we can leave the unit test fix till the end

  1   2   3   4   5   >