[jira] [Updated] (HDFS-11287) Storage class member storageDirs should be private to avoid unprotected access by derived classes

2017-01-18 Thread Lei (Eddy) Xu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lei (Eddy) Xu updated HDFS-11287: - Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-alpha2

[jira] [Commented] (HDFS-7344) [umbrella] Erasure Coding worker and support in DataNode

2017-01-18 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15829448#comment-15829448 ] Zhe Zhang commented on HDFS-7344: - Thanks for taking care of this [~andrew.wang]. I think HDFS-8014 and

[jira] [Resolved] (HDFS-8015) Erasure Coding: local and remote block writer for coding work in DataNode

2017-01-18 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang resolved HDFS-8015. - Resolution: Duplicate Resolving this issue since HDFS-9719 already introduced {{StripedBlockWriter}} >

[jira] [Resolved] (HDFS-8014) Erasure Coding: local and remote block reader for coding work in DataNode

2017-01-18 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang resolved HDFS-8014. - Resolution: Duplicate Resolving the issue as HDFS-9719 already creates {{StripedBlockReader}} > Erasure

[jira] [Updated] (HDFS-11259) Update fsck to display maintenance state info

2017-01-18 Thread Manoj Govindassamy (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manoj Govindassamy updated HDFS-11259: -- Attachment: HDFS-11259.04.patch HDFS-11259-branch-2.03.patch Thanks for

[jira] [Comment Edited] (HDFS-11268) Correctly reconstruct erasure coding file from FSImage

2017-01-18 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15829394#comment-15829394 ] SammiChen edited comment on HDFS-11268 at 1/19/17 6:31 AM: --- Hi [~andrew.wang],

[jira] [Updated] (HDFS-11268) Correctly reconstruct erasure coding file from FSImage

2017-01-18 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-11268: - Attachment: HDFS-11268-002.patch Hi [~andrew.wang]], thanks for reviewing the patch! I rebased it. >

[jira] [Commented] (HDFS-11316) TestDataNodeVolumeFailure#testUnderReplicationAfterVolFailure fails in trunk

2017-01-18 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15829302#comment-15829302 ] Hudson commented on HDFS-11316: --- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11139 (See

[jira] [Comment Edited] (HDFS-11316) TestDataNodeVolumeFailure#testUnderReplicationAfterVolFailure fails in trunk

2017-01-18 Thread Yiqun Lin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15829284#comment-15829284 ] Yiqun Lin edited comment on HDFS-11316 at 1/19/17 4:10 AM: --- The failure tests

[jira] [Updated] (HDFS-11316) TestDataNodeVolumeFailure#testUnderReplicationAfterVolFailure fails in trunk

2017-01-18 Thread Yiqun Lin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin updated HDFS-11316: - Resolution: Fixed Status: Resolved (was: Patch Available) Committed to trunk. Thanks

[jira] [Updated] (HDFS-11316) TestDataNodeVolumeFailure#testUnderReplicationAfterVolFailure fails in trunk

2017-01-18 Thread Yiqun Lin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin updated HDFS-11316: - Hadoop Flags: Reviewed Fix Version/s: 3.0.0-alpha2 >

[jira] [Updated] (HDFS-11316) TestDataNodeVolumeFailure#testUnderReplicationAfterVolFailure fails in trunk

2017-01-18 Thread Yiqun Lin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin updated HDFS-11316: - Summary: TestDataNodeVolumeFailure#testUnderReplicationAfterVolFailure fails in trunk (was:

[jira] [Commented] (HDFS-11316) TestDataNodeVolumeFailure fails frequently

2017-01-18 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15829266#comment-15829266 ] Hadoop QA commented on HDFS-11316: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-11259) Update fsck to display maintenance state info

2017-01-18 Thread Lei (Eddy) Xu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15829263#comment-15829263 ] Lei (Eddy) Xu commented on HDFS-11259: -- Hi, [~manojg]. LGTM overall. One small nit: You might want

[jira] [Resolved] (HDFS-8125) Erasure Coding: Expose refreshECSchemas command to reload predefined schemas

2017-01-18 Thread Rakesh R (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R resolved HDFS-8125. Resolution: Not A Problem Agreed, the way we are plugging EC policy has been changed and its hard coded

[jira] [Commented] (HDFS-11309) [SPS]: chooseTargetTypeInSameNode should pass accurate block size to chooseStorage4Block while choosing target

2017-01-18 Thread Uma Maheswara Rao G (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15829211#comment-15829211 ] Uma Maheswara Rao G commented on HDFS-11309:

[jira] [Assigned] (HDFS-10999) Use more generic "low redundancy" blocks instead of "under replicated" blocks

2017-01-18 Thread Manoj Govindassamy (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manoj Govindassamy reassigned HDFS-10999: - Assignee: Manoj Govindassamy > Use more generic "low redundancy" blocks instead

[jira] [Assigned] (HDFS-10983) OIV tool should make an EC file explicit

2017-01-18 Thread Manoj Govindassamy (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manoj Govindassamy reassigned HDFS-10983: - Assignee: Manoj Govindassamy (was: Lei (Eddy) Xu) > OIV tool should make an EC

[jira] [Commented] (HDFS-11167) IBM java can not start secure datanode with HDFS_DATANODE_SECURE_EXTRA_OPTS "-jvm server"

2017-01-18 Thread Pan Yuxuan (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15829157#comment-15829157 ] Pan Yuxuan commented on HDFS-11167: --- [~brahmareddy] [~ajisakaa] Could you please help to review this

[jira] [Updated] (HDFS-11316) TestDataNodeVolumeFailure fails frequently

2017-01-18 Thread Yiqun Lin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin updated HDFS-11316: - Attachment: HDFS-11316.002.patch I think maybe there is other test method in

[jira] [Updated] (HDFS-11316) TestDataNodeVolumeFailure fails frequently

2017-01-18 Thread Yiqun Lin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin updated HDFS-11316: - Attachment: (was: HDFS-11316.002.patch) > TestDataNodeVolumeFailure fails frequently >

[jira] [Commented] (HDFS-8125) Erasure Coding: Expose refreshECSchemas command to reload predefined schemas

2017-01-18 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15829069#comment-15829069 ] Andrew Wang commented on HDFS-8125: --- Is this JIRA related to pluggable EC policies? Right now I believe

[jira] [Commented] (HDFS-11314) Validate client-provided EC schema on the NameNode

2017-01-18 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15829063#comment-15829063 ] Andrew Wang commented on HDFS-11314: Sorry I missed this comment, JIRA has been really unhappy over

[jira] [Updated] (HDFS-10461) Erasure Coding: Optimize block checksum recalculation logic on the fly by reconstructing multiple missed blocks at a time

2017-01-18 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-10461: --- Parent Issue: HDFS-8425 (was: HDFS-8031) > Erasure Coding: Optimize block checksum recalculation

[jira] [Commented] (HDFS-8138) Refine or extend EC command to support all cases

2017-01-18 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15829030#comment-15829030 ] Andrew Wang commented on HDFS-8138: --- I'm not quite sure what the other cases are for the DN command, but

[jira] [Updated] (HDFS-8138) Refine or extend EC command to support all cases

2017-01-18 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-8138: -- Parent Issue: HDFS-11347 (was: HDFS-8031) > Refine or extend EC command to support all cases >

[jira] [Commented] (HDFS-8957) Consolidate client striping input stream codes for stateful read and positional read

2017-01-18 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15829023#comment-15829023 ] Hadoop QA commented on HDFS-8957: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-11259) Update fsck to display maintenance state info

2017-01-18 Thread Manoj Govindassamy (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15829020#comment-15829020 ] Manoj Govindassamy commented on HDFS-11259: --- I don't see the failed tests related to the patch

[jira] [Updated] (HDFS-8957) Consolidate client striping input stream codes for stateful read and positional read

2017-01-18 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-8957: -- Issue Type: Improvement (was: Sub-task) Parent: (was: HDFS-8031) > Consolidate client

[jira] [Updated] (HDFS-8907) Configurable striping read buffer threhold

2017-01-18 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-8907: -- Component/s: erasure-coding > Configurable striping read buffer threhold >

[jira] [Updated] (HDFS-8907) Configurable striping read buffer threhold

2017-01-18 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-8907: -- Issue Type: Improvement (was: Sub-task) Parent: (was: HDFS-8031) > Configurable striping

[jira] [Updated] (HDFS-8667) Erasure Coding: refine striped block recovery logic and reuse in client and Datanode

2017-01-18 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-8667: -- Issue Type: Improvement (was: Sub-task) Parent: (was: HDFS-8031) > Erasure Coding: refine

[jira] [Updated] (HDFS-8667) Erasure Coding: refine striped block recovery logic and reuse in client and Datanode

2017-01-18 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-8667: -- Component/s: erasure-coding > Erasure Coding: refine striped block recovery logic and reuse in client

[jira] [Updated] (HDFS-11348) [Umbrella] Append and truncate for striped EC files

2017-01-18 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-11348: --- Description: Umbrella JIRA for adding append and truncate support to striped erasure coded files.

[jira] [Commented] (HDFS-11259) Update fsck to display maintenance state info

2017-01-18 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15828989#comment-15828989 ] Hadoop QA commented on HDFS-11259: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (HDFS-11259) Update fsck to display maintenance state info

2017-01-18 Thread Manoj Govindassamy (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manoj Govindassamy updated HDFS-11259: -- Hadoop Flags: (was: Incompatible change) Removing the 'incompatible changes' flag as

[jira] [Commented] (HDFS-8020) Erasure Coding: constructing and mapping BlockGroup info to utilize and call ErasureCoder API

2017-01-18 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15828973#comment-15828973 ] Hadoop QA commented on HDFS-8020: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-9079) Erasure coding: preallocate multiple generation stamps and serialize updates from data streamers

2017-01-18 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15828977#comment-15828977 ] Hadoop QA commented on HDFS-9079: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-8501) Erasure Coding: BlockInfoStriped.removeStorage(..) move storage with same blockIndex to front slot

2017-01-18 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15828975#comment-15828975 ] Hadoop QA commented on HDFS-8501: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-8287) DFSStripedOutputStream.writeChunk should not wait for writing parity

2017-01-18 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15828972#comment-15828972 ] Hadoop QA commented on HDFS-8287: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-8935) Erasure Coding: createErasureCodingZone api should accept the policyname as argument instead of ErasureCodingPolicy

2017-01-18 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15828976#comment-15828976 ] Hadoop QA commented on HDFS-8935: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-11239) [SPS]: Check Mover file ID lease also to determine whether Mover is running

2017-01-18 Thread Uma Maheswara Rao G (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15828970#comment-15828970 ] Uma Maheswara Rao G commented on HDFS-11239: Hi [~zhouwei], would you be able to revise the

[jira] [Commented] (HDFS-8065) Erasure coding: Support truncate at striped group boundary

2017-01-18 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15828955#comment-15828955 ] Hadoop QA commented on HDFS-8065: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (HDFS-9079) Erasure coding: preallocate multiple generation stamps and serialize updates from data streamers

2017-01-18 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-9079: -- Issue Type: Improvement (was: Sub-task) Parent: (was: HDFS-8031) > Erasure coding:

[jira] [Updated] (HDFS-9424) Erasure Coding: enhance the ErasureCodeBenchmarkThroughput tool

2017-01-18 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-9424: -- Assignee: (was: Rui Li) Issue Type: Improvement (was: Sub-task) Parent: (was:

[jira] [Updated] (HDFS-8287) DFSStripedOutputStream.writeChunk should not wait for writing parity

2017-01-18 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-8287: -- Parent Issue: HDFS-8425 (was: HDFS-8031) > DFSStripedOutputStream.writeChunk should not wait for

[jira] [Updated] (HDFS-8501) Erasure Coding: BlockInfoStriped.removeStorage(..) move storage with same blockIndex to front slot

2017-01-18 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-8501: -- Parent Issue: HDFS-8425 (was: HDFS-8031) > Erasure Coding: BlockInfoStriped.removeStorage(..) move

[jira] [Updated] (HDFS-7613) Block placement policy for erasure coding groups

2017-01-18 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-7613: -- Assignee: (was: Walter Su) Issue Type: Improvement (was: Sub-task) Parent: (was:

[jira] [Updated] (HDFS-7613) Block placement policy for erasure coding groups

2017-01-18 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-7613: -- Component/s: erasure-coding > Block placement policy for erasure coding groups >

[jira] [Updated] (HDFS-7613) Block placement policy for erasure coding groups

2017-01-18 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-7613: -- Target Version/s: (was: HDFS-7285) > Block placement policy for erasure coding groups >

[jira] [Commented] (HDFS-7344) [umbrella] Erasure Coding worker and support in DataNode

2017-01-18 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15828945#comment-15828945 ] Andrew Wang commented on HDFS-7344: --- [~zhz] I could use some help triaging this JIRA. I'd like to

[jira] [Updated] (HDFS-8935) Erasure Coding: createErasureCodingZone api should accept the policyname as argument instead of ErasureCodingPolicy

2017-01-18 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-8935: -- Parent Issue: HDFS-7337 (was: HDFS-8031) > Erasure Coding: createErasureCodingZone api should accept

[jira] [Updated] (HDFS-8020) Erasure Coding: constructing and mapping BlockGroup info to utilize and call ErasureCoder API

2017-01-18 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-8020: -- Parent Issue: HDFS-7337 (was: HDFS-8031) > Erasure Coding: constructing and mapping BlockGroup info to

[jira] [Updated] (HDFS-8019) Erasure Coding: erasure coding chunk buffer allocation and management

2017-01-18 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-8019: -- Parent Issue: HDFS-8425 (was: HDFS-8031) > Erasure Coding: erasure coding chunk buffer allocation and

[jira] [Updated] (HDFS-8415) Erasure coding: rotated parity placement (RAID5/6)

2017-01-18 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-8415: -- Component/s: erasure-coding > Erasure coding: rotated parity placement (RAID5/6) >

[jira] [Updated] (HDFS-8415) Erasure coding: rotated parity placement (RAID5/6)

2017-01-18 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-8415: -- Issue Type: New Feature (was: Sub-task) Parent: (was: HDFS-8031) > Erasure coding: rotated

[jira] [Updated] (HDFS-7346) Erasure Coding: perform stripping erasure encoding work given block reader and writer

2017-01-18 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-7346: -- Target Version/s: (was: HDFS-7285) > Erasure Coding: perform stripping erasure encoding work given

[jira] [Updated] (HDFS-7346) Erasure Coding: perform stripping erasure encoding work given block reader and writer

2017-01-18 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-7346: -- Parent Issue: HDFS-7337 (was: HDFS-8031) > Erasure Coding: perform stripping erasure encoding work

[jira] [Updated] (HDFS-7363) Pluggable algorithms to form block groups in erasure coding

2017-01-18 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-7363: -- Parent Issue: HDFS-7337 (was: HDFS-8031) > Pluggable algorithms to form block groups in erasure coding

[jira] [Updated] (HDFS-7363) Pluggable algorithms to form block groups in erasure coding

2017-01-18 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-7363: -- Target Version/s: (was: HDFS-7285) > Pluggable algorithms to form block groups in erasure coding >

[jira] [Updated] (HDFS-7337) Configurable and pluggable Erasure Codec and schema

2017-01-18 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-7337: -- Issue Type: New Feature (was: Sub-task) Parent: (was: HDFS-8031) > Configurable and

[jira] [Commented] (HDFS-8799) Erasure Coding: add tests for namenode processing corrupt striped blocks

2017-01-18 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15828923#comment-15828923 ] Hadoop QA commented on HDFS-8799: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-9098) Erasure coding: emulate race conditions among striped streamers in write pipeline

2017-01-18 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15828925#comment-15828925 ] Hadoop QA commented on HDFS-9098: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (HDFS-7786) Handle slow writers for DFSStripedOutputStream

2017-01-18 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-7786: -- Parent Issue: HDFS-8425 (was: HDFS-8031) > Handle slow writers for DFSStripedOutputStream >

[jira] [Updated] (HDFS-8032) Erasure coding: shrink size of indices in BlockInfoStriped

2017-01-18 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-8032: -- Parent Issue: HDFS-8425 (was: HDFS-8031) > Erasure coding: shrink size of indices in BlockInfoStriped >

[jira] [Updated] (HDFS-7622) Erasure Coding: handling file truncation

2017-01-18 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-7622: -- Parent Issue: HDFS-11348 (was: HDFS-8031) > Erasure Coding: handling file truncation >

[jira] [Updated] (HDFS-7345) Local Repairable Codes (LRC)

2017-01-18 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-7345: -- Component/s: erasure-coding > Local Repairable Codes (LRC) > > >

[jira] [Updated] (HDFS-7354) Optimized handling of parity blocks in block management

2017-01-18 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-7354: -- Parent Issue: HDFS-8425 (was: HDFS-8031) > Optimized handling of parity blocks in block management >

[jira] [Updated] (HDFS-7345) Local Repairable Codes (LRC)

2017-01-18 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-7345: -- Issue Type: New Feature (was: Sub-task) Parent: (was: HDFS-8031) > Local Repairable Codes

[jira] [Updated] (HDFS-8425) [umbrella] Performance tuning, investigation and optimization for erasure coding

2017-01-18 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-8425: -- Component/s: erasure-coding > [umbrella] Performance tuning, investigation and optimization for erasure

[jira] [Updated] (HDFS-8425) [umbrella] Performance tuning, investigation and optimization for erasure coding

2017-01-18 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-8425: -- Issue Type: Improvement (was: Sub-task) Parent: (was: HDFS-8031) > [umbrella] Performance

[jira] [Updated] (HDFS-9098) Erasure coding: emulate race conditions among striped streamers in write pipeline

2017-01-18 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-9098: -- Parent Issue: HDFS-11349 (was: HDFS-8031) > Erasure coding: emulate race conditions among striped

[jira] [Updated] (HDFS-8799) Erasure Coding: add tests for namenode processing corrupt striped blocks

2017-01-18 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-8799: -- Parent Issue: HDFS-11349 (was: HDFS-8031) > Erasure Coding: add tests for namenode processing corrupt

[jira] [Updated] (HDFS-8199) Erasure Coding: System test of creating ECZone and EC files.

2017-01-18 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-8199: -- Parent Issue: HDFS-8197 (was: HDFS-8031) > Erasure Coding: System test of creating ECZone and EC files.

[jira] [Updated] (HDFS-8198) Erasure Coding: system test of TeraSort

2017-01-18 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-8198: -- Parent Issue: HDFS-8197 (was: HDFS-8031) > Erasure Coding: system test of TeraSort >

[jira] [Updated] (HDFS-8265) Erasure Coding: System Test of Quota calculation for EC files

2017-01-18 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-8265: -- Issue Type: Sub-task (was: Test) Parent: HDFS-8197 > Erasure Coding: System Test of Quota

[jira] [Updated] (HDFS-8260) Erasure Coding: system test of writing EC file

2017-01-18 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-8260: -- Assignee: (was: Xinwei Qin ) Issue Type: Sub-task (was: Test) Parent: HDFS-8197 >

[jira] [Updated] (HDFS-8264) Erasure Coding: System Test of version update

2017-01-18 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-8264: -- Issue Type: Sub-task (was: Test) Parent: HDFS-8197 > Erasure Coding: System Test of version

[jira] [Updated] (HDFS-8263) Erasure Coding: System Test of fsck for EC files

2017-01-18 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-8263: -- Issue Type: Sub-task (was: Test) Parent: HDFS-8197 > Erasure Coding: System Test of fsck for

[jira] [Updated] (HDFS-8262) Erasure Coding: System Test of datanode decommission which EC blocks are stored

2017-01-18 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-8262: -- Assignee: (was: Xinwei Qin ) Issue Type: Sub-task (was: Test) Parent: HDFS-8197 >

[jira] [Updated] (HDFS-8261) Erasure Coding: System Test of EC file reconstruction

2017-01-18 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-8261: -- Issue Type: Sub-task (was: Test) Parent: HDFS-8197 > Erasure Coding: System Test of EC file

[jira] [Commented] (HDFS-4025) QJM: Sychronize past log segments to JNs that missed them

2017-01-18 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15828903#comment-15828903 ] Jing Zhao commented on HDFS-4025: - Thanks for the update, [~hkoneru]. The current patch looks better.

[jira] [Updated] (HDFS-8197) [umbrella] System tests for EC feature

2017-01-18 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-8197: -- Issue Type: Test (was: Sub-task) Parent: (was: HDFS-8031) > [umbrella] System tests for EC

[jira] [Updated] (HDFS-8201) Refactor the end to end test for stripping file writing and reading

2017-01-18 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-8201: -- Parent Issue: HDFS-11349 (was: HDFS-8031) > Refactor the end to end test for stripping file writing and

[jira] [Updated] (HDFS-8293) Erasure Coding: test the retry logic of DFSStripedInputStream

2017-01-18 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-8293: -- Parent Issue: HDFS-11349 (was: HDFS-8031) > Erasure Coding: test the retry logic of

[jira] [Updated] (HDFS-8065) Erasure coding: Support truncate at striped group boundary

2017-01-18 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-8065: -- Parent Issue: HDFS-11348 (was: HDFS-8031) > Erasure coding: Support truncate at striped group boundary

[jira] [Created] (HDFS-11349) [Umbrella] Additional testing for striped erasure coding

2017-01-18 Thread Andrew Wang (JIRA)
Andrew Wang created HDFS-11349: -- Summary: [Umbrella] Additional testing for striped erasure coding Key: HDFS-11349 URL: https://issues.apache.org/jira/browse/HDFS-11349 Project: Hadoop HDFS

[jira] [Updated] (HDFS-7663) Erasure Coding: Append on striped file

2017-01-18 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-7663: -- Parent Issue: HDFS-11348 (was: HDFS-8031) > Erasure Coding: Append on striped file >

[jira] [Updated] (HDFS-7619) Erasure Coding: Handling truncated files in snapshots

2017-01-18 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-7619: -- Parent Issue: HDFS-11348 (was: HDFS-8031) > Erasure Coding: Handling truncated files in snapshots >

[jira] [Created] (HDFS-11348) [Umbrella] Append and truncate for striped EC files

2017-01-18 Thread Andrew Wang (JIRA)
Andrew Wang created HDFS-11348: -- Summary: [Umbrella] Append and truncate for striped EC files Key: HDFS-11348 URL: https://issues.apache.org/jira/browse/HDFS-11348 Project: Hadoop HDFS Issue

[jira] [Updated] (HDFS-7618) Support read when a file is being converted

2017-01-18 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-7618: -- Parent Issue: HDFS-11347 (was: HDFS-8031) > Support read when a file is being converted >

[jira] [Commented] (HDFS-10759) Change fsimage bool isStriped from boolean to an enum

2017-01-18 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15828890#comment-15828890 ] Hudson commented on HDFS-10759: --- ABORTED: Integrated in Jenkins build Hadoop-trunk-Commit #11136 (See

[jira] [Updated] (HDFS-7717) Erasure Coding: distribute replication to EC conversion work to DataNode

2017-01-18 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-7717: -- Parent Issue: HDFS-11347 (was: HDFS-8031) > Erasure Coding: distribute replication to EC conversion

[jira] [Created] (HDFS-11347) [Umbrella] Converting files from replicated to striped EC

2017-01-18 Thread Andrew Wang (JIRA)
Andrew Wang created HDFS-11347: -- Summary: [Umbrella] Converting files from replicated to striped EC Key: HDFS-11347 URL: https://issues.apache.org/jira/browse/HDFS-11347 Project: Hadoop HDFS

[jira] [Commented] (HDFS-8031) Follow-on work for erasure coding phase I (striping layout)

2017-01-18 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15828882#comment-15828882 ] Andrew Wang commented on HDFS-8031: --- +1 to Kai's statement, I plan to use a mix of umbrella JIRAs for

[jira] [Commented] (HDFS-11268) Correctly reconstruct erasure coding file from FSImage

2017-01-18 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15828821#comment-15828821 ] Andrew Wang commented on HDFS-11268: Fix and unit test LGTM thanks Sammi, but looks like we need a

[jira] [Updated] (HDFS-11326) FSNamesystem closeFileCommitBlocks block FSNamesystem Ops

2017-01-18 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-11326: --- Target Version/s: 3.0.0-alpha2 (was: 3.0.0-alpha1) > FSNamesystem closeFileCommitBlocks block

[jira] [Deleted] (HDFS-11328) FSNamesystem closeFileCommitBlocks block FSNamesystem Ops

2017-01-18 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang deleted HDFS-11328: --- > FSNamesystem closeFileCommitBlocks block FSNamesystem Ops >

[jira] [Commented] (HDFS-10733) NameNode terminated after full GC thinking QJM is unresponsive.

2017-01-18 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10733?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15828811#comment-15828811 ] Hudson commented on HDFS-10733: --- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11135 (See

[jira] [Deleted] (HDFS-11327) FSNamesystem closeFileCommitBlocks block FSNamesystem Ops

2017-01-18 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang deleted HDFS-11327: --- > FSNamesystem closeFileCommitBlocks block FSNamesystem Ops >

  1   2   >