[jira] [Updated] (HDFS-1508) Ability to do savenamespace without being in safemode

2015-05-05 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HDFS-1508: --- Labels: BB2015-05-TBR (was: ) > Ability to do savenamespace without being in safemode > --

[jira] [Updated] (HDFS-6108) make the length of the range in sync_file_range() configurable

2015-05-05 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HDFS-6108: --- Labels: BB2015-05-TBR (was: ) > make the length of the range in sync_file_range() configurable

[jira] [Updated] (HDFS-6743) Put IP address into a new column on the new NN webUI

2015-05-05 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HDFS-6743: --- Labels: BB2015-05-TBR (was: ) > Put IP address into a new column on the new NN webUI > ---

[jira] [Updated] (HDFS-6524) chooseDataNode decides retry times considering with block replica number

2015-05-05 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HDFS-6524: --- Labels: BB2015-05-TBR (was: ) > chooseDataNode decides retry times considering with block repl

[jira] [Updated] (HDFS-7630) TestConnCache hardcode block size without considering native OS

2015-05-05 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HDFS-7630: --- Labels: BB2015-05-TBR (was: ) > TestConnCache hardcode block size without considering native O

[jira] [Updated] (HDFS-6698) try to optimize DFSInputStream.getFileLength()

2015-05-05 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HDFS-6698: --- Labels: BB2015-05-TBR (was: ) > try to optimize DFSInputStream.getFileLength() > -

[jira] [Updated] (HDFS-6766) optimize ack notify mechanism to avoid thundering herd issue

2015-05-05 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HDFS-6766: --- Labels: BB2015-05-TBR (was: ) > optimize ack notify mechanism to avoid thundering herd issue >

[jira] [Updated] (HDFS-7553) fix the TestDFSUpgradeWithHA due to BindException

2015-05-05 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HDFS-7553: --- Labels: BB2015-05-TBR (was: ) > fix the TestDFSUpgradeWithHA due to BindException > --

[jira] [Updated] (HDFS-7314) Aborted DFSClient's impact on long running service like YARN

2015-05-05 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HDFS-7314: --- Labels: BB2015-05-TBR (was: ) > Aborted DFSClient's impact on long running service like YARN >

[jira] [Updated] (HDFS-7692) DataStorage#addStorageLocations(...) should support MultiThread to speedup the upgrade of block pool at multi storage directories.

2015-05-05 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HDFS-7692: --- Labels: BB2015-05-TBR (was: ) > DataStorage#addStorageLocations(...) should support MultiThrea

[jira] [Updated] (HDFS-5270) Use thread pools in the datenode daemons

2015-05-05 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HDFS-5270: --- Labels: BB2015-05-TBR (was: ) > Use thread pools in the datenode daemons > ---

[jira] [Updated] (HDFS-6681) TestRBWBlockInvalidation#testBlockInvalidationWhenRBWReplicaMissedInDN is flaky and sometimes gets stuck in infinite loops

2015-05-05 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HDFS-6681: --- Labels: BB2015-05-TBR (was: ) > TestRBWBlockInvalidation#testBlockInvalidationWhenRBWReplicaMi

[jira] [Updated] (HDFS-5917) Have an ability to refresh deadNodes list periodically

2015-05-05 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HDFS-5917: --- Labels: BB2015-05-TBR (was: ) > Have an ability to refresh deadNodes list periodically > -

[jira] [Updated] (HDFS-6744) Improve decommissioning nodes and dead nodes access on the new NN webUI

2015-05-05 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HDFS-6744: --- Labels: BB2015-05-TBR (was: ) > Improve decommissioning nodes and dead nodes access on the new

[jira] [Updated] (HDFS-6136) PacketReceiver#doRead calls setFieldsFromData with wrong argument

2015-05-05 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HDFS-6136: --- Labels: BB2015-05-TBR patch (was: patch) > PacketReceiver#doRead calls setFieldsFromData with

[jira] [Updated] (HDFS-2484) checkLease should throw FileNotFoundException when file does not exist

2015-05-05 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HDFS-2484: --- Labels: (was: BB2015-05-RFC) > checkLease should throw FileNotFoundException when file does n

[jira] [Updated] (HDFS-4185) Add a metric for number of active leases

2015-05-05 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HDFS-4185: --- Labels: (was: BB2015-05-RFC) > Add a metric for number of active leases > ---

[jira] [Updated] (HDFS-6300) Shouldn't allows to run multiple balancer simultaneously

2015-05-05 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HDFS-6300: --- Labels: (was: BB2015-05-RFC) > Shouldn't allows to run multiple balancer simultaneously > ---

[jira] [Updated] (HDFS-6348) Secondary namenode - RMI Thread prevents JVM from exiting after main() completes

2015-05-05 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer updated HDFS-6348: --- Labels: (was: BB2015-05-RFC) > Secondary namenode - RMI Thread prevents JVM from exiting aft

[jira] [Commented] (HDFS-8321) CacheDirectives and CachePool operations should throw RetriableException in safemode

2015-05-05 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14529817#comment-14529817 ] Hadoop QA commented on HDFS-8321: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vo

[jira] [Updated] (HDFS-8329) Erasure coding: Rename Striped block recovery to reconstruction to eliminate confusion.

2015-05-05 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-8329: - Description: Both in NN and DN, we use striped block recovery and sometime use reconstruction. The striped block

[jira] [Assigned] (HDFS-8329) Erasure coding: Rename Striped block recovery to reconstruction to eliminate confusion.

2015-05-05 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu reassigned HDFS-8329: Assignee: Yi Liu > Erasure coding: Rename Striped block recovery to reconstruction to eliminate > confusion

[jira] [Created] (HDFS-8329) Erasure coding: Rename Striped block recovery to reconstruction to eliminate confusion.

2015-05-05 Thread Yi Liu (JIRA)
Yi Liu created HDFS-8329: Summary: Erasure coding: Rename Striped block recovery to reconstruction to eliminate confusion. Key: HDFS-8329 URL: https://issues.apache.org/jira/browse/HDFS-8329 Project: Hadoop H

[jira] [Created] (HDFS-8328) Follow-on to update decode for DataNode striped blocks reconstruction

2015-05-05 Thread Yi Liu (JIRA)
Yi Liu created HDFS-8328: Summary: Follow-on to update decode for DataNode striped blocks reconstruction Key: HDFS-8328 URL: https://issues.apache.org/jira/browse/HDFS-8328 Project: Hadoop HDFS Issu

[jira] [Commented] (HDFS-8019) Erasure Coding: erasure coding chunk buffer allocation and management

2015-05-05 Thread Kai Zheng (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14529754#comment-14529754 ] Kai Zheng commented on HDFS-8019: - Looks like not much thing to be done for the moment on t

[jira] [Commented] (HDFS-8291) Modify NN WebUI to display correct unit

2015-05-05 Thread Benoy Antony (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14529751#comment-14529751 ] Benoy Antony commented on HDFS-8291: Terabyte (TB) is more widely used than Tebibyte(Ti

[jira] [Commented] (HDFS-7348) Erasure Coding: DataNode reconstruct striped blocks

2015-05-05 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14529745#comment-14529745 ] Yi Liu commented on HDFS-7348: -- Thanks Zhe for the review and commit! > Erasure Coding: DataN

[jira] [Commented] (HDFS-7980) Incremental BlockReport will dramatically slow down the startup of a namenode

2015-05-05 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14529737#comment-14529737 ] Tsz Wo Nicholas Sze commented on HDFS-7980: --- The patch could not be applied. Cou

[jira] [Comment Edited] (HDFS-8203) Erasure Coding: Seek and other Ops in DFSStripedInputStream.

2015-05-05 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14529735#comment-14529735 ] Yi Liu edited comment on HDFS-8203 at 5/6/15 1:47 AM: -- Update the patc

[jira] [Updated] (HDFS-8203) Erasure Coding: Seek and other Ops in DFSStripedInputStream.

2015-05-05 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-8203: - Attachment: HDFS-8203.002.patch Update the patch. Current we have the {{curStripedBuf}}, so {{seek}} could be simp

[jira] [Commented] (HDFS-8284) Add usage of tracing originated in DFSClient to doc

2015-05-05 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14529714#comment-14529714 ] Hadoop QA commented on HDFS-8284: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vo

[jira] [Commented] (HDFS-6440) Support more than 2 NameNodes

2015-05-05 Thread Aaron T. Myers (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14529699#comment-14529699 ] Aaron T. Myers commented on HDFS-6440: -- Hi Jesse and Lars, My sincere apologies it t

[jira] [Commented] (HDFS-8314) Move HdfsServerConstants#IO_FILE_BUFFER_SIZE and SMALL_BUFFER_SIZE to the users

2015-05-05 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14529688#comment-14529688 ] Hadoop QA commented on HDFS-8314: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vo

[jira] [Commented] (HDFS-7678) Erasure coding: DFSInputStream with decode functionality (pread)

2015-05-05 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14529685#comment-14529685 ] Jing Zhao commented on HDFS-7678: - Thanks for updating the patch, Zhe! Here are some furthe

[jira] [Commented] (HDFS-8062) Remove hard-coded values in favor of EC schema

2015-05-05 Thread Kai Zheng (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14529617#comment-14529617 ] Kai Zheng commented on HDFS-8062: - Nicholas, I was mentioning May 8th because I thought [~

[jira] [Updated] (HDFS-8327) Compute storage type quotas in INodeFile.computeQuotaDeltaForTruncate()

2015-05-05 Thread Haohui Mai (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haohui Mai updated HDFS-8327: - Attachment: HDFS-8327.001.patch > Compute storage type quotas in INodeFile.computeQuotaDeltaForTruncate() >

[jira] [Updated] (HDFS-8327) Compute storage type quotas in INodeFile.computeQuotaDeltaForTruncate()

2015-05-05 Thread Haohui Mai (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haohui Mai updated HDFS-8327: - Status: Patch Available (was: Open) > Compute storage type quotas in INodeFile.computeQuotaDeltaForTruncat

[jira] [Updated] (HDFS-8327) Compute storage type quotas in INodeFile.computeQuotaDeltaForTruncate()

2015-05-05 Thread Haohui Mai (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haohui Mai updated HDFS-8327: - Status: Open (was: Patch Available) > Compute storage type quotas in INodeFile.computeQuotaDeltaForTruncat

[jira] [Commented] (HDFS-7859) Erasure Coding: Persist EC schemas in NameNode

2015-05-05 Thread Kai Zheng (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14529562#comment-14529562 ] Kai Zheng commented on HDFS-7859: - Nicholas, This was already move out of HDFS-7285 you di

[jira] [Commented] (HDFS-8255) Rename getBlockReplication to getPreferredBlockStorageNum

2015-05-05 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14529559#comment-14529559 ] Hadoop QA commented on HDFS-8255: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vo

[jira] [Updated] (HDFS-8327) Compute storage type quotas in INodeFile.computeQuotaDeltaForTruncate()

2015-05-05 Thread Haohui Mai (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haohui Mai updated HDFS-8327: - Status: Patch Available (was: Open) > Compute storage type quotas in INodeFile.computeQuotaDeltaForTruncat

[jira] [Updated] (HDFS-8327) Compute storage type quotas in INodeFile.computeQuotaDeltaForTruncate()

2015-05-05 Thread Haohui Mai (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haohui Mai updated HDFS-8327: - Attachment: HDFS-8327.000.patch > Compute storage type quotas in INodeFile.computeQuotaDeltaForTruncate() >

[jira] [Updated] (HDFS-8321) CacheDirectives and CachePool operations should throw RetriableException in safemode

2015-05-05 Thread Haohui Mai (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haohui Mai updated HDFS-8321: - Attachment: HDFS-8321.001.patch > CacheDirectives and CachePool operations should throw RetriableException

[jira] [Commented] (HDFS-7348) Erasure Coding: DataNode reconstruct striped blocks

2015-05-05 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14529550#comment-14529550 ] Zhe Zhang commented on HDFS-7348: - I committed the patch. > Erasure Coding: DataNode recon

[jira] [Resolved] (HDFS-7348) Erasure Coding: DataNode reconstruct striped blocks

2015-05-05 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang resolved HDFS-7348. - Resolution: Fixed Fix Version/s: HDFS-7285 Hadoop Flags: Reviewed > Erasure Coding: DataNode re

[jira] [Created] (HDFS-8327) Compute storage type quotas in INodeFile.computeQuotaDeltaForTruncate()

2015-05-05 Thread Haohui Mai (JIRA)
Haohui Mai created HDFS-8327: Summary: Compute storage type quotas in INodeFile.computeQuotaDeltaForTruncate() Key: HDFS-8327 URL: https://issues.apache.org/jira/browse/HDFS-8327 Project: Hadoop HDFS

[jira] [Updated] (HDFS-7348) Erasure Coding: DataNode reconstruct striped blocks

2015-05-05 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-7348: Summary: Erasure Coding: DataNode reconstruct striped blocks (was: Erasure Coding: striped block recovery)

[jira] [Updated] (HDFS-7672) Erasure Coding: consolidate streamer coordination logic and handle failure when writing striped blocks

2015-05-05 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-7672: Summary: Erasure Coding: consolidate streamer coordination logic and handle failure when writing striped bloc

[jira] [Commented] (HDFS-7672) Erasure Coding: consolidate streamer coordination logic and handle write failure when writing striped blocks

2015-05-05 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14529541#comment-14529541 ] Zhe Zhang commented on HDFS-7672: - Thanks Nicholas. I updated the JIRA summary and please f

[jira] [Updated] (HDFS-7672) Erasure Coding: consolidate streamer coordination logic and handle write failure when writing striped blocks

2015-05-05 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-7672: Summary: Erasure Coding: consolidate streamer coordination logic and handle write failure when writing stripe

[jira] [Updated] (HDFS-8288) Refactor DFSStripedOutputStream and StripedDataStreamer

2015-05-05 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-8288: -- Resolution: Duplicate Status: Resolved (was: Patch Available) The code was commi

[jira] [Resolved] (HDFS-7672) Erasure Coding: handle write failure for stripping coding blocks

2015-05-05 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze resolved HDFS-7672. --- Resolution: Fixed Fix Version/s: HDFS-7285 Hadoop Flags: Reviewed Thanks Ji

[jira] [Commented] (HDFS-7348) Erasure Coding: striped block recovery

2015-05-05 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14529532#comment-14529532 ] Zhe Zhang commented on HDFS-7348: - Thanks Yi for the update! The patch LGTM. The only conce

[jira] [Updated] (HDFS-7672) Erasure Coding: handle write failure for stripping coding blocks

2015-05-05 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-7672: -- Description: In *stripping* case, for (6, 3)-Reed-Solomon, a client writes to 6 data bloc

[jira] [Commented] (HDFS-8286) Scaling out the namespace using KV store

2015-05-05 Thread Konstantin Shvachko (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14529524#comment-14529524 ] Konstantin Shvachko commented on HDFS-8286: --- Hey guys, I read the design doc, and

[jira] [Commented] (HDFS-7672) Erasure Coding: handle write failure for stripping coding blocks

2015-05-05 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14529519#comment-14529519 ] Zhe Zhang commented on HDFS-7672: - The latest patch LGTM with Nicholas's explanations. {{s

[jira] [Commented] (HDFS-7672) Erasure Coding: handle write failure for stripping coding blocks

2015-05-05 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14529516#comment-14529516 ] Zhe Zhang commented on HDFS-7672: - I started reviewing HDFS-8288 patch but it seems outdate

[jira] [Commented] (HDFS-8314) Move HdfsServerConstants#IO_FILE_BUFFER_SIZE and SMALL_BUFFER_SIZE to the users

2015-05-05 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14529503#comment-14529503 ] Hudson commented on HDFS-8314: -- FAILURE: Integrated in Hadoop-trunk-Commit #7742 (See [https:

[jira] [Updated] (HDFS-8314) Move HdfsServerConstants#IO_FILE_BUFFER_SIZE and SMALL_BUFFER_SIZE to the users

2015-05-05 Thread Haohui Mai (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haohui Mai updated HDFS-8314: - Resolution: Fixed Fix Version/s: 2.8.0 Hadoop Flags: Reviewed Status: Resolved (was:

[jira] [Commented] (HDFS-7672) Erasure Coding: handle write failure for stripping coding blocks

2015-05-05 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14529459#comment-14529459 ] Zhe Zhang commented on HDFS-7672: - bq. I do have a separated JIRA HDFS-8288. Do you want to

[jira] [Commented] (HDFS-8314) Move HdfsServerConstants#IO_FILE_BUFFER_SIZE and SMALL_BUFFER_SIZE to the users

2015-05-05 Thread Haohui Mai (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14529439#comment-14529439 ] Haohui Mai commented on HDFS-8314: -- +1 > Move HdfsServerConstants#IO_FILE_BUFFER_SIZE and

[jira] [Commented] (HDFS-5223) Allow edit log/fsimage format changes without changing layout version

2015-05-05 Thread Chris Nauroth (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14529416#comment-14529416 ] Chris Nauroth commented on HDFS-5223: - bq. The fact that presently all features are alw

[jira] [Commented] (HDFS-8311) DataStreamer.transfer() should timeout the socket InputStream.

2015-05-05 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14529413#comment-14529413 ] Hadoop QA commented on HDFS-8311: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vo

[jira] [Updated] (HDFS-7672) Erasure Coding: handle write failure for stripping coding blocks

2015-05-05 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-7672: -- Attachment: h7672_20150505b.patch h7672_20150505b.patch: fixes NPE. > Erasure Coding: ha

[jira] [Commented] (HDFS-7672) Erasure Coding: handle write failure for stripping coding blocks

2015-05-05 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14529400#comment-14529400 ] Tsz Wo Nicholas Sze commented on HDFS-7672: --- 2. I also found it when running unit

[jira] [Commented] (HDFS-2484) checkLease should throw FileNotFoundException when file does not exist

2015-05-05 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14529396#comment-14529396 ] Hadoop QA commented on HDFS-2484: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vo

[jira] [Commented] (HDFS-8306) Generate ACL and Xattr outputs in OIV XML outputs

2015-05-05 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14529393#comment-14529393 ] Hadoop QA commented on HDFS-8306: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vo

[jira] [Commented] (HDFS-7672) Erasure Coding: handle write failure for stripping coding blocks

2015-05-05 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14529341#comment-14529341 ] Zhe Zhang commented on HDFS-7672: - Thanks for the new patch Nicholas. A few comments: Nits:

[jira] [Commented] (HDFS-5223) Allow edit log/fsimage format changes without changing layout version

2015-05-05 Thread Aaron T. Myers (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14529336#comment-14529336 ] Aaron T. Myers commented on HDFS-5223: -- bq. Complexity in HDFS often arises from combi

[jira] [Updated] (HDFS-8314) Move HdfsServerConstants#IO_FILE_BUFFER_SIZE and SMALL_BUFFER_SIZE to the users

2015-05-05 Thread Li Lu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Lu updated HDFS-8314: Attachment: HDFS-8314-trunk.004.patch Could not reproduce the UT failure locally. Can anybody double check this? I'm

[jira] [Commented] (HDFS-5223) Allow edit log/fsimage format changes without changing layout version

2015-05-05 Thread Chris Nauroth (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14529313#comment-14529313 ] Chris Nauroth commented on HDFS-5223: - Hi [~atm]. bq. Seems like this approach would c

[jira] [Updated] (HDFS-8314) Move HdfsServerConstants#IO_FILE_BUFFER_SIZE and SMALL_BUFFER_SIZE to the users

2015-05-05 Thread Li Lu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Lu updated HDFS-8314: Attachment: (was: HDFS-8314-trunk.004.patch) > Move HdfsServerConstants#IO_FILE_BUFFER_SIZE and SMALL_BUFFER_SIZE

[jira] [Updated] (HDFS-8284) Add usage of tracing originated in DFSClient to doc

2015-05-05 Thread Masatake Iwasaki (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki updated HDFS-8284: --- Attachment: HDFS-8284.003.patch bq. The only thing I would suggest is that we should remove the

[jira] [Commented] (HDFS-2484) checkLease should throw FileNotFoundException when file does not exist

2015-05-05 Thread Konstantin Shvachko (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-2484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14529302#comment-14529302 ] Konstantin Shvachko commented on HDFS-2484: --- +1 pending Jenkins. > checkLease sh

[jira] [Commented] (HDFS-8291) Modify NN WebUI to display correct unit

2015-05-05 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14529293#comment-14529293 ] Hadoop QA commented on HDFS-8291: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vo

[jira] [Commented] (HDFS-7310) Mover can give first priority to local DN if it has target storage type available in local DN

2015-05-05 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14529292#comment-14529292 ] Tsz Wo Nicholas Sze commented on HDFS-7310: --- FYI, Mover/Balancer may schedule two

[jira] [Commented] (HDFS-8314) Move HdfsServerConstants#IO_FILE_BUFFER_SIZE and SMALL_BUFFER_SIZE to the users

2015-05-05 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14529287#comment-14529287 ] Hadoop QA commented on HDFS-8314: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vo

[jira] [Commented] (HDFS-8157) Writes to RAM DISK reserve locked memory for block files

2015-05-05 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14529275#comment-14529275 ] Colin Patrick McCabe commented on HDFS-8157: Thanks for this, [~arpitagarwal].

[jira] [Commented] (HDFS-8246) Get HDFS file name based on block pool id and block id

2015-05-05 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14529256#comment-14529256 ] Colin Patrick McCabe commented on HDFS-8246: How about having {{/.reserved/.blo

[jira] [Commented] (HDFS-8219) setStoragePolicy with folder behavior is different after cluster restart

2015-05-05 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14529242#comment-14529242 ] Hudson commented on HDFS-8219: -- FAILURE: Integrated in Hadoop-trunk-Commit #7740 (See [https:

[jira] [Updated] (HDFS-8219) setStoragePolicy with folder behavior is different after cluster restart

2015-05-05 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HDFS-8219: Release Note: (was: The fix has been committed to trunk and branch-2.) > setStoragePolicy with fold

[jira] [Updated] (HDFS-8219) setStoragePolicy with folder behavior is different after cluster restart

2015-05-05 Thread Xiaoyu Yao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8219?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoyu Yao updated HDFS-8219: - Resolution: Fixed Fix Version/s: 2.8.0 Release Note: The fix has been committed to trunk and bra

[jira] [Updated] (HDFS-8014) Erasure Coding: local and remote block reader for coding work in DataNode

2015-05-05 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-8014: Parent Issue: HDFS-8031 (was: HDFS-7285) > Erasure Coding: local and remote block reader for coding work in

[jira] [Commented] (HDFS-5223) Allow edit log/fsimage format changes without changing layout version

2015-05-05 Thread Aaron T. Myers (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14529231#comment-14529231 ] Aaron T. Myers commented on HDFS-5223: -- Hey Chris, thanks a lot for working on this.

[jira] [Updated] (HDFS-8255) Rename getBlockReplication to getPreferredBlockStorageNum

2015-05-05 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-8255: Affects Version/s: 2.8.0 > Rename getBlockReplication to getPreferredBlockStorageNum > --

[jira] [Updated] (HDFS-5223) Allow edit log/fsimage format changes without changing layout version

2015-05-05 Thread Chris Nauroth (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Nauroth updated HDFS-5223: Attachment: HDFS-5223.005.patch > Allow edit log/fsimage format changes without changing layout versi

[jira] [Commented] (HDFS-8271) NameNode should bind on both IPv6 and IPv4 if running on dual-stack machine and IPv6 enabled

2015-05-05 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14529197#comment-14529197 ] Colin Patrick McCabe commented on HDFS-8271: An uber jira is a good idea. Also

[jira] [Updated] (HDFS-5223) Allow edit log/fsimage format changes without changing layout version

2015-05-05 Thread Chris Nauroth (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Nauroth updated HDFS-5223: Attachment: HDFS-5223-HDFS-Downgrade-Extended-Support.pdf I've been testing a patch based on Sanjay's

[jira] [Commented] (HDFS-8315) Datanode does not log reads

2015-05-05 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14529178#comment-14529178 ] Gopal V commented on HDFS-8315: --- [~kihwal]: the HDFS in-memory layer is IOPS bound on logging

[jira] [Updated] (HDFS-7672) Erasure Coding: handle write failure for stripping coding blocks

2015-05-05 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-7672: -- Attachment: h7672_20150505.patch h7672_20150505.patch Here is a new patch. Please let m

[jira] [Commented] (HDFS-6407) new namenode UI, lost ability to sort columns in datanode tab

2015-05-05 Thread Haohui Mai (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14529143#comment-14529143 ] Haohui Mai commented on HDFS-6407: -- {code} +jQuery.fn.dataTable.ext.type.detect.unshift( f

[jira] [Updated] (HDFS-8255) Rename getBlockReplication to getPreferredBlockStorageNum

2015-05-05 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-8255: Attachment: HDFS-8255.001.patch Thanks Andrew for the comment. Yes we do want to support striping + replicati

[jira] [Commented] (HDFS-7672) Erasure Coding: handle write failure for stripping coding blocks

2015-05-05 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14529129#comment-14529129 ] Jing Zhao commented on HDFS-7672: - Thanks for updating the patch, Nicholas! The latest patc

[jira] [Updated] (HDFS-7678) Erasure coding: DFSInputStream with decode functionality (pread)

2015-05-05 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-7678: Attachment: HDFS-7678-HDFS-7285.010.patch Thanks Jing for the review; it's very helpful. Updating the patch t

[jira] [Commented] (HDFS-8284) Add usage of tracing originated in DFSClient to doc

2015-05-05 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14529094#comment-14529094 ] Colin Patrick McCabe commented on HDFS-8284: Thanks, [~iwasakims]. This looks

[jira] [Commented] (HDFS-7980) Incremental BlockReport will dramatically slow down the startup of a namenode

2015-05-05 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14529045#comment-14529045 ] Hadoop QA commented on HDFS-7980: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vo

[jira] [Commented] (HDFS-8219) setStoragePolicy with folder behavior is different after cluster restart

2015-05-05 Thread Xiaoyu Yao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14529044#comment-14529044 ] Xiaoyu Yao commented on HDFS-8219: -- The patch looks good to me. Thanks [~surendrasingh] fo

[jira] [Commented] (HDFS-7859) Erasure Coding: Persist EC schemas in NameNode

2015-05-05 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14529040#comment-14529040 ] Tsz Wo Nicholas Sze commented on HDFS-7859: --- Please do not commit this JIRA to th

[jira] [Commented] (HDFS-7847) Modify NNThroughputBenchmark to be able to operate on a remote NameNode

2015-05-05 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14529036#comment-14529036 ] Hudson commented on HDFS-7847: -- FAILURE: Integrated in Hadoop-trunk-Commit #7739 (See [https:

[jira] [Updated] (HDFS-8311) DataStreamer.transfer() should timeout the socket InputStream.

2015-05-05 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-8311: --- Target Version/s: 2.8.0 Status: Patch Available (was: Open) > DataStream

<    1   2   3   4   5   6   >