[jira] [Commented] (HDFS-8725) Use std::chrono to implement the timer in the asio library

2015-07-07 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14617418#comment-14617418 ] Jing Zhao commented on HDFS-8725: - +1 > Use std::chrono to implement the timer in the asio

[jira] [Updated] (HDFS-8669) Erasure Coding: handle missing internal block locations in DFSStripedInputStream

2015-07-07 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8669: Attachment: HDFS-8669.001.patch Thanks for the review, Walter! Update the patch to address your comments. Als

[jira] [Commented] (HDFS-7285) Erasure Coding Support inside HDFS

2015-07-07 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14617684#comment-14617684 ] Jing Zhao commented on HDFS-7285: - Thanks for the work, Zhe! So for the BlockInfoStriped re

[jira] [Commented] (HDFS-7285) Erasure Coding Support inside HDFS

2015-07-07 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14617695#comment-14617695 ] Jing Zhao commented on HDFS-7285: - Similar for the changes on INodeFile. In general I sugge

[jira] [Commented] (HDFS-8563) Erasure Coding: fsck handles file smaller than a full stripe

2015-07-07 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14617787#comment-14617787 ] Jing Zhao commented on HDFS-8563: - The patch looks good to me. Just some minor comments: #

[jira] [Commented] (HDFS-8729) Fix testTruncateWithDataNodesRestartImmediately occasionally failed

2015-07-08 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14619035#comment-14619035 ] Jing Zhao commented on HDFS-8729: - To trigger block report or not before restarting DataNod

[jira] [Commented] (HDFS-8563) Erasure Coding: fsck handles file smaller than a full stripe

2015-07-08 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14619056#comment-14619056 ] Jing Zhao commented on HDFS-8563: - Forgot to include the attachment? > Erasure Coding: fsc

[jira] [Commented] (HDFS-8719) Erasure Coding: client generates too many small packets when writing parity data

2015-07-08 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14619376#comment-14619376 ] Jing Zhao commented on HDFS-8719: - Looks like switching streamer and refreshing {{chunksPer

[jira] [Commented] (HDFS-8734) Erasure Coding: fix one cell need two packets

2015-07-08 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14619467#comment-14619467 ] Jing Zhao commented on HDFS-8734: - The analysis makes sense to me. But looks like we cannot

[jira] [Updated] (HDFS-8563) Erasure Coding: fsck handles file smaller than a full stripe

2015-07-09 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8563: Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: HDFS-7285 Status: Resolved (wa

[jira] [Commented] (HDFS-8497) ErasureCodingWorker fails to do decode work

2015-07-09 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14621012#comment-14621012 ] Jing Zhao commented on HDFS-8497: - Looks like we can resolve this jira now? > ErasureCodin

[jira] [Commented] (HDFS-8732) Erasure Coding: Fail to read a file with corrupted blocks

2015-07-09 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14621060#comment-14621060 ] Jing Zhao commented on HDFS-8732: - HDFS-8669 can fix this I think. > Erasure Coding: Fail

[jira] [Commented] (HDFS-8729) Fix testTruncateWithDataNodesRestartImmediately occasionally failed

2015-07-09 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14621182#comment-14621182 ] Jing Zhao commented on HDFS-8729: - bq. That's not what I do. In the 01 patch, checkBlockRec

[jira] [Updated] (HDFS-8729) Fix testTruncateWithDataNodesRestartImmediately occasionally failed

2015-07-09 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8729: Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.8.0 Status: Resolved (was: P

[jira] [Commented] (HDFS-8484) Erasure coding: Two contiguous blocks occupy IDs belong to same striped group

2015-07-09 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14621212#comment-14621212 ] Jing Zhao commented on HDFS-8484: - Good catch! +1. I will commit it shortly. > Erasure cod

[jira] [Resolved] (HDFS-8484) Erasure coding: Two contiguous blocks occupy IDs belong to same striped group

2015-07-09 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao resolved HDFS-8484. - Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: HDFS-7285 I've committed this to the fe

[jira] [Commented] (HDFS-8744) Erasure Coding: the number of chunks in packet is not updated when writing parity data

2015-07-09 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14621340#comment-14621340 ] Jing Zhao commented on HDFS-8744: - The patch looks good to me. One minor is that the old {{

[jira] [Commented] (HDFS-8058) Erasure coding: use BlockInfo[] for both striped and contiguous blocks in INodeFile

2015-07-09 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14621444#comment-14621444 ] Jing Zhao commented on HDFS-8058: - Thanks for working on this, Zhe. The patch looks good to

[jira] [Commented] (HDFS-8433) blockToken is not set in constructInternalBlock and parseStripedBlockGroup in StripedBlockUtil

2015-07-09 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14621499#comment-14621499 ] Jing Zhao commented on HDFS-8433: - The issue with the changes on BlockTokenIdentifier's se

[jira] [Commented] (HDFS-8751) Remove setBlocks API from INodeFile and misc code cleanup

2015-07-10 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14622655#comment-14622655 ] Jing Zhao commented on HDFS-8751: - +1 pending Jenkins. > Remove setBlocks API from INodeFi

[jira] [Commented] (HDFS-8744) Erasure Coding: the number of chunks in packet is not updated when writing parity data

2015-07-10 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14622683#comment-14622683 ] Jing Zhao commented on HDFS-8744: - bq. packet_0 has 504 checksum + 64512 data bq. packet_1

[jira] [Updated] (HDFS-8751) Remove setBlocks API from INodeFile and misc code cleanup

2015-07-10 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8751: Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) I've commit

[jira] [Commented] (HDFS-8702) Erasure coding: update BlockManager.blockHasEnoughRacks(..) logic for striped block

2015-07-10 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14623049#comment-14623049 ] Jing Zhao commented on HDFS-8702: - Thanks for working on this, [~kaisasak]! Here are the co

[jira] [Created] (HDFS-8760) Erasure Coding: reuse BlockReader when reading the same block in pread

2015-07-10 Thread Jing Zhao (JIRA)
Jing Zhao created HDFS-8760: --- Summary: Erasure Coding: reuse BlockReader when reading the same block in pread Key: HDFS-8760 URL: https://issues.apache.org/jira/browse/HDFS-8760 Project: Hadoop HDFS

[jira] [Commented] (HDFS-8745) Use Doxygen to generate documents for libhdfspp

2015-07-10 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14623088#comment-14623088 ] Jing Zhao commented on HDFS-8745: - +1 > Use Doxygen to generate documents for libhdfspp >

[jira] [Commented] (HDFS-8737) Implement the Hadoop RPC v9 protocol

2015-07-10 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14623092#comment-14623092 ] Jing Zhao commented on HDFS-8737: - The patch looks good to me. But looks like this is just

[jira] [Commented] (HDFS-8744) Erasure Coding: the number of chunks in packet is not updated when writing parity data

2015-07-10 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14623112#comment-14623112 ] Jing Zhao commented on HDFS-8744: - But I think this is the format of the current packet: {c

[jira] [Updated] (HDFS-8669) Erasure Coding: handle missing internal block locations in DFSStripedInputStream

2015-07-13 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8669: Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: HDFS-7285 Status: Resolved (wa

[jira] [Reopened] (HDFS-8652) Track BlockInfo instead of Block in CorruptReplicasMap

2015-07-13 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao reopened HDFS-8652: - > Track BlockInfo instead of Block in CorruptReplicasMap >

[jira] [Updated] (HDFS-8652) Track BlockInfo instead of Block in CorruptReplicasMap

2015-07-13 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8652: Fix Version/s: (was: 2.8.0) > Track BlockInfo instead of Block in CorruptReplicasMap > --

[jira] [Resolved] (HDFS-8652) Track BlockInfo instead of Block in CorruptReplicasMap

2015-07-13 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao resolved HDFS-8652. - Resolution: Invalid Hadoop Flags: (was: Reviewed) > Track BlockInfo instead of Block in CorruptRepl

[jira] [Commented] (HDFS-8058) Erasure coding: use BlockInfo[] for both striped and contiguous blocks in INodeFile

2015-07-13 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14625453#comment-14625453 ] Jing Zhao commented on HDFS-8058: - Thanks for updating the patch, Zhe. The new patch looks

[jira] [Commented] (HDFS-8058) Erasure coding: use BlockInfo[] for both striped and contiguous blocks in INodeFile

2015-07-13 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14625622#comment-14625622 ] Jing Zhao commented on HDFS-8058: - The difference between {{IS_STRIPED}} and {{REPLICATION}

[jira] [Commented] (HDFS-8758) Implement the continuation library for libhdfspp

2015-07-13 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14625611#comment-14625611 ] Jing Zhao commented on HDFS-8758: - The continuation/pipeline definition looks pretty good t

[jira] [Commented] (HDFS-8702) Erasure coding: update BlockManager.blockHasEnoughRacks(..) logic for striped block

2015-07-14 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14626748#comment-14626748 ] Jing Zhao commented on HDFS-8702: - bq. I think we can use getRealDataBlockNum(); Hmm, you'

[jira] [Updated] (HDFS-8702) Erasure coding: update BlockManager.blockHasEnoughRacks(..) logic for striped block

2015-07-14 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8702: Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: HDFS-7285 Status: Resolved (wa

[jira] [Commented] (HDFS-8058) Erasure coding: use BlockInfo[] for both striped and contiguous blocks in INodeFile

2015-07-14 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14626810#comment-14626810 ] Jing Zhao commented on HDFS-8058: - The 007 patch looks good to me. Just two minors: # In {{

[jira] [Commented] (HDFS-8058) Erasure coding: use BlockInfo[] for both striped and contiguous blocks in INodeFile

2015-07-14 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14626956#comment-14626956 ] Jing Zhao commented on HDFS-8058: - Here {{fileInPb.getIsStriped()}} should be {{file.isStri

[jira] [Commented] (HDFS-8734) Erasure Coding: fix one cell need two packets

2015-07-14 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14626973#comment-14626973 ] Jing Zhao commented on HDFS-8734: - The patch looks good to me. +1. One concern is that we

[jira] [Updated] (HDFS-8734) Erasure Coding: fix one cell need two packets

2015-07-14 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8734: Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: HDFS-7285 Status: Resolved (wa

[jira] [Commented] (HDFS-8433) blockToken is not set in constructInternalBlock and parseStripedBlockGroup in StripedBlockUtil

2015-07-14 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14627005#comment-14627005 ] Jing Zhao commented on HDFS-8433: - bq. This time, we don't need BlockIdRange class. We exte

[jira] [Commented] (HDFS-8759) Implement remote block reader in libhdfspp

2015-07-14 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14627037#comment-14627037 ] Jing Zhao commented on HDFS-8759: - Agree with [~James Clampffer] that we need to have detai

[jira] [Commented] (HDFS-8058) Erasure coding: use BlockInfo[] for both striped and contiguous blocks in INodeFile

2015-07-14 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14627104#comment-14627104 ] Jing Zhao commented on HDFS-8058: - Thanks for the update, Zhe. +1 for the 09 patch. {{test

[jira] [Created] (HDFS-8777) Erasure Coding: add tests for taking snapshots on EC files

2015-07-14 Thread Jing Zhao (JIRA)
Jing Zhao created HDFS-8777: --- Summary: Erasure Coding: add tests for taking snapshots on EC files Key: HDFS-8777 URL: https://issues.apache.org/jira/browse/HDFS-8777 Project: Hadoop HDFS Issue Type

[jira] [Commented] (HDFS-8058) Erasure coding: use BlockInfo[] for both striped and contiguous blocks in INodeFile

2015-07-14 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14627112#comment-14627112 ] Jing Zhao commented on HDFS-8058: - Created HDFS-8777 to add more tests for (snapshot + EC).

[jira] [Commented] (HDFS-8058) Erasure coding: use BlockInfo[] for both striped and contiguous blocks in INodeFile

2015-07-14 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14627180#comment-14627180 ] Jing Zhao commented on HDFS-8058: - Looks like the timeout of TestDFSStripedOutputStreamWith

[jira] [Updated] (HDFS-8619) Erasure Coding: revisit replica counting for striped blocks

2015-07-14 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8619: Attachment: HDFS-8619-HDFS-7285.001.patch Update the patch. Instead of tracking BlockInfo in CorruptReplicasM

[jira] [Commented] (HDFS-8777) Erasure Coding: add tests for taking snapshots on EC files

2015-07-14 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14627500#comment-14627500 ] Jing Zhao commented on HDFS-8777: - Yes. Let me assign this jira to you. > Erasure Coding:

[jira] [Updated] (HDFS-8777) Erasure Coding: add tests for taking snapshots on EC files

2015-07-14 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8777: Assignee: Rakesh R > Erasure Coding: add tests for taking snapshots on EC files > ---

[jira] [Commented] (HDFS-8433) blockToken is not set in constructInternalBlock and parseStripedBlockGroup in StripedBlockUtil

2015-07-15 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14628459#comment-14628459 ] Jing Zhao commented on HDFS-8433: - In 03 patch, when checking the block token, the {{BlockT

[jira] [Updated] (HDFS-8760) Erasure Coding: reuse BlockReader when reading the same block in pread

2015-07-15 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8760: Attachment: HDFS-8760.000.patch Initial patch. Now pread and stateful read implementation has less difference

[jira] [Commented] (HDFS-8433) blockToken is not set in constructInternalBlock and parseStripedBlockGroup in StripedBlockUtil

2015-07-15 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14628868#comment-14628868 ] Jing Zhao commented on HDFS-8433: - For striped block where range is necessary, the range is

[jira] [Commented] (HDFS-8483) Erasure coding: test DataNode reporting bad/corrupted blocks which belongs to a striped block.

2015-07-15 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14628870#comment-14628870 ] Jing Zhao commented on HDFS-8483: - HDFS-8619 only adds test covering client side reporting.

[jira] [Commented] (HDFS-8787) Erasure coding: rename BlockInfoContiguousUC and BlockInfoStripedUC to be consistent with trunk

2015-07-15 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14628917#comment-14628917 ] Jing Zhao commented on HDFS-8787: - +1 pending Jenkins. > Erasure coding: rename BlockInfoC

[jira] [Updated] (HDFS-8760) Erasure Coding: reuse BlockReader when reading the same block in pread

2015-07-15 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8760: Attachment: (was: HDFS-8760.000.patch) > Erasure Coding: reuse BlockReader when reading the same block in

[jira] [Updated] (HDFS-8760) Erasure Coding: reuse BlockReader when reading the same block in pread

2015-07-15 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8760: Attachment: HDFS-8760.000.patch > Erasure Coding: reuse BlockReader when reading the same block in pread > --

[jira] [Commented] (HDFS-8764) Generate Hadoop RPC stubs from protobuf definitions

2015-07-16 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14630127#comment-14630127 ] Jing Zhao commented on HDFS-8764: - +1 > Generate Hadoop RPC stubs from protobuf definition

[jira] [Commented] (HDFS-8433) blockToken is not set in constructInternalBlock and parseStripedBlockGroup in StripedBlockUtil

2015-07-16 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14630215#comment-14630215 ] Jing Zhao commented on HDFS-8433: - The 04 patch still has issue: the old DN cannot parse th

[jira] [Commented] (HDFS-8728) Erasure coding: revisit and simplify BlockInfoStriped and INodeFile

2015-07-16 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14630343#comment-14630343 ] Jing Zhao commented on HDFS-8728: - If {{EC+repl}} refers to multiple replicas of internal b

[jira] [Commented] (HDFS-8781) Erasure Coding: Correctly handle BlockManager#InvalidateBlocks for striped block

2015-07-17 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14631649#comment-14631649 ] Jing Zhao commented on HDFS-8781: - The patch looks good to me as well. Also agree with Walt

[jira] [Commented] (HDFS-8433) blockToken is not set in constructInternalBlock and parseStripedBlockGroup in StripedBlockUtil

2015-07-17 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14632021#comment-14632021 ] Jing Zhao commented on HDFS-8433: - The 05 patch looks good to me. Maybe we can skip the tes

[jira] [Commented] (HDFS-8499) Refactor BlockInfo class hierarchy with static helper class

2015-07-17 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14632067#comment-14632067 ] Jing Zhao commented on HDFS-8499: - Thanks for sharing the thoughts, Nicholas! bq. We shoul

[jira] [Created] (HDFS-8797) WebHdfsFileSystem creates too many connections for pread

2015-07-17 Thread Jing Zhao (JIRA)
Jing Zhao created HDFS-8797: --- Summary: WebHdfsFileSystem creates too many connections for pread Key: HDFS-8797 URL: https://issues.apache.org/jira/browse/HDFS-8797 Project: Hadoop HDFS Issue Type:

[jira] [Commented] (HDFS-8797) WebHdfsFileSystem creates too many connections for pread

2015-07-17 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14632158#comment-14632158 ] Jing Zhao commented on HDFS-8797: - Simply removing "seek(oldPos)" can avoid the issue but a

[jira] [Assigned] (HDFS-8797) WebHdfsFileSystem creates too many connections for pread

2015-07-17 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao reassigned HDFS-8797: --- Assignee: Jing Zhao > WebHdfsFileSystem creates too many connections for pread > -

[jira] [Updated] (HDFS-8760) Erasure Coding: reuse BlockReader when reading the same block in pread

2015-07-20 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8760: Status: Patch Available (was: Open) > Erasure Coding: reuse BlockReader when reading the same block in pread

[jira] [Updated] (HDFS-8760) Erasure Coding: reuse BlockReader when reading the same block in pread

2015-07-20 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8760: Attachment: HDFS-8760-HDFS-7285.001.patch Thanks for the review, Walter! Update the patch to address all your

[jira] [Commented] (HDFS-8059) Erasure coding: revisit how to store EC schema and cellSize in NameNode

2015-07-20 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14634198#comment-14634198 ] Jing Zhao commented on HDFS-8059: - bq. Breaking rename as I stated above is a huge limitati

[jira] [Updated] (HDFS-8797) WebHdfsFileSystem creates too many connections for pread

2015-07-20 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8797: Status: Patch Available (was: Open) > WebHdfsFileSystem creates too many connections for pread > ---

[jira] [Updated] (HDFS-8797) WebHdfsFileSystem creates too many connections for pread

2015-07-20 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8797: Attachment: HDFS-8797.000.patch One possible way to fix can be to override the {{readFully}} method in {{Byt

[jira] [Updated] (HDFS-8797) WebHdfsFileSystem creates too many connections for pread

2015-07-21 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8797: Attachment: HDFS-8797.001.patch Thanks for the review, Yi and Nicholas! Update the patch to address comments.

[jira] [Commented] (HDFS-8760) Erasure Coding: reuse BlockReader when reading the same block in pread

2015-07-21 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14635972#comment-14635972 ] Jing Zhao commented on HDFS-8760: - Yeah, looks like the Jenkins had some issue. I will comm

[jira] [Updated] (HDFS-8760) Erasure Coding: reuse BlockReader when reading the same block in pread

2015-07-21 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8760: Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: HDFS-7285 Status: Resolved (wa

[jira] [Updated] (HDFS-8797) WebHdfsFileSystem creates too many connections for pread

2015-07-21 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8797: Attachment: HDFS-8797.002.patch Update the patch to add the check suggested by Nicholas. Also added a unit te

[jira] [Commented] (HDFS-8781) Erasure Coding: Correctly handle BlockManager#InvalidateBlocks for striped block

2015-07-21 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14636037#comment-14636037 ] Jing Zhao commented on HDFS-8781: - Thanks for updating the patch, Yi! The new patch looks g

[jira] [Updated] (HDFS-8781) Erasure Coding: Correctly handle BlockManager#InvalidateBlocks for striped block

2015-07-21 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8781: Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: HDFS-7285 Status: Resolved (wa

[jira] [Created] (HDFS-8804) Erasure Coding: use DirectBufferPool in DFSStripedInputStream for buffer allocation

2015-07-21 Thread Jing Zhao (JIRA)
Jing Zhao created HDFS-8804: --- Summary: Erasure Coding: use DirectBufferPool in DFSStripedInputStream for buffer allocation Key: HDFS-8804 URL: https://issues.apache.org/jira/browse/HDFS-8804 Project: Hadoop

[jira] [Updated] (HDFS-8804) Erasure Coding: use DirectBufferPool in DFSStripedInputStream for buffer allocation

2015-07-21 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8804: Attachment: HDFS-8804.000.patch Upload the initial patch. > Erasure Coding: use DirectBufferPool in DFSStrip

[jira] [Commented] (HDFS-8483) Erasure coding: test DataNode reporting bad/corrupted blocks which belongs to a striped block.

2015-07-21 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14636091#comment-14636091 ] Jing Zhao commented on HDFS-8483: - Yeah, sure. Please feel free to close the jira. Thanks f

[jira] [Commented] (HDFS-8805) Archival Storage: it is unable to get StoragePolicy of a directory

2015-07-22 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14637225#comment-14637225 ] Jing Zhao commented on HDFS-8805: - Thanks for reporting the issue, Hui! The reason of the i

[jira] [Updated] (HDFS-8797) WebHdfsFileSystem creates too many connections for pread

2015-07-22 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8797: Attachment: HDFS-8797.003.patch Yeah, it will also be good to keep read/readFully consistent. Update the patc

[jira] [Commented] (HDFS-7858) Improve HA Namenode Failover detection on the client

2015-07-22 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14637494#comment-14637494 ] Jing Zhao commented on HDFS-7858: - Thanks for updating the patch, [~asuresh]! The current a

[jira] [Updated] (HDFS-8797) WebHdfsFileSystem creates too many connections for pread

2015-07-22 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8797: Resolution: Fixed Fix Version/s: 2.8.0 Status: Resolved (was: Patch Available) I've commit

[jira] [Updated] (HDFS-8813) Erasure Coding: Client no need to decode missing parity blocks

2015-07-23 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8813: Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: HDFS-7285 Status: Resolved (wa

[jira] [Commented] (HDFS-8798) Erasure Coding: fix the retry logic of DFSStripedInputStream

2015-07-23 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14639584#comment-14639584 ] Jing Zhao commented on HDFS-8798: - Thanks for the fix, Walter! The patch looks good to me i

[jira] [Commented] (HDFS-7858) Improve HA Namenode Failover detection on the client

2015-07-23 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14639676#comment-14639676 ] Jing Zhao commented on HDFS-7858: - bq. So, in the case of the ReqHedgingProxy, performFailo

[jira] [Comment Edited] (HDFS-7858) Improve HA Namenode Failover detection on the client

2015-07-23 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14639676#comment-14639676 ] Jing Zhao edited comment on HDFS-7858 at 7/23/15 11:39 PM: --- Thank

[jira] [Commented] (HDFS-8804) Erasure Coding: use DirectBufferPool in DFSStripedInputStream for buffer allocation

2015-07-24 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14640768#comment-14640768 ] Jing Zhao commented on HDFS-8804: - Both {{duplicate}} and {{slice}} only play with the limi

[jira] [Commented] (HDFS-8816) Improve visualization for the Datanode tab in the NN UI

2015-07-24 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14640776#comment-14640776 ] Jing Zhao commented on HDFS-8816: - The latest patch looks good to me. +1. > Improve visual

[jira] [Commented] (HDFS-8805) Archival Storage: getStoragePolicy should not need superuser privilege

2015-07-24 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14640787#comment-14640787 ] Jing Zhao commented on HDFS-8805: - Thanks for working on this, [~brahmareddy]. The patch lo

[jira] [Commented] (HDFS-7858) Improve HA Namenode Failover detection on the client

2015-07-24 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14640857#comment-14640857 ] Jing Zhao commented on HDFS-7858: - Thanks again for updating the patch, [~asuresh]! Some mi

[jira] [Commented] (HDFS-8798) Erasure Coding: fix DFSStripedInputStream/DFSStripedOutputStream re-fetch token when expired

2015-07-24 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14640919#comment-14640919 ] Jing Zhao commented on HDFS-8798: - +1. I will commit the patch shortly. > Erasure Coding:

[jira] [Updated] (HDFS-8798) Erasure Coding: fix DFSStripedInputStream/DFSStripedOutputStream re-fetch token when expired

2015-07-24 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8798: Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: HDFS-7285 Status: Resolved (wa

[jira] [Commented] (HDFS-7858) Improve HA Namenode Failover detection on the client

2015-07-24 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14641067#comment-14641067 ] Jing Zhao commented on HDFS-7858: - bq. I need this to get name of the proxy which was succe

[jira] [Commented] (HDFS-7858) Improve HA Namenode Failover detection on the client

2015-07-27 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14643339#comment-14643339 ] Jing Zhao commented on HDFS-7858: - Thanks for working on this, [~asuresh]! The latest patch

[jira] [Commented] (HDFS-8833) Erasure coding: store EC schema and cell size with INodeFile and eliminate EC zones

2015-07-29 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14646085#comment-14646085 ] Jing Zhao commented on HDFS-8833: - Agree with [~rakeshr] here. Also in general pushing ec s

[jira] [Updated] (HDFS-8804) Erasure Coding: use DirectBufferPool in DFSStripedInputStream for buffer allocation

2015-08-03 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8804: Attachment: HDFS-8804.001.patch Thanks Nicholas for the review! Update the patch to address the comments. I d

[jira] [Commented] (HDFS-8828) Utilize Snapshot diff report to build copy list in distcp

2015-08-03 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14652559#comment-14652559 ] Jing Zhao commented on HDFS-8828: - Thanks for working on this, Yufei! One quick comment is

[jira] [Commented] (HDFS-8828) Utilize Snapshot diff report to build copy list in distcp

2015-08-03 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14652789#comment-14652789 ] Jing Zhao commented on HDFS-8828: - Thanks for the explanation, Yufei! Yes, you're right tha

[jira] [Resolved] (HDFS-8804) Erasure Coding: use DirectBufferPool in DFSStripedInputStream for buffer allocation

2015-08-03 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao resolved HDFS-8804. - Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: HDFS-7285 I've committed this to the fe

<    3   4   5   6   7   8   9   10   11   12   >