[jira] [Commented] (HDFS-9496) Erasure coding: an erasure codec throughput benchmark tool

[ https://issues.apache.org/jira/browse/HDFS-9496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15037240#comment-15037240 ] Hui Zheng commented on HDFS-9496: - HADOOP-11588 is good enough to me.Thanks [~lirui]. > Er

[jira] [Created] (HDFS-9496) Erasure coding: an erasure codec throughput benchmark tool

Hui Zheng created HDFS-9496: --- Summary: Erasure coding: an erasure codec throughput benchmark tool Key: HDFS-9496 URL: https://issues.apache.org/jira/browse/HDFS-9496 Project: Hadoop HDFS Issue Type

[jira] [Updated] (HDFS-8805) Archival Storage: getStoragePolicy should not need superuser privilege

[ https://issues.apache.org/jira/browse/HDFS-8805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Zheng updated HDFS-8805: Summary: Archival Storage: getStoragePolicy should not need superuser privilege (was: Archival Storage: getS

[jira] [Commented] (HDFS-8805) Archival Storage: getStoragePolicy should not need superuser privilege

[ https://issues.apache.org/jira/browse/HDFS-8805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14638056#comment-14638056 ] Hui Zheng commented on HDFS-8805: - Hi [~jingzhao] I tried it by super user then found it wo

[jira] [Updated] (HDFS-8805) Archival Storage: getStoragePolicy

[ https://issues.apache.org/jira/browse/HDFS-8805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Zheng updated HDFS-8805: Summary: Archival Storage: getStoragePolicy (was: Archival Storage: it is unable to get StoragePolicy of a d

[jira] [Created] (HDFS-8805) Archival Storage: it is unable to get StoragePolicy of a directory

Hui Zheng created HDFS-8805: --- Summary: Archival Storage: it is unable to get StoragePolicy of a directory Key: HDFS-8805 URL: https://issues.apache.org/jira/browse/HDFS-8805 Project: Hadoop HDFS I

[jira] [Commented] (HDFS-8468) 2 RPC calls for every file read in DFSClient#open(..) resulting in double Audit log entries

[ https://issues.apache.org/jira/browse/HDFS-8468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14605337#comment-14605337 ] Hui Zheng commented on HDFS-8468: - hi [~vinayrpet] Are you working on this issue? If not pl

[jira] [Updated] (HDFS-7398) Reset cached thread-local FSEditLogOp's on every FSEditLog#logEdit

[ https://issues.apache.org/jira/browse/HDFS-7398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Zheng updated HDFS-7398: Fix Version/s: 2.7.0 > Reset cached thread-local FSEditLogOp's on every FSEditLog#logEdit > -

[jira] [Updated] (HDFS-7398) Reset cached thread-local FSEditLogOp's on every FSEditLog#logEdit

[ https://issues.apache.org/jira/browse/HDFS-7398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Zheng updated HDFS-7398: Fix Version/s: (was: 2.7.0) > Reset cached thread-local FSEditLogOp's on every FSEditLog#logEdit > --

[jira] [Commented] (HDFS-8167) BlockManager.addBlockCollectionWithCheck should check if the block is a striped block

[ https://issues.apache.org/jira/browse/HDFS-8167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14502321#comment-14502321 ] Hui Zheng commented on HDFS-8167: - Hi Walter This jira has been committed. Are you working

[jira] [Commented] (HDFS-8167) BlockManager.addBlockCollectionWithCheck should check if the block is a striped block

[ https://issues.apache.org/jira/browse/HDFS-8167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14499555#comment-14499555 ] Hui Zheng commented on HDFS-8167: - This is because the test case used a StripedId(which < 0

[jira] [Commented] (HDFS-8167) BlockManager.addBlockCollectionWithCheck should check if the block is a striped block

[ https://issues.apache.org/jira/browse/HDFS-8167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14499283#comment-14499283 ] Hui Zheng commented on HDFS-8167: - >> Seems to be a little weird to have a setter and a get

[jira] [Updated] (HDFS-8167) BlockManager.addBlockCollectionWithCheck should check if the block is a striped block

[ https://issues.apache.org/jira/browse/HDFS-8167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Zheng updated HDFS-8167: Attachment: HDFS-8167_001.patch I attached a patch. > BlockManager.addBlockCollectionWithCheck should check

[jira] [Commented] (HDFS-7994) Detect if resevered EC Block ID is already used

[ https://issues.apache.org/jira/browse/HDFS-7994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14499105#comment-14499105 ] Hui Zheng commented on HDFS-7994: - Thanks [~zhz] and [~szetszwo] I will work on [HDFS-8167|

[jira] [Updated] (HDFS-7994) Detect if resevered EC Block ID is already used

[ https://issues.apache.org/jira/browse/HDFS-7994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Zheng updated HDFS-7994: Attachment: HDFS-7994_002.patch Thanks [~szetszwo] for reviewing! Consider we are using BlockManager to manag

[jira] [Updated] (HDFS-7994) Detect if resevered EC Block ID is already used

[ https://issues.apache.org/jira/browse/HDFS-7994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Zheng updated HDFS-7994: Attachment: HDFS-7994_001.patch Hi Nicholas Thank you for clarification. I attached a patch for the purpose.

[jira] [Commented] (HDFS-7994) Detect if resevered EC Block ID is already used

[ https://issues.apache.org/jira/browse/HDFS-7994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14490659#comment-14490659 ] Hui Zheng commented on HDFS-7994: - Suppose that there are some blocks using StripedBlockID

[jira] [Commented] (HDFS-7994) Detect if resevered EC Block ID is already used

[ https://issues.apache.org/jira/browse/HDFS-7994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14487162#comment-14487162 ] Hui Zheng commented on HDFS-7994: - Hi [~szetszwo] I want to confirm my understanding about

[jira] [Updated] (HDFS-8109) ECManager should be able to manage multiple ECSchemas

[ https://issues.apache.org/jira/browse/HDFS-8109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Zheng updated HDFS-8109: Issue Type: Sub-task (was: Improvement) Parent: HDFS-8031 > ECManager should be able to manage multi

[jira] [Created] (HDFS-8109) ECManager should be able to manage multiple ECSchemas

Hui Zheng created HDFS-8109: --- Summary: ECManager should be able to manage multiple ECSchemas Key: HDFS-8109 URL: https://issues.apache.org/jira/browse/HDFS-8109 Project: Hadoop HDFS Issue Type: Imp

[jira] [Commented] (HDFS-7994) Detect if resevered EC Block ID is already used

[ https://issues.apache.org/jira/browse/HDFS-7994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14484823#comment-14484823 ] Hui Zheng commented on HDFS-7994: - Hi Nicholas I would like to work on this issue. > Detec

[jira] [Assigned] (HDFS-7994) Detect if resevered EC Block ID is already used

[ https://issues.apache.org/jira/browse/HDFS-7994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Zheng reassigned HDFS-7994: --- Assignee: Hui Zheng (was: Tsz Wo Nicholas Sze) > Detect if resevered EC Block ID is already used > --

[jira] [Commented] (HDFS-7617) Add unit tests for editlog transactions for EC

[ https://issues.apache.org/jira/browse/HDFS-7617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14481024#comment-14481024 ] Hui Zheng commented on HDFS-7617: - I am sorry for late reply. > Add unit tests for editlog

[jira] [Commented] (HDFS-7617) Add unit tests for editlog transactions for EC

[ https://issues.apache.org/jira/browse/HDFS-7617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14481023#comment-14481023 ] Hui Zheng commented on HDFS-7617: - I am sorry for reply.Thanks [~zhz]. > Add unit tests f

[jira] [Commented] (HDFS-7980) Incremental BlockReport will dramatically slow down the startup of a namenode

[ https://issues.apache.org/jira/browse/HDFS-7980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377756#comment-14377756 ] Hui Zheng commented on HDFS-7980: - Hi [~walter.k.su] dfs.blockreport.initailDelay is 600 (1

[jira] [Commented] (HDFS-7980) Incremental BlockReport will dramatically slow down the startup of a namenode

[ https://issues.apache.org/jira/browse/HDFS-7980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377689#comment-14377689 ] Hui Zheng commented on HDFS-7980: - In our environment there are over 3000 datanodes, 100 mi

[jira] [Created] (HDFS-7980) Incremental BlockReport will dramatically slow down the startup of a namenode

Hui Zheng created HDFS-7980: --- Summary: Incremental BlockReport will dramatically slow down the startup of a namenode Key: HDFS-7980 URL: https://issues.apache.org/jira/browse/HDFS-7980 Project: Hadoop HDFS

[jira] [Updated] (HDFS-7617) Add editlog transactions for EC

[ https://issues.apache.org/jira/browse/HDFS-7617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Zheng updated HDFS-7617: Attachment: HDFS-7617.001.patch > Add editlog transactions for EC > --- > >

[jira] [Commented] (HDFS-7617) Add editlog transactions for EC

[ https://issues.apache.org/jira/browse/HDFS-7617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14375569#comment-14375569 ] Hui Zheng commented on HDFS-7617: - Thanks [~zhz] and [~jingzhao] The implement of this func

[jira] [Work stopped] (HDFS-7827) Erasure Coding: support striped blocks in non-protobuf fsimage

[ https://issues.apache.org/jira/browse/HDFS-7827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-7827 stopped by Hui Zheng. --- > Erasure Coding: support striped blocks in non-protobuf fsimage > -

[jira] [Commented] (HDFS-7827) Erasure Coding: support striped blocks in non-protobuf fsimage

[ https://issues.apache.org/jira/browse/HDFS-7827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14375309#comment-14375309 ] Hui Zheng commented on HDFS-7827: - Hi Jing Thank you for your help.The 004 patch looks good

[jira] [Work started] (HDFS-7617) Add editlog transactions for EC

[ https://issues.apache.org/jira/browse/HDFS-7617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-7617 started by Hui Zheng. --- > Add editlog transactions for EC > --- > > Key:

[jira] [Commented] (HDFS-7617) Add editlog transactions for EC

[ https://issues.apache.org/jira/browse/HDFS-7617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14366788#comment-14366788 ] Hui Zheng commented on HDFS-7617: - Hi Jing Are you going to move the "dataBlockNum" and "pa

[jira] [Work started] (HDFS-7827) Erasure Coding: support striped blocks in non-protobuf fsimage

[ https://issues.apache.org/jira/browse/HDFS-7827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-7827 started by Hui Zheng. --- > Erasure Coding: support striped blocks in non-protobuf fsimage > -

[jira] [Commented] (HDFS-7617) Add editlog transactions for EC

[ https://issues.apache.org/jira/browse/HDFS-7617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14366768#comment-14366768 ] Hui Zheng commented on HDFS-7617: - [HDFS-7749|https://issues.apache.org/jira/browse/HDFS-77

[jira] [Updated] (HDFS-7827) Erasure Coding: support striped blocks in non-protobuf fsimage

[ https://issues.apache.org/jira/browse/HDFS-7827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Zheng updated HDFS-7827: Attachment: HDFS-7827.003.patch > Erasure Coding: support striped blocks in non-protobuf fsimage > --

[jira] [Updated] (HDFS-7827) Erasure Coding: support striped blocks in non-protobuf fsimage

[ https://issues.apache.org/jira/browse/HDFS-7827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Zheng updated HDFS-7827: Attachment: (was: HDFS-7827.001.patch) > Erasure Coding: support striped blocks in non-protobuf fsimage >

[jira] [Updated] (HDFS-7827) Erasure Coding: support striped blocks in non-protobuf fsimage

[ https://issues.apache.org/jira/browse/HDFS-7827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Zheng updated HDFS-7827: Attachment: (was: HDFS-7285.003.patch) > Erasure Coding: support striped blocks in non-protobuf fsimage >

[jira] [Updated] (HDFS-7827) Erasure Coding: support striped blocks in non-protobuf fsimage

[ https://issues.apache.org/jira/browse/HDFS-7827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Zheng updated HDFS-7827: Attachment: HDFS-7285.003.patch Hi Jing I updated the patch,please review it. 1) Corrected the {code}writeIN

[jira] [Created] (HDFS-7949) WebImageViewer need support file size calculation with striped blocks

Hui Zheng created HDFS-7949: --- Summary: WebImageViewer need support file size calculation with striped blocks Key: HDFS-7949 URL: https://issues.apache.org/jira/browse/HDFS-7949 Project: Hadoop HDFS

[jira] [Commented] (HDFS-7827) Erasure Coding: support striped blocks in non-protobuf fsimage

[ https://issues.apache.org/jira/browse/HDFS-7827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14362863#comment-14362863 ] Hui Zheng commented on HDFS-7827: - Thanks [~wheat9] Is there any plan to remove the non-pro

[jira] [Commented] (HDFS-7827) Erasure Coding: support striped blocks in non-protobuf fsimage

[ https://issues.apache.org/jira/browse/HDFS-7827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14362853#comment-14362853 ] Hui Zheng commented on HDFS-7827: - I have add a testcase for saving and loading a INodeFile

[jira] [Updated] (HDFS-7827) Erasure Coding: support striped blocks in non-protobuf fsimage

[ https://issues.apache.org/jira/browse/HDFS-7827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Zheng updated HDFS-7827: Attachment: HDFS-7827.002.patch > Erasure Coding: support striped blocks in non-protobuf fsimage > --

[jira] [Updated] (HDFS-7827) Erasure Coding: support striped blocks in non-protobuf fsimage

[ https://issues.apache.org/jira/browse/HDFS-7827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Zheng updated HDFS-7827: Attachment: HDFS-7827.001.patch Hi Jing I updated the patch. But how should we test it without EC client? >

[jira] [Commented] (HDFS-7827) Erasure Coding: support striped blocks in non-protobuf fsimage

[ https://issues.apache.org/jira/browse/HDFS-7827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14358398#comment-14358398 ] Hui Zheng commented on HDFS-7827: - Hi Jing I made a mistake and found that the INodeFile d

[jira] [Assigned] (HDFS-7617) Add editlog transactions for EC

[ https://issues.apache.org/jira/browse/HDFS-7617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Zheng reassigned HDFS-7617: --- Assignee: Hui Zheng (was: Hui Zheng) > Add editlog transactions for EC >

[jira] [Commented] (HDFS-7827) Erasure Coding: support striped blocks in non-protobuf fsimage

[ https://issues.apache.org/jira/browse/HDFS-7827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14356515#comment-14356515 ] Hui Zheng commented on HDFS-7827: - Hi Jing I agree that we need a flag to identify the stri

[jira] [Commented] (HDFS-7827) Erasure Coding: support striped blocks in non-protobuf fsimage

[ https://issues.apache.org/jira/browse/HDFS-7827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14354549#comment-14354549 ] Hui Zheng commented on HDFS-7827: - I think we don't need to implement the loading of a non-

[jira] [Updated] (HDFS-7827) Erasure Coding: support striped blocks in non-protobuf fsimage

[ https://issues.apache.org/jira/browse/HDFS-7827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Zheng updated HDFS-7827: Attachment: HDFS-7827.000.patch Hi Jing I upload the patch which override the write method of BlockInfoStripe

[jira] [Commented] (HDFS-7827) Erasure Coding: support striped blocks in non-protobuf fsimage

[ https://issues.apache.org/jira/browse/HDFS-7827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14352435#comment-14352435 ] Hui Zheng commented on HDFS-7827: - To support striped block in no-protobuf fsimage we only

[jira] [Commented] (HDFS-7827) Erasure Coding: support striped blocks in non-protobuf fsimage

[ https://issues.apache.org/jira/browse/HDFS-7827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14336117#comment-14336117 ] Hui Zheng commented on HDFS-7827: - Hi Jing I would like to work on this jira. Could you ass

[jira] [Assigned] (HDFS-7749) Erasure Coding: Add striped block support in INodeFile

[ https://issues.apache.org/jira/browse/HDFS-7749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hui Zheng reassigned HDFS-7749: --- Assignee: Hui Zheng (was: Jing Zhao) > Erasure Coding: Add striped block support in INodeFile > -