[jira] [Updated] (HDFS-8792) Use LightWeightLinkedSet for BlockManager#postponedMisreplicatedBlocks

[ https://issues.apache.org/jira/browse/HDFS-8792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-8792: - Status: Patch Available (was: Open) > Use LightWeightLinkedSet for BlockManager#postponedMisreplicatedBlocks > ---

[jira] [Updated] (HDFS-8792) Use LightWeightLinkedSet for BlockManager#postponedMisreplicatedBlocks

[ https://issues.apache.org/jira/browse/HDFS-8792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-8792: - Attachment: HDFS-8792.001.patch > Use LightWeightLinkedSet for BlockManager#postponedMisreplicatedBlocks >

[jira] [Created] (HDFS-8792) Use LightWeightLinkedSet for BlockManager#postponedMisreplicatedBlocks

Yi Liu created HDFS-8792: Summary: Use LightWeightLinkedSet for BlockManager#postponedMisreplicatedBlocks Key: HDFS-8792 URL: https://issues.apache.org/jira/browse/HDFS-8792 Project: Hadoop HDFS Iss

[jira] [Commented] (HDFS-8608) Merge HDFS-7912 to trunk and branch-2 (track BlockInfo instead of Block in UnderReplicatedBlocks and PendingReplicationBlocks)

[ https://issues.apache.org/jira/browse/HDFS-8608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14630828#comment-14630828 ] Zhe Zhang commented on HDFS-8608: - [~szetszwo]: To help track original dev history I can li

[jira] [Commented] (HDFS-8779) WebUI can't display randomly generated block ID

[ https://issues.apache.org/jira/browse/HDFS-8779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14630815#comment-14630815 ] Hadoop QA commented on HDFS-8779: - \\ \\ | (/) *{color:green}+1 overall{color}* | \\ \\ ||

[jira] [Updated] (HDFS-8779) WebUI can't display randomly generated block ID

[ https://issues.apache.org/jira/browse/HDFS-8779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-8779: Attachment: HDFS-8779.03.patch Uploaded 03 patch. I replace the response body before parsing it to json. 03 p

[jira] [Commented] (HDFS-8728) Erasure coding: revisit and simplify BlockInfoStriped and INodeFile

[ https://issues.apache.org/jira/browse/HDFS-8728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14630804#comment-14630804 ] Kai Sasaki commented on HDFS-8728: -- [~zhz] Thank you for informing me. As mentioned [her

[jira] [Commented] (HDFS-8688) replace shouldCheckForEnoughRacks with hasClusterEverBeenMultiRack

[ https://issues.apache.org/jira/browse/HDFS-8688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14630746#comment-14630746 ] Walter Su commented on HDFS-8688: - Sorry, My previous comment has big mistake. -line 2707 t

[jira] [Commented] (HDFS-8387) Erasure Coding: Revisit the long and int datatypes usage in striping logic

[ https://issues.apache.org/jira/browse/HDFS-8387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14630727#comment-14630727 ] Rakesh R commented on HDFS-8387: Agreed, I've moved under HDFS-8031 > Erasure Coding: Revi

[jira] [Updated] (HDFS-8387) Erasure Coding: Revisit the long and int datatypes usage in striping logic

[ https://issues.apache.org/jira/browse/HDFS-8387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R updated HDFS-8387: --- Parent Issue: HDFS-8031 (was: HDFS-7285) > Erasure Coding: Revisit the long and int datatypes usage in stripin

[jira] [Commented] (HDFS-8688) replace shouldCheckForEnoughRacks with hasClusterEverBeenMultiRack

[ https://issues.apache.org/jira/browse/HDFS-8688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14630723#comment-14630723 ] Walter Su commented on HDFS-8688: - By the way, the patch changes the logic in the special c

[jira] [Commented] (HDFS-8688) replace shouldCheckForEnoughRacks with hasClusterEverBeenMultiRack

[ https://issues.apache.org/jira/browse/HDFS-8688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14630715#comment-14630715 ] Walter Su commented on HDFS-8688: - bq. There are several other places that call blockHasEno

[jira] [Commented] (HDFS-8688) replace shouldCheckForEnoughRacks with hasClusterEverBeenMultiRack

[ https://issues.apache.org/jira/browse/HDFS-8688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14630680#comment-14630680 ] Walter Su commented on HDFS-8688: - bq. So in the case all replicas are in decommission_inpr

[jira] [Commented] (HDFS-7858) Improve HA Namenode Failover detection on the client

[ https://issues.apache.org/jira/browse/HDFS-7858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14630673#comment-14630673 ] Hadoop QA commented on HDFS-7858: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vo

[jira] [Commented] (HDFS-8688) replace shouldCheckForEnoughRacks with hasClusterEverBeenMultiRack

[ https://issues.apache.org/jira/browse/HDFS-8688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14630671#comment-14630671 ] Ming Ma commented on HDFS-8688: --- Thanks [~walter.k.su]. Your explanation makes sense. But I s

[jira] [Commented] (HDFS-8772) fix TestStandbyIsHot#testDatanodeRestarts which occasionally fails

[ https://issues.apache.org/jira/browse/HDFS-8772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14630656#comment-14630656 ] Hadoop QA commented on HDFS-8772: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vo

[jira] [Commented] (HDFS-8781) Erasure Coding: Correctly handle BlockManager#InvalidateBlocks for striped block

[ https://issues.apache.org/jira/browse/HDFS-8781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14630612#comment-14630612 ] Walter Su commented on HDFS-8781: - Overall it looks good. BTW, I saw TestAddStripedBlocks#t

[jira] [Commented] (HDFS-8608) Merge HDFS-7912 to trunk and branch-2 (track BlockInfo instead of Block in UnderReplicatedBlocks and PendingReplicationBlocks)

[ https://issues.apache.org/jira/browse/HDFS-8608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14630594#comment-14630594 ] Tsz Wo Nicholas Sze commented on HDFS-8608: --- > ... I think we would all agree tha

[jira] [Commented] (HDFS-8608) Merge HDFS-7912 to trunk and branch-2 (track BlockInfo instead of Block in UnderReplicatedBlocks and PendingReplicationBlocks)

[ https://issues.apache.org/jira/browse/HDFS-8608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14630587#comment-14630587 ] Andrew Wang commented on HDFS-8608: --- Hi Nicholas, could you provide some usecases where t

[jira] [Commented] (HDFS-7314) When the DFSClient lease cannot be renewed, abort open-for-write files rather than the entire DFSClient

[ https://issues.apache.org/jira/browse/HDFS-7314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14630577#comment-14630577 ] Hudson commented on HDFS-7314: -- FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #247 (See [htt

[jira] [Commented] (HDFS-7483) Display information per tier on the Namenode UI

[ https://issues.apache.org/jira/browse/HDFS-7483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14630563#comment-14630563 ] Benoy Antony commented on HDFS-7483: In the existing use cases of fmt_percentage, the p

[jira] [Commented] (HDFS-8608) Merge HDFS-7912 to trunk and branch-2 (track BlockInfo instead of Block in UnderReplicatedBlocks and PendingReplicationBlocks)

[ https://issues.apache.org/jira/browse/HDFS-8608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14630533#comment-14630533 ] Tsz Wo Nicholas Sze commented on HDFS-8608: --- Please don't make up a new git histo

[jira] [Commented] (HDFS-8791) block ID-based DN storage layout can be very slow for datanode on ext4

[ https://issues.apache.org/jira/browse/HDFS-8791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14630517#comment-14630517 ] Andrew Wang commented on HDFS-8791: --- [~nroberts] is it possible you could check how many

[jira] [Commented] (HDFS-8608) Merge HDFS-7912 to trunk and branch-2 (track BlockInfo instead of Block in UnderReplicatedBlocks and PendingReplicationBlocks)

[ https://issues.apache.org/jira/browse/HDFS-8608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14630511#comment-14630511 ] Andrew Wang commented on HDFS-8608: --- If the worry is about the original commit history, w

[jira] [Commented] (HDFS-8059) Erasure coding: revisit how to store EC schema and cellSize in NameNode

[ https://issues.apache.org/jira/browse/HDFS-8059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14630506#comment-14630506 ] Andrew Wang commented on HDFS-8059: --- Well, we keep replication in the namespace and use i

[jira] [Commented] (HDFS-8772) fix TestStandbyIsHot#testDatanodeRestarts which occasionally fails

[ https://issues.apache.org/jira/browse/HDFS-8772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14630503#comment-14630503 ] Andrew Wang commented on HDFS-8772: --- Hey Walter, We have this function FSNamesystem#getN

[jira] [Commented] (HDFS-8344) NameNode doesn't recover lease for files with missing blocks

[ https://issues.apache.org/jira/browse/HDFS-8344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14630501#comment-14630501 ] Ravi Prakash commented on HDFS-8344: The retry time is a reflection of how long the adm

[jira] [Commented] (HDFS-8688) replace shouldCheckForEnoughRacks with hasClusterEverBeenMultiRack

[ https://issues.apache.org/jira/browse/HDFS-8688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14630494#comment-14630494 ] Walter Su commented on HDFS-8688: - if live nodes less than expected, we don't call {{blockH

[jira] [Commented] (HDFS-8791) block ID-based DN storage layout can be very slow for datanode on ext4

[ https://issues.apache.org/jira/browse/HDFS-8791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14630493#comment-14630493 ] Nathan Roberts commented on HDFS-8791: -- For reference, the stack trace when du is obvi

[jira] [Created] (HDFS-8791) block ID-based DN storage layout can be very slow for datanode on ext4

Nathan Roberts created HDFS-8791: Summary: block ID-based DN storage layout can be very slow for datanode on ext4 Key: HDFS-8791 URL: https://issues.apache.org/jira/browse/HDFS-8791 Project: Hadoop HD

[jira] [Commented] (HDFS-8767) RawLocalFileSystem.listStatus() returns null for UNIX pipefile

[ https://issues.apache.org/jira/browse/HDFS-8767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14630462#comment-14630462 ] Hudson commented on HDFS-8767: -- FAILURE: Integrated in Hadoop-trunk-Commit #8175 (See [https:

[jira] [Updated] (HDFS-8767) RawLocalFileSystem.listStatus() returns null for UNIX pipefile

[ https://issues.apache.org/jira/browse/HDFS-8767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haohui Mai updated HDFS-8767: - Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.7.2 Status: Resolved (was:

[jira] [Updated] (HDFS-8772) fix TestStandbyIsHot#testDatanodeRestarts which occasionally fails

[ https://issues.apache.org/jira/browse/HDFS-8772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-8772: Attachment: HDFS-8772.02.patch Thanks [~andrew.wang] for review! >... could we wait for that startup FBR... T

[jira] [Commented] (HDFS-8767) RawLocalFileSystem.listStatus() returns null for UNIX pipefile

[ https://issues.apache.org/jira/browse/HDFS-8767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14630450#comment-14630450 ] Haohui Mai commented on HDFS-8767: -- +1. I'll commit it shortly. > RawLocalFileSystem.list

[jira] [Commented] (HDFS-8344) NameNode doesn't recover lease for files with missing blocks

[ https://issues.apache.org/jira/browse/HDFS-8344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14630439#comment-14630439 ] Haohui Mai commented on HDFS-8344: -- A quick question: What would make a good value of the

[jira] [Commented] (HDFS-8059) Erasure coding: revisit how to store EC schema and cellSize in NameNode

[ https://issues.apache.org/jira/browse/HDFS-8059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14630433#comment-14630433 ] Haohui Mai commented on HDFS-8059: -- -1 EC schema / cellSize are information at the storag

[jira] [Commented] (HDFS-8344) NameNode doesn't recover lease for files with missing blocks

[ https://issues.apache.org/jira/browse/HDFS-8344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14630425#comment-14630425 ] Hadoop QA commented on HDFS-8344: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vo

[jira] [Commented] (HDFS-8059) Erasure coding: revisit how to store EC schema and cellSize in NameNode

[ https://issues.apache.org/jira/browse/HDFS-8059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14630418#comment-14630418 ] Andrew Wang commented on HDFS-8059: --- Sorry for coming to this issue late. My feeling righ

[jira] [Commented] (HDFS-8608) Merge HDFS-7912 to trunk and branch-2 (track BlockInfo instead of Block in UnderReplicatedBlocks and PendingReplicationBlocks)

[ https://issues.apache.org/jira/browse/HDFS-8608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14630420#comment-14630420 ] Tsz Wo Nicholas Sze commented on HDFS-8608: --- > ..., a clean git history helps bac

[jira] [Commented] (HDFS-8688) replace shouldCheckForEnoughRacks with hasClusterEverBeenMultiRack

[ https://issues.apache.org/jira/browse/HDFS-8688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14630417#comment-14630417 ] Ming Ma commented on HDFS-8688: --- Thanks [~walter.k.su] for the investigation. According to th

[jira] [Commented] (HDFS-8608) Merge HDFS-7912 to trunk and branch-2 (track BlockInfo instead of Block in UnderReplicatedBlocks and PendingReplicationBlocks)

[ https://issues.apache.org/jira/browse/HDFS-8608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14630411#comment-14630411 ] Zhe Zhang commented on HDFS-8608: - bq. 150+ commits should not be a problem for git. Techni

[jira] [Updated] (HDFS-7858) Improve HA Namenode Failover detection on the client

[ https://issues.apache.org/jira/browse/HDFS-7858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Suresh updated HDFS-7858: -- Attachment: HDFS-7858.4.patch Updating patch: Modified failure handling based on [~jingzhao]'s comments.

[jira] [Commented] (HDFS-8608) Merge HDFS-7912 to trunk and branch-2 (track BlockInfo instead of Block in UnderReplicatedBlocks and PendingReplicationBlocks)

[ https://issues.apache.org/jira/browse/HDFS-8608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14630383#comment-14630383 ] Tsz Wo Nicholas Sze commented on HDFS-8608: --- > git does make merging easier. But

[jira] [Commented] (HDFS-8608) Merge HDFS-7912 to trunk and branch-2 (track BlockInfo instead of Block in UnderReplicatedBlocks and PendingReplicationBlocks)

[ https://issues.apache.org/jira/browse/HDFS-8608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14630372#comment-14630372 ] Zhe Zhang commented on HDFS-8608: - Thanks for the comment Nicholas. Sorry my wording was a

[jira] [Commented] (HDFS-8728) Erasure coding: revisit and simplify BlockInfoStriped and INodeFile

[ https://issues.apache.org/jira/browse/HDFS-8728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14630343#comment-14630343 ] Jing Zhao commented on HDFS-8728: - If {{EC+repl}} refers to multiple replicas of internal b

[jira] [Commented] (HDFS-8779) WebUI can't display randomly generated block ID

[ https://issues.apache.org/jira/browse/HDFS-8779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14630327#comment-14630327 ] Andrew Wang commented on HDFS-8779: --- Upon thinking about it more, I'm worried about compa

[jira] [Commented] (HDFS-8728) Erasure coding: revisit and simplify BlockInfoStriped and INodeFile

[ https://issues.apache.org/jira/browse/HDFS-8728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14630321#comment-14630321 ] Zhe Zhang commented on HDFS-8728: - Thanks Andrew again for the review! It's really helpful.

[jira] [Created] (HDFS-8790) Add Filesystem level stress tests

James Clampffer created HDFS-8790: - Summary: Add Filesystem level stress tests Key: HDFS-8790 URL: https://issues.apache.org/jira/browse/HDFS-8790 Project: Hadoop HDFS Issue Type: Sub-task

[jira] [Commented] (HDFS-8750) FIleSystem does not honor Configuration.getClassLoader() while loading FileSystem implementations

[ https://issues.apache.org/jira/browse/HDFS-8750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14630310#comment-14630310 ] Hadoop QA commented on HDFS-8750: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vo

[jira] [Commented] (HDFS-8785) TestDistributedFileSystem is failing in trunk

[ https://issues.apache.org/jira/browse/HDFS-8785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14630297#comment-14630297 ] Hadoop QA commented on HDFS-8785: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vo

[jira] [Commented] (HDFS-8788) Implement unit tests for remote block reader in libhdfspp

[ https://issues.apache.org/jira/browse/HDFS-8788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14630287#comment-14630287 ] James Clampffer commented on HDFS-8788: --- Are all tests going to be done through a moc

[jira] [Commented] (HDFS-8764) Generate Hadoop RPC stubs from protobuf definitions

[ https://issues.apache.org/jira/browse/HDFS-8764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14630278#comment-14630278 ] James Clampffer commented on HDFS-8764: --- +1 > Generate Hadoop RPC stubs from protobu

[jira] [Commented] (HDFS-8608) Merge HDFS-7912 to trunk and branch-2 (track BlockInfo instead of Block in UnderReplicatedBlocks and PendingReplicationBlocks)

[ https://issues.apache.org/jira/browse/HDFS-8608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14630265#comment-14630265 ] Tsz Wo Nicholas Sze commented on HDFS-8608: --- > This JIRA aims to merges HDFS-7912

[jira] [Commented] (HDFS-8750) FIleSystem does not honor Configuration.getClassLoader() while loading FileSystem implementations

[ https://issues.apache.org/jira/browse/HDFS-8750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14630262#comment-14630262 ] Steve Loughran commented on HDFS-8750: -- -1 the existing codepath handles the {{config

[jira] [Commented] (HDFS-7314) When the DFSClient lease cannot be renewed, abort open-for-write files rather than the entire DFSClient

[ https://issues.apache.org/jira/browse/HDFS-7314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14630243#comment-14630243 ] Hudson commented on HDFS-7314: -- FAILURE: Integrated in Hadoop-trunk-Commit #8174 (See [https:

[jira] [Updated] (HDFS-7314) When the DFSClient lease cannot be renewed, abort open-for-write files rather than the entire DFSClient

[ https://issues.apache.org/jira/browse/HDFS-7314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated HDFS-7314: -- Resolution: Fixed Fix Version/s: 2.8.0 Status: Resolved (was: Patch Available) I have committe

[jira] [Commented] (HDFS-8433) blockToken is not set in constructInternalBlock and parseStripedBlockGroup in StripedBlockUtil

[ https://issues.apache.org/jira/browse/HDFS-8433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14630215#comment-14630215 ] Jing Zhao commented on HDFS-8433: - The 04 patch still has issue: the old DN cannot parse th

[jira] [Commented] (HDFS-8728) Erasure coding: revisit and simplify BlockInfoStriped and INodeFile

[ https://issues.apache.org/jira/browse/HDFS-8728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14630214#comment-14630214 ] Tsz Wo Nicholas Sze commented on HDFS-8728: --- > ... The latest patch pretty much i

[jira] [Commented] (HDFS-8776) Decom manager should not be active on standby

[ https://issues.apache.org/jira/browse/HDFS-8776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14630185#comment-14630185 ] Andrew Wang commented on HDFS-8776: --- Related question to Ming's, this means node state wi

[jira] [Updated] (HDFS-8789) Block Placement policy migrator

[ https://issues.apache.org/jira/browse/HDFS-8789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Trezzo updated HDFS-8789: --- Description: As we start to add new block placement policies to HDFS, it will be necessary to have a ro

[jira] [Updated] (HDFS-8750) FIleSystem does not honor Configuration.getClassLoader() while loading FileSystem implementations

[ https://issues.apache.org/jira/browse/HDFS-8750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kanaka kumar avvaru updated HDFS-8750: -- Status: Patch Available (was: Open) > FIleSystem does not honor Configuration.getClassLo

[jira] [Commented] (HDFS-8772) fix TestStandbyIsHot#testDatanodeRestarts which occasionally fails

[ https://issues.apache.org/jira/browse/HDFS-8772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14630166#comment-14630166 ] Andrew Wang commented on HDFS-8772: --- Hey Walter, the DN will send FBRs on startup right?

[jira] [Commented] (HDFS-8750) FIleSystem does not honor Configuration.getClassLoader() while loading FileSystem implementations

[ https://issues.apache.org/jira/browse/HDFS-8750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14630169#comment-14630169 ] kanaka kumar avvaru commented on HDFS-8750: --- Thanks for the updated patch [~himan

[jira] [Commented] (HDFS-8789) Block Placement policy migrator

[ https://issues.apache.org/jira/browse/HDFS-8789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14630162#comment-14630162 ] Chris Trezzo commented on HDFS-8789: I have an initial patch for a migrator that we can

[jira] [Updated] (HDFS-8785) TestDistributedFileSystem is failing in trunk

[ https://issues.apache.org/jira/browse/HDFS-8785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoyu Yao updated HDFS-8785: - Status: Patch Available (was: Open) > TestDistributedFileSystem is failing in trunk >

[jira] [Created] (HDFS-8789) Block Placement policy migrator

Chris Trezzo created HDFS-8789: -- Summary: Block Placement policy migrator Key: HDFS-8789 URL: https://issues.apache.org/jira/browse/HDFS-8789 Project: Hadoop HDFS Issue Type: New Feature

[jira] [Updated] (HDFS-8785) TestDistributedFileSystem is failing in trunk

[ https://issues.apache.org/jira/browse/HDFS-8785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoyu Yao updated HDFS-8785: - Attachment: HDFS-8785.00.patch Based on my local tests in a loop, the write timeout happened between 1003~1

[jira] [Commented] (HDFS-8767) RawLocalFileSystem.listStatus() returns null for UNIX pipefile

[ https://issues.apache.org/jira/browse/HDFS-8767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14630148#comment-14630148 ] Hadoop QA commented on HDFS-8767: - \\ \\ | (/) *{color:green}+1 overall{color}* | \\ \\ ||

[jira] [Commented] (HDFS-7314) When the DFSClient lease cannot be renewed, abort open-for-write files rather than the entire DFSClient

[ https://issues.apache.org/jira/browse/HDFS-7314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14630137#comment-14630137 ] Colin Patrick McCabe commented on HDFS-7314: Thanks, [~mingma]. +1. > When th

[jira] [Commented] (HDFS-8764) Generate Hadoop RPC stubs from protobuf definitions

[ https://issues.apache.org/jira/browse/HDFS-8764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14630127#comment-14630127 ] Jing Zhao commented on HDFS-8764: - +1 > Generate Hadoop RPC stubs from protobuf definition

[jira] [Updated] (HDFS-8344) NameNode doesn't recover lease for files with missing blocks

[ https://issues.apache.org/jira/browse/HDFS-8344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ravi Prakash updated HDFS-8344: --- Attachment: HDFS-8344.06.patch Here's a patch where the number of maximum attempts is configurable. [~

[jira] [Commented] (HDFS-7483) Display information per tier on the Namenode UI

[ https://issues.apache.org/jira/browse/HDFS-7483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14630115#comment-14630115 ] Haohui Mai commented on HDFS-7483: -- bq. As I mentioned, there is no good way of displaying

[jira] [Commented] (HDFS-8735) Inotify : All events classes should implement toString() API.

[ https://issues.apache.org/jira/browse/HDFS-8735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14630096#comment-14630096 ] Surendra Singh Lilhore commented on HDFS-8735: -- [~ajisakaa] Would you please r

[jira] [Commented] (HDFS-8767) RawLocalFileSystem.listStatus() returns null for UNIX pipefile

[ https://issues.apache.org/jira/browse/HDFS-8767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14630070#comment-14630070 ] kanaka kumar avvaru commented on HDFS-8767: --- Updated patch.. Compared to v3, v4

[jira] [Updated] (HDFS-8767) RawLocalFileSystem.listStatus() returns null for UNIX pipefile

[ https://issues.apache.org/jira/browse/HDFS-8767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kanaka kumar avvaru updated HDFS-8767: -- Attachment: HDFS-8767.004.patch > RawLocalFileSystem.listStatus() returns null for UNIX p

[jira] [Commented] (HDFS-8059) Erasure coding: revisit how to store EC schema and cellSize in NameNode

[ https://issues.apache.org/jira/browse/HDFS-8059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14630027#comment-14630027 ] Zhe Zhang commented on HDFS-8059: - Good thoughts Walter. If we don't plan to support too ma

[jira] [Updated] (HDFS-8059) Erasure coding: revisit how to store EC schema and cellSize in NameNode

[ https://issues.apache.org/jira/browse/HDFS-8059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-8059: Summary: Erasure coding: revisit how to store EC schema and cellSize in NameNode (was: Erasure coding: move

[jira] [Commented] (HDFS-8778) TestBlockReportRateLimiting#testLeaseExpiration can deadlock

[ https://issues.apache.org/jira/browse/HDFS-8778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14629727#comment-14629727 ] Hudson commented on HDFS-8778: -- ABORTED: Integrated in Hadoop-Mapreduce-trunk #2204 (See [htt

[jira] [Commented] (HDFS-8778) TestBlockReportRateLimiting#testLeaseExpiration can deadlock

[ https://issues.apache.org/jira/browse/HDFS-8778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14629726#comment-14629726 ] Hudson commented on HDFS-8778: -- ABORTED: Integrated in Hadoop-Hdfs-trunk #2185 (See [https://

[jira] [Commented] (HDFS-8778) TestBlockReportRateLimiting#testLeaseExpiration can deadlock

[ https://issues.apache.org/jira/browse/HDFS-8778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14629731#comment-14629731 ] Hudson commented on HDFS-8778: -- ABORTED: Integrated in Hadoop-Hdfs-trunk-Java8 #246 (See [htt

[jira] [Commented] (HDFS-8688) replace shouldCheckForEnoughRacks with hasClusterEverBeenMultiRack

[ https://issues.apache.org/jira/browse/HDFS-8688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14629702#comment-14629702 ] Walter Su commented on HDFS-8688: - Thanks [~mingma] for review! I checked 《Hadoop: The Def

[jira] [Commented] (HDFS-8093) BP does not exist or is not under Constructionnull

[ https://issues.apache.org/jira/browse/HDFS-8093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14629671#comment-14629671 ] Neill Lima commented on HDFS-8093: -- ./hadoop fsck / Returns what on your case? I am havin

[jira] [Updated] (HDFS-8735) Inotify : All events classes should implement toString() API.

[ https://issues.apache.org/jira/browse/HDFS-8735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Surendra Singh Lilhore updated HDFS-8735: - Status: Patch Available (was: Open) Attached patch , Please review .. > Inotify :

[jira] [Updated] (HDFS-8735) Inotify : All events classes should implement toString() API.

[ https://issues.apache.org/jira/browse/HDFS-8735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Surendra Singh Lilhore updated HDFS-8735: - Attachment: HDFS-8735.patch > Inotify : All events classes should implement toStrin

[jira] [Updated] (HDFS-8781) Erasure Coding: Correctly handle BlockManager#InvalidateBlocks for striped block

[ https://issues.apache.org/jira/browse/HDFS-8781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-8781: - Status: Patch Available (was: Open) > Erasure Coding: Correctly handle BlockManager#InvalidateBlocks for striped

[jira] [Updated] (HDFS-8781) Erasure Coding: Correctly handle BlockManager#InvalidateBlocks for striped block

[ https://issues.apache.org/jira/browse/HDFS-8781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-8781: - Attachment: HDFS-8781-HDFS-7285.001.patch For striped block to invalidate, we track its invalidate internal blocks.

[jira] [Commented] (HDFS-8778) TestBlockReportRateLimiting#testLeaseExpiration can deadlock

[ https://issues.apache.org/jira/browse/HDFS-8778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14629626#comment-14629626 ] Hudson commented on HDFS-8778: -- SUCCESS: Integrated in Hadoop-Yarn-trunk #988 (See [https://b

[jira] [Commented] (HDFS-8778) TestBlockReportRateLimiting#testLeaseExpiration can deadlock

[ https://issues.apache.org/jira/browse/HDFS-8778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14629619#comment-14629619 ] Hudson commented on HDFS-8778: -- SUCCESS: Integrated in Hadoop-Yarn-trunk-Java8 #258 (See [htt

[jira] [Commented] (HDFS-8778) TestBlockReportRateLimiting#testLeaseExpiration can deadlock

[ https://issues.apache.org/jira/browse/HDFS-8778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14629611#comment-14629611 ] Walter Su commented on HDFS-8778: - TestStandbyIsHot failed again. I have filed HDFS-8772 to

[jira] [Updated] (HDFS-8779) WebUI can't display randomly generated block ID

[ https://issues.apache.org/jira/browse/HDFS-8779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-8779: Status: Patch Available (was: Open) > WebUI can't display randomly generated block ID >

[jira] [Updated] (HDFS-8779) WebUI can't display randomly generated block ID

[ https://issues.apache.org/jira/browse/HDFS-8779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-8779: Attachment: after-02-patch.png before.png Sorry, I can't. The JSON data is just a string in H

[jira] [Commented] (HDFS-8622) Implement GETCONTENTSUMMARY operation for WebImageViewer

[ https://issues.apache.org/jira/browse/HDFS-8622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14629531#comment-14629531 ] Jagadesh Kiran N commented on HDFS-8622: Thanks [~ajisakaa] for your review, i wil

[jira] [Updated] (HDFS-8622) Implement GETCONTENTSUMMARY operation for WebImageViewer

[ https://issues.apache.org/jira/browse/HDFS-8622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira AJISAKA updated HDFS-8622: Summary: Implement GETCONTENTSUMMARY operation for WebImageViewer (was: Implement GETCONTENTSUMMARY

[jira] [Commented] (HDFS-8622) Implement GETCONTENTSUMMARY operation for WebImageViewe

[ https://issues.apache.org/jira/browse/HDFS-8622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14629518#comment-14629518 ] Akira AJISAKA commented on HDFS-8622: - Tried your patch. Additional comments: 1. Conten

[jira] [Updated] (HDFS-8433) blockToken is not set in constructInternalBlock and parseStripedBlockGroup in StripedBlockUtil

[ https://issues.apache.org/jira/browse/HDFS-8433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-8433: Attachment: HDFS-8433.04.patch I have another idea. Uploaded 04 patch. I think it's better than 01~03 patch.

[jira] [Commented] (HDFS-8622) Implement GETCONTENTSUMMARY operation for WebImageViewe

[ https://issues.apache.org/jira/browse/HDFS-8622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14629458#comment-14629458 ] Akira AJISAKA commented on HDFS-8622: - Thanks [~jagadesh.kiran] for taking this issue a

[jira] [Commented] (HDFS-6697) Make NN lease soft and hard limits configurable

[ https://issues.apache.org/jira/browse/HDFS-6697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14629444#comment-14629444 ] Hadoop QA commented on HDFS-6697: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vo

[jira] [Updated] (HDFS-8433) blockToken is not set in constructInternalBlock and parseStripedBlockGroup in StripedBlockUtil

[ https://issues.apache.org/jira/browse/HDFS-8433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-8433: Attachment: HDFS-8433.03.txt 03 patch is impossible to be implemented. Please ignore 03 patch. I'll keep the

[jira] [Updated] (HDFS-8433) blockToken is not set in constructInternalBlock and parseStripedBlockGroup in StripedBlockUtil

[ https://issues.apache.org/jira/browse/HDFS-8433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Walter Su updated HDFS-8433: Attachment: (was: HDFS-8433.03.PoC.patch) > blockToken is not set in constructInternalBlock and parseStri