[GitHub] [hadoop] bilaharith commented on a change in pull request #2707: HADOOP-17536. ABFS: Supporting customer provided encryption key

2021-04-15 Thread GitBox
bilaharith commented on a change in pull request #2707: URL: https://github.com/apache/hadoop/pull/2707#discussion_r613989593 ## File path: hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestCustomerProvidedKey.java ## @@ -0,0 +1,937 @@ +/** + * License

[GitHub] [hadoop] bilaharith commented on a change in pull request #2707: HADOOP-17536. ABFS: Supporting customer provided encryption key

2021-04-15 Thread GitBox
bilaharith commented on a change in pull request #2707: URL: https://github.com/apache/hadoop/pull/2707#discussion_r613989746 ## File path: hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestCustomerProvidedKey.java ## @@ -0,0 +1,937 @@ +/** + * License

[GitHub] [hadoop] bilaharith commented on a change in pull request #2707: HADOOP-17536. ABFS: Supporting customer provided encryption key

2021-04-15 Thread GitBox
bilaharith commented on a change in pull request #2707: URL: https://github.com/apache/hadoop/pull/2707#discussion_r613990480 ## File path: hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestCustomerProvidedKey.java ## @@ -0,0 +1,937 @@ +/** + * License

[GitHub] [hadoop] iwasakims merged pull request #2886: HADOOP-17569. Building native code fails on Fedora 33.

2021-04-15 Thread GitBox
iwasakims merged pull request #2886: URL: https://github.com/apache/hadoop/pull/2886 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, pleas

[GitHub] [hadoop] iwasakims commented on pull request #2886: HADOOP-17569. Building native code fails on Fedora 33.

2021-04-15 Thread GitBox
iwasakims commented on pull request #2886: URL: https://github.com/apache/hadoop/pull/2886#issuecomment-820386755 Thanks, @aajisaka. I merged this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

[GitHub] [hadoop] hadoop-yetus commented on pull request #2889: HDFS-15963. Unreleased volume references cause an infinite loop.

2021-04-15 Thread GitBox
hadoop-yetus commented on pull request #2889: URL: https://github.com/apache/hadoop/pull/2889#issuecomment-820392449 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[GitHub] [hadoop] hadoop-yetus commented on pull request #2913: HDFS-15978.Solve DatanodeManager#getBlockRecoveryCommand() printing IOException.

2021-04-15 Thread GitBox
hadoop-yetus commented on pull request #2913: URL: https://github.com/apache/hadoop/pull/2913#issuecomment-820397465 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[GitHub] [hadoop] iwasakims commented on pull request #2914: HDFS-15977. Call explicit_bzero only if it is available.

2021-04-15 Thread GitBox
iwasakims commented on pull request #2914: URL: https://github.com/apache/hadoop/pull/2914#issuecomment-820399933 > I found the latest list: https://ci.bigtop.apache.org/job/Bigtop-trunk-repos/ Yeah. You can see the discussion on [BIGTOP-3471](https://issues.apache.org/jira/browse/B

[GitHub] [hadoop] hadoop-yetus commented on pull request #2737: HDFS-15869. Network issue while FSEditLogAsync is executing RpcEdit.logSyncNotify can cause the namenode to hang

2021-04-15 Thread GitBox
hadoop-yetus commented on pull request #2737: URL: https://github.com/apache/hadoop/pull/2737#issuecomment-820433086 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[GitHub] [hadoop] hadoop-yetus commented on pull request #2520: HADOOP-17290. ABFS: Add Identifiers to Client Request Header

2021-04-15 Thread GitBox
hadoop-yetus commented on pull request #2520: URL: https://github.com/apache/hadoop/pull/2520#issuecomment-820436221 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[GitHub] [hadoop] tasanuma commented on pull request #2896: HDFS-15970. Print network topology on the web

2021-04-15 Thread GitBox
tasanuma commented on pull request #2896: URL: https://github.com/apache/hadoop/pull/2896#issuecomment-820451390 Mostly looks good to me. Is it possible to extend to Router UI as @ayushtkn commented? (maybe another jira?) -- This is an automated message from the Apache Git Service. To re

[GitHub] [hadoop] hadoop-yetus commented on pull request #2907: HDFS-15975. Use LongAdder instead of AtomicLong

2021-04-15 Thread GitBox
hadoop-yetus commented on pull request #2907: URL: https://github.com/apache/hadoop/pull/2907#issuecomment-820500567 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[GitHub] [hadoop] tasanuma commented on a change in pull request #2907: HDFS-15975. Use LongAdder instead of AtomicLong

2021-04-15 Thread GitBox
tasanuma commented on a change in pull request #2907: URL: https://github.com/apache/hadoop/pull/2907#discussion_r614222329 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetCache.java ## @@ -607,11 +608,

[GitHub] [hadoop] tasanuma commented on pull request #2907: HDFS-15975. Use LongAdder instead of AtomicLong

2021-04-15 Thread GitBox
tasanuma commented on pull request #2907: URL: https://github.com/apache/hadoop/pull/2907#issuecomment-820568403 > Nice one. For metrics and statistics, LongAdder provides much better throughput. Agreed with @virajjasani. Thanks for working on this, @tomscut. -- This is an automat

[GitHub] [hadoop] ayushtkn commented on a change in pull request #2833: HDFS-15934: Make DirectoryScanner reconcile blocks batch size and int…

2021-04-15 Thread GitBox
ayushtkn commented on a change in pull request #2833: URL: https://github.com/apache/hadoop/pull/2833#discussion_r614224919 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DirectoryScanner.java ## @@ -315,6 +316,40 @@ public D

[GitHub] [hadoop] virajjasani removed a comment on pull request #2895: HADOOP-17633. Bump json-smart to 2.4.2 and nimbus-jose-jwt to 9.8.1 due to CVEs

2021-04-15 Thread GitBox
virajjasani removed a comment on pull request #2895: URL: https://github.com/apache/hadoop/pull/2895#issuecomment-820126450 Build#8 has timed-out after 20 hr and has these test failures recorded. Pasting screenshots here as the build data will be archived very soon. https://user-imag

[GitHub] [hadoop] hadoop-yetus commented on pull request #2520: HADOOP-17290. ABFS: Add Identifiers to Client Request Header

2021-04-15 Thread GitBox
hadoop-yetus commented on pull request #2520: URL: https://github.com/apache/hadoop/pull/2520#issuecomment-820684101 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :o

[GitHub] [hadoop] hadoop-yetus commented on pull request #2913: HDFS-15978.Solve DatanodeManager#getBlockRecoveryCommand() printing IOException.

2021-04-15 Thread GitBox
hadoop-yetus commented on pull request #2913: URL: https://github.com/apache/hadoop/pull/2913#issuecomment-820749035 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[GitHub] [hadoop] xiaoyuyao merged pull request #2784: HDFS-15850. Superuser actions should be reported to external enforcers

2021-04-15 Thread GitBox
xiaoyuyao merged pull request #2784: URL: https://github.com/apache/hadoop/pull/2784 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, pleas

[GitHub] [hadoop] goiri commented on a change in pull request #2896: HDFS-15970. Print network topology on the web

2021-04-15 Thread GitBox
goiri commented on a change in pull request #2896: URL: https://github.com/apache/hadoop/pull/2896#discussion_r614449369 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NetworkTopologyServlet.java ## @@ -0,0 +1,115 @@ +/** + *

[GitHub] [hadoop] goiri merged pull request #2903: HDFS-15423 RBF: WebHDFS create shouldn't choose DN from all sub-clusters

2021-04-15 Thread GitBox
goiri merged pull request #2903: URL: https://github.com/apache/hadoop/pull/2903 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please co

[GitHub] [hadoop] hadoop-yetus commented on pull request #2895: HADOOP-17633. Bump json-smart to 2.4.2 and nimbus-jose-jwt to 9.8.1 due to CVEs

2021-04-15 Thread GitBox
hadoop-yetus commented on pull request #2895: URL: https://github.com/apache/hadoop/pull/2895#issuecomment-820788402 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[GitHub] [hadoop] hadoop-yetus commented on pull request #2737: HDFS-15869. Network issue while FSEditLogAsync is executing RpcEdit.logSyncNotify can cause the namenode to hang

2021-04-15 Thread GitBox
hadoop-yetus commented on pull request #2737: URL: https://github.com/apache/hadoop/pull/2737#issuecomment-820829126 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[GitHub] [hadoop] tomscut commented on a change in pull request #2907: HDFS-15975. Use LongAdder instead of AtomicLong

2021-04-15 Thread GitBox
tomscut commented on a change in pull request #2907: URL: https://github.com/apache/hadoop/pull/2907#discussion_r614496017 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetCache.java ## @@ -607,11 +608,1

[GitHub] [hadoop] qizhu-lucas commented on pull request #2911: YARN-10737: Fix typos in CapacityScheduler#schedule.

2021-04-15 Thread GitBox
qizhu-lucas commented on pull request #2911: URL: https://github.com/apache/hadoop/pull/2911#issuecomment-820857577 Thanks @fdalsotto for review. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [hadoop] qizhu-lucas commented on pull request #2833: HDFS-15934: Make DirectoryScanner reconcile blocks batch size and int…

2021-04-15 Thread GitBox
qizhu-lucas commented on pull request #2833: URL: https://github.com/apache/hadoop/pull/2833#issuecomment-820872414 Thanks very much for @ayushtkn patient review. I have updated above in latest PR. -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [hadoop] zhuxiangyi opened a new pull request #2915: Fix RBF does not display datanode UI

2021-04-15 Thread GitBox
zhuxiangyi opened a new pull request #2915: URL: https://github.com/apache/hadoop/pull/2915 [HDFS-15974](https://issues.apache.org/jira/browse/HDFS-15974) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [hadoop] aajisaka merged pull request #2914: HDFS-15977. Call explicit_bzero only if it is available.

2021-04-15 Thread GitBox
aajisaka merged pull request #2914: URL: https://github.com/apache/hadoop/pull/2914 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please

[GitHub] [hadoop] aajisaka commented on pull request #2914: HDFS-15977. Call explicit_bzero only if it is available.

2021-04-15 Thread GitBox
aajisaka commented on pull request #2914: URL: https://github.com/apache/hadoop/pull/2914#issuecomment-820897074 Merged. Thank you @GauthamBanasandra, @iwasakims, and @goiri. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub an

[GitHub] [hadoop] hadoop-yetus commented on pull request #2915: HDFS-15974. Fix RBF does not display datanode UI

2021-04-15 Thread GitBox
hadoop-yetus commented on pull request #2915: URL: https://github.com/apache/hadoop/pull/2915#issuecomment-820903604 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[GitHub] [hadoop] langlaile1221 opened a new pull request #2916: HDFS-15981. Removing redundant block queues will slow down block reporting

2021-04-15 Thread GitBox
langlaile1221 opened a new pull request #2916: URL: https://github.com/apache/hadoop/pull/2916 When the block report satisfies the block distribution strategy, the block is removed from the lowredundancyBlocks. But removing the block from the lowredundancyBlocks is a redundant operation.

[GitHub] [hadoop] surendralilhore merged pull request #2885: HADOOP-17576. ABFS: Disable throttling update for auth failures

2021-04-15 Thread GitBox
surendralilhore merged pull request #2885: URL: https://github.com/apache/hadoop/pull/2885 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[GitHub] [hadoop] surendralilhore commented on pull request #2842: HADOOP-17615. ADLS Gen1: Update adls SDK to 2.3.9

2021-04-15 Thread GitBox
surendralilhore commented on pull request #2842: URL: https://github.com/apache/hadoop/pull/2842#issuecomment-820920271 Triggered build again -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [hadoop] jianghuazhu commented on pull request #2913: HDFS-15978.Solve DatanodeManager#getBlockRecoveryCommand() printing IOException.

2021-04-15 Thread GitBox
jianghuazhu commented on pull request #2913: URL: https://github.com/apache/hadoop/pull/2913#issuecomment-820937321 @virajjasani , thank you very much for your comment. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [hadoop] qizhu-lucas commented on pull request #2912: YARN-10738: When multi thread scheduling with multi node, we should s…

2021-04-15 Thread GitBox
qizhu-lucas commented on pull request #2912: URL: https://github.com/apache/hadoop/pull/2912#issuecomment-820942379 Fixed checkstyle in latest PR. @bibinchundatt @jbrennan333 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [hadoop] hadoop-yetus commented on pull request #2860: Test Pre-Commits.

2021-04-15 Thread GitBox
hadoop-yetus commented on pull request #2860: URL: https://github.com/apache/hadoop/pull/2860#issuecomment-820960349 (!) A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at https://ci-hadoop.apac

[GitHub] [hadoop] ayushtkn commented on pull request #2895: HADOOP-17633. Bump json-smart to 2.4.2 and nimbus-jose-jwt to 9.8.1 due to CVEs

2021-04-16 Thread GitBox
ayushtkn commented on pull request #2895: URL: https://github.com/apache/hadoop/pull/2895#issuecomment-820963341 Test failures doesn't seems related, mostly due to timeouts, or are known flaky -- This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [hadoop] ayushtkn merged pull request #2895: HADOOP-17633. Bump json-smart to 2.4.2 and nimbus-jose-jwt to 9.8.1 due to CVEs

2021-04-16 Thread GitBox
ayushtkn merged pull request #2895: URL: https://github.com/apache/hadoop/pull/2895 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please

[GitHub] [hadoop] surendralilhore commented on a change in pull request #2549: Hadoop 17428. ABFS: Implementation for getContentSummary

2021-04-16 Thread GitBox
surendralilhore commented on a change in pull request #2549: URL: https://github.com/apache/hadoop/pull/2549#discussion_r614609260 ## File path: hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AzureBlobFileSystem.java ## @@ -393,6 +397,32 @@ public boolea

[GitHub] [hadoop] jianghuazhu opened a new pull request #2917: HDFS-15985.Incorrect sorting will cause failure to load an FsImage file.

2021-04-16 Thread GitBox
jianghuazhu opened a new pull request #2917: URL: https://github.com/apache/hadoop/pull/2917 ## NOTICE Please create an issue in ASF JIRA before opening a pull request, and you need to set the title of the pull request which starts with the corresponding JIRA issue number. (e.g.

[GitHub] [hadoop] hadoop-yetus commented on pull request #2833: HDFS-15934: Make DirectoryScanner reconcile blocks batch size and int…

2021-04-16 Thread GitBox
hadoop-yetus commented on pull request #2833: URL: https://github.com/apache/hadoop/pull/2833#issuecomment-821010516 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[GitHub] [hadoop] surendralilhore commented on a change in pull request #2549: Hadoop 17428. ABFS: Implementation for getContentSummary

2021-04-16 Thread GitBox
surendralilhore commented on a change in pull request #2549: URL: https://github.com/apache/hadoop/pull/2549#discussion_r614661413 ## File path: hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/ContentSummaryProcessor.java ## @@ -0,0 +1,144 @@ +/*

[GitHub] [hadoop] surendralilhore commented on a change in pull request #2549: Hadoop 17428. ABFS: Implementation for getContentSummary

2021-04-16 Thread GitBox
surendralilhore commented on a change in pull request #2549: URL: https://github.com/apache/hadoop/pull/2549#discussion_r614661413 ## File path: hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/ContentSummaryProcessor.java ## @@ -0,0 +1,144 @@ +/*

[GitHub] [hadoop] surendralilhore commented on a change in pull request #2549: Hadoop 17428. ABFS: Implementation for getContentSummary

2021-04-16 Thread GitBox
surendralilhore commented on a change in pull request #2549: URL: https://github.com/apache/hadoop/pull/2549#discussion_r614671618 ## File path: hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/ContentSummaryProcessor.java ## @@ -0,0 +1,144 @@ +/*

[GitHub] [hadoop] surendralilhore commented on a change in pull request #2549: Hadoop 17428. ABFS: Implementation for getContentSummary

2021-04-16 Thread GitBox
surendralilhore commented on a change in pull request #2549: URL: https://github.com/apache/hadoop/pull/2549#discussion_r614674625 ## File path: hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AzureBlobFileSystem.java ## @@ -393,6 +397,32 @@ public boolea

[GitHub] [hadoop] hadoop-yetus commented on pull request #2845: HADOOP-17618. ABFS: Partially obfuscate SAS object IDs in Logs

2021-04-16 Thread GitBox
hadoop-yetus commented on pull request #2845: URL: https://github.com/apache/hadoop/pull/2845#issuecomment-821037219 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[GitHub] [hadoop] hadoop-yetus commented on pull request #2907: HDFS-15975. Use LongAdder instead of AtomicLong

2021-04-16 Thread GitBox
hadoop-yetus commented on pull request #2907: URL: https://github.com/apache/hadoop/pull/2907#issuecomment-821037653 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[GitHub] [hadoop] tomscut commented on a change in pull request #2907: HDFS-15975. Use LongAdder instead of AtomicLong

2021-04-16 Thread GitBox
tomscut commented on a change in pull request #2907: URL: https://github.com/apache/hadoop/pull/2907#discussion_r614688538 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetCache.java ## @@ -607,11 +608,1

[GitHub] [hadoop] whbing opened a new pull request #2918: HDFS-15987. Improve oiv tool to parse fsimage file in parallel with d…

2021-04-16 Thread GitBox
whbing opened a new pull request #2918: URL: https://github.com/apache/hadoop/pull/2918 …elimited format ## NOTICE Please create an issue in ASF JIRA before opening a pull request, and you need to set the title of the pull request which starts with the corresponding JIRA

[GitHub] [hadoop] Hexiaoqiao commented on pull request #2889: HDFS-15963. Unreleased volume references cause an infinite loop.

2021-04-16 Thread GitBox
Hexiaoqiao commented on pull request #2889: URL: https://github.com/apache/hadoop/pull/2889#issuecomment-821042143 @jojochuang Hi Wei-Chiu, any furthermore comment here? If not, I would like to commit this PR shortly. Thanks. -- This is an automated message from the Apache Git Service.

[GitHub] [hadoop] tasanuma commented on a change in pull request #2907: HDFS-15975. Use LongAdder instead of AtomicLong

2021-04-16 Thread GitBox
tasanuma commented on a change in pull request #2907: URL: https://github.com/apache/hadoop/pull/2907#discussion_r614693047 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetCache.java ## @@ -607,11 +608,

[GitHub] [hadoop] tomscut commented on a change in pull request #2907: HDFS-15975. Use LongAdder instead of AtomicLong

2021-04-16 Thread GitBox
tomscut commented on a change in pull request #2907: URL: https://github.com/apache/hadoop/pull/2907#discussion_r614696047 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetCache.java ## @@ -607,11 +608,1

[GitHub] [hadoop] tomscut commented on pull request #2896: HDFS-15970. Print network topology on the web

2021-04-16 Thread GitBox
tomscut commented on pull request #2896: URL: https://github.com/apache/hadoop/pull/2896#issuecomment-821056074 > Mostly looks good to me. Is it possible to extend to Router UI as @ayushtkn commented? (maybe another jira?) @tasanuma @ayushtkn I'll create another JIRA later to support

[GitHub] [hadoop] hadoop-yetus commented on pull request #2912: YARN-10738: When multi thread scheduling with multi node, we should s…

2021-04-16 Thread GitBox
hadoop-yetus commented on pull request #2912: URL: https://github.com/apache/hadoop/pull/2912#issuecomment-821056361 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[GitHub] [hadoop] jojochuang commented on pull request #2889: HDFS-15963. Unreleased volume references cause an infinite loop.

2021-04-16 Thread GitBox
jojochuang commented on pull request #2889: URL: https://github.com/apache/hadoop/pull/2889#issuecomment-82112 Go ahead. Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [hadoop] amahussein opened a new pull request #2919: HDFS-15979. Move within EZ fails and cannot remove nested EZs. Contri…

2021-04-16 Thread GitBox
amahussein opened a new pull request #2919: URL: https://github.com/apache/hadoop/pull/2919 [HDFS-15979](https://issues.apache.org/jira/browse/HDFS-15979) Move within EZ fails and cannot remove nested EZs The test failures are not related to the changes. Those three tests have relat

[GitHub] [hadoop] hadoop-yetus commented on pull request #2781: HADOOP-17548. ABFS: Toggle Store Mkdirs request overwrite parameter (#2729)

2021-04-16 Thread GitBox
hadoop-yetus commented on pull request #2781: URL: https://github.com/apache/hadoop/pull/2781#issuecomment-821158673 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :o

[GitHub] [hadoop] hadoop-yetus commented on pull request #2916: HDFS-15981. Removing redundant block queues will slow down block reporting

2021-04-16 Thread GitBox
hadoop-yetus commented on pull request #2916: URL: https://github.com/apache/hadoop/pull/2916#issuecomment-821161737 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[GitHub] [hadoop] hadoop-yetus commented on pull request #2860: Test Pre-Commits.

2021-04-16 Thread GitBox
hadoop-yetus commented on pull request #2860: URL: https://github.com/apache/hadoop/pull/2860#issuecomment-821163395 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[GitHub] [hadoop] hadoop-yetus commented on pull request #2917: HDFS-15985.Incorrect sorting will cause failure to load an FsImage file.

2021-04-16 Thread GitBox
hadoop-yetus commented on pull request #2917: URL: https://github.com/apache/hadoop/pull/2917#issuecomment-821165566 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[GitHub] [hadoop] steveloughran opened a new pull request #2920: HADOOP-17535. ABFS: ITestAzureBlobFileSystemCheckAccess test failure if no oauth key.

2021-04-16 Thread GitBox
steveloughran opened a new pull request #2920: URL: https://github.com/apache/hadoop/pull/2920 Contributed by Steve Loughran. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [hadoop] steveloughran commented on pull request #2920: HADOOP-17535. ABFS: ITestAzureBlobFileSystemCheckAccess test failure if no oauth key.

2021-04-16 Thread GitBox
steveloughran commented on pull request #2920: URL: https://github.com/apache/hadoop/pull/2920#issuecomment-821181146 testing: azure cardiff. Test now downgrades to skipping those cases for which oauth creds are required -- This is an automated message from the Apache Git Service. To res

[GitHub] [hadoop] daryn-sharp commented on a change in pull request #2878: HDFS-15957. The ignored IOException in the RPC response sent by FSEditLogAsync can cause the HDFS client to hang

2021-04-16 Thread GitBox
daryn-sharp commented on a change in pull request #2878: URL: https://github.com/apache/hadoop/pull/2878#discussion_r614865039 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogAsync.java ## @@ -378,13 +381,18 @@ public

[GitHub] [hadoop] hadoop-yetus commented on pull request #2860: HDFS-15988. Stabilise HDFS Pre-Commit.

2021-04-16 Thread GitBox
hadoop-yetus commented on pull request #2860: URL: https://github.com/apache/hadoop/pull/2860#issuecomment-821218156 (!) A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at https://ci-hadoop.apac

[GitHub] [hadoop] hadoop-yetus commented on pull request #2918: HDFS-15987. Improve oiv tool to parse fsimage file in parallel with d…

2021-04-16 Thread GitBox
hadoop-yetus commented on pull request #2918: URL: https://github.com/apache/hadoop/pull/2918#issuecomment-821233568 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[GitHub] [hadoop] hadoop-yetus commented on pull request #2920: HADOOP-17535. ABFS: ITestAzureBlobFileSystemCheckAccess test failure if no oauth key.

2021-04-16 Thread GitBox
hadoop-yetus commented on pull request #2920: URL: https://github.com/apache/hadoop/pull/2920#issuecomment-821247866 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :o

[GitHub] [hadoop] goiri commented on a change in pull request #2915: HDFS-15974. RBF: Unable to display the datanode UI of the router

2021-04-16 Thread GitBox
goiri commented on a change in pull request #2915: URL: https://github.com/apache/hadoop/pull/2915#discussion_r614937252 ## File path: hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/metrics/NamenodeBeanMetrics.java ## @@ -855,7 +855,

[GitHub] [hadoop] Hexiaoqiao merged pull request #2889: HDFS-15963. Unreleased volume references cause an infinite loop.

2021-04-16 Thread GitBox
Hexiaoqiao merged pull request #2889: URL: https://github.com/apache/hadoop/pull/2889 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, plea

[GitHub] [hadoop] Hexiaoqiao commented on pull request #2889: HDFS-15963. Unreleased volume references cause an infinite loop.

2021-04-16 Thread GitBox
Hexiaoqiao commented on pull request #2889: URL: https://github.com/apache/hadoop/pull/2889#issuecomment-821284155 Committed to trunk. Thanks @zhangshuyan0 for your works! Thanks @jojochuang for your reviews! Will backport to other active branches for a while. -- This is an automated

[GitHub] [hadoop] tomscut commented on a change in pull request #2896: HDFS-15970. Print network topology on the web

2021-04-16 Thread GitBox
tomscut commented on a change in pull request #2896: URL: https://github.com/apache/hadoop/pull/2896#discussion_r614993153 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NetworkTopologyServlet.java ## @@ -0,0 +1,115 @@ +/** +

[GitHub] [hadoop] hadoop-yetus commented on pull request #2907: HDFS-15975. Use LongAdder instead of AtomicLong

2021-04-16 Thread GitBox
hadoop-yetus commented on pull request #2907: URL: https://github.com/apache/hadoop/pull/2907#issuecomment-821314734 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[GitHub] [hadoop] uzshao opened a new pull request #2921: HDFS-15980: Fix tests for HDFS-15754 Create packet metrics for DataNode

2021-04-16 Thread GitBox
uzshao opened a new pull request #2921: URL: https://github.com/apache/hadoop/pull/2921 See https://issues.apache.org/jira/browse/HDFS-15980 HDFS-15754 introduces 4 new metrics in DataNodeMetrics. However the test associated with the patch has some bugs. This issue is to fix th

[GitHub] [hadoop] hadoop-yetus commented on pull request #2919: HDFS-15979. Move within EZ fails and cannot remove nested EZs. Contri…

2021-04-16 Thread GitBox
hadoop-yetus commented on pull request #2919: URL: https://github.com/apache/hadoop/pull/2919#issuecomment-821354712 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[GitHub] [hadoop] virajjasani opened a new pull request #2922: HADOOP-17642. Remove appender EventCounter to avoid instantiation

2021-04-16 Thread GitBox
virajjasani opened a new pull request #2922: URL: https://github.com/apache/hadoop/pull/2922 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this servic

[GitHub] [hadoop] hadoop-yetus commented on pull request #2860: HDFS-15988. Stabilise HDFS Pre-Commit.

2021-04-16 Thread GitBox
hadoop-yetus commented on pull request #2860: URL: https://github.com/apache/hadoop/pull/2860#issuecomment-821541081 (!) A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at https://ci-hadoop.apac

[GitHub] [hadoop] hadoop-yetus commented on pull request #2860: HDFS-15988. Stabilise HDFS Pre-Commit.

2021-04-16 Thread GitBox
hadoop-yetus commented on pull request #2860: URL: https://github.com/apache/hadoop/pull/2860#issuecomment-821556485 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[GitHub] [hadoop] hadoop-yetus commented on pull request #2918: HDFS-15987. Improve oiv tool to parse fsimage file in parallel with d…

2021-04-16 Thread GitBox
hadoop-yetus commented on pull request #2918: URL: https://github.com/apache/hadoop/pull/2918#issuecomment-821559317 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[GitHub] [hadoop] hadoop-yetus commented on pull request #2896: HDFS-15970. Print network topology on the web

2021-04-16 Thread GitBox
hadoop-yetus commented on pull request #2896: URL: https://github.com/apache/hadoop/pull/2896#issuecomment-821608597 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[GitHub] [hadoop] hadoop-yetus commented on pull request #2922: HADOOP-17642. Remove appender EventCounter to avoid instantiation

2021-04-16 Thread GitBox
hadoop-yetus commented on pull request #2922: URL: https://github.com/apache/hadoop/pull/2922#issuecomment-821687427 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[GitHub] [hadoop] hadoop-yetus commented on pull request #2921: HDFS-15980: Fix tests for HDFS-15754 Create packet metrics for DataNode

2021-04-16 Thread GitBox
hadoop-yetus commented on pull request #2921: URL: https://github.com/apache/hadoop/pull/2921#issuecomment-821743561 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[GitHub] [hadoop] hadoop-yetus commented on pull request #2860: HDFS-15988. Stabilise HDFS Pre-Commit.

2021-04-16 Thread GitBox
hadoop-yetus commented on pull request #2860: URL: https://github.com/apache/hadoop/pull/2860#issuecomment-821753013 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[GitHub] [hadoop] tasanuma merged pull request #2907: HDFS-15975. Use LongAdder instead of AtomicLong

2021-04-17 Thread GitBox
tasanuma merged pull request #2907: URL: https://github.com/apache/hadoop/pull/2907 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please

[GitHub] [hadoop] tasanuma commented on pull request #2907: HDFS-15975. Use LongAdder instead of AtomicLong

2021-04-17 Thread GitBox
tasanuma commented on pull request #2907: URL: https://github.com/apache/hadoop/pull/2907#issuecomment-821784149 Merged it. Thanks for your contribution, @tomscut. Thanks for your reviews, @virajjasani. -- This is an automated message from the Apache Git Service. To respond to the messag

[GitHub] [hadoop] tasanuma commented on pull request #2907: HDFS-15975. Use LongAdder instead of AtomicLong

2021-04-17 Thread GitBox
tasanuma commented on pull request #2907: URL: https://github.com/apache/hadoop/pull/2907#issuecomment-821784876 @tomscut I think this is a good optimization. Would you mind creating another PR for branch-3.3? -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [hadoop] ayushtkn commented on pull request #2860: HDFS-15988. Stabilise HDFS Pre-Commit.

2021-04-17 Thread GitBox
ayushtkn commented on pull request #2860: URL: https://github.com/apache/hadoop/pull/2860#issuecomment-821797015 TestDirectoryScanner ran this time and passed: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2860/15/testReport/org.apache.hadoop.hdfs.server.datanode/TestDirecto

[GitHub] [hadoop] hadoop-yetus commented on pull request #2860: HDFS-15988. Stabilise HDFS Pre-Commit.

2021-04-17 Thread GitBox
hadoop-yetus commented on pull request #2860: URL: https://github.com/apache/hadoop/pull/2860#issuecomment-821797317 (!) A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at https://ci-hadoop.apac

[GitHub] [hadoop] virajjasani opened a new pull request #2923: HDFS-15989. Split TestBalancer and De-flake testMaxIterationTime()

2021-04-17 Thread GitBox
virajjasani opened a new pull request #2923: URL: https://github.com/apache/hadoop/pull/2923 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this servic

[GitHub] [hadoop] Hexiaoqiao commented on a change in pull request #2918: HDFS-15987. Improve oiv tool to parse fsimage file in parallel with d…

2021-04-17 Thread GitBox
Hexiaoqiao commented on a change in pull request #2918: URL: https://github.com/apache/hadoop/pull/2918#discussion_r615238591 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/PBImageTextWriter.java ## @@ -649,14 +679,1

[GitHub] [hadoop] Hexiaoqiao commented on a change in pull request #2918: HDFS-15987. Improve oiv tool to parse fsimage file in parallel with d…

2021-04-17 Thread GitBox
Hexiaoqiao commented on a change in pull request #2918: URL: https://github.com/apache/hadoop/pull/2918#discussion_r615238591 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/PBImageTextWriter.java ## @@ -649,14 +679,1

[GitHub] [hadoop] Hexiaoqiao commented on pull request #2737: HDFS-15869. Network issue while FSEditLogAsync is executing RpcEdit.logSyncNotify can cause the namenode to hang

2021-04-17 Thread GitBox
Hexiaoqiao commented on pull request #2737: URL: https://github.com/apache/hadoop/pull/2737#issuecomment-821803540 @functioner would you mind to submit addendum blank commit to trigger CI again. Just notice that there are many failed unit tests but it seems not related to this PR(not very

[GitHub] [hadoop] aajisaka merged pull request #2922: HADOOP-17642. Remove appender EventCounter to avoid instantiation

2021-04-17 Thread GitBox
aajisaka merged pull request #2922: URL: https://github.com/apache/hadoop/pull/2922 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please

[GitHub] [hadoop] linyiqun commented on a change in pull request #2737: HDFS-15869. Network issue while FSEditLogAsync is executing RpcEdit.logSyncNotify can cause the namenode to hang

2021-04-17 Thread GitBox
linyiqun commented on a change in pull request #2737: URL: https://github.com/apache/hadoop/pull/2737#discussion_r615246158 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogAsync.java ## @@ -67,6 +77,12 @@

[GitHub] [hadoop] linyiqun commented on pull request #2737: HDFS-15869. Network issue while FSEditLogAsync is executing RpcEdit.logSyncNotify can cause the namenode to hang

2021-04-17 Thread GitBox
linyiqun commented on pull request #2737: URL: https://github.com/apache/hadoop/pull/2737#issuecomment-821811306 > @functioner would you mind to submit addendum blank commit to trigger CI again. Just notice that there are many failed unit tests but it seems not related to this PR(not very

[GitHub] [hadoop] ayushtkn commented on pull request #2861: HADOOP-17620. DistCp: Use Iterator for listing target directory as well.

2021-04-17 Thread GitBox
ayushtkn commented on pull request #2861: URL: https://github.com/apache/hadoop/pull/2861#issuecomment-821818236 Circling back: @steveloughran let me know, if that makes sense, if not I can add an extra option for target as well. or if you have any other suggestions or concerns, Let me

[GitHub] [hadoop] whbing commented on a change in pull request #2918: HDFS-15987. Improve oiv tool to parse fsimage file in parallel with d…

2021-04-17 Thread GitBox
whbing commented on a change in pull request #2918: URL: https://github.com/apache/hadoop/pull/2918#discussion_r615265357 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/PBImageTextWriter.java ## @@ -649,14 +679,123 @

[GitHub] [hadoop] tomscut commented on pull request #2907: HDFS-15975. Use LongAdder instead of AtomicLong

2021-04-17 Thread GitBox
tomscut commented on pull request #2907: URL: https://github.com/apache/hadoop/pull/2907#issuecomment-821837006 > @tomscut I think this is a good optimization. Would you mind creating another PR for branch-3.3? Thanks @tasanuma for the merge. I will create PR for branch-3.3 later.

[GitHub] [hadoop] whbing commented on a change in pull request #2918: HDFS-15987. Improve oiv tool to parse fsimage file in parallel with d…

2021-04-17 Thread GitBox
whbing commented on a change in pull request #2918: URL: https://github.com/apache/hadoop/pull/2918#discussion_r615265357 ## File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/PBImageTextWriter.java ## @@ -649,14 +679,123 @

[GitHub] [hadoop] hadoop-yetus commented on pull request #2860: HDFS-15988. Stabilise HDFS Pre-Commit.

2021-04-17 Thread GitBox
hadoop-yetus commented on pull request #2860: URL: https://github.com/apache/hadoop/pull/2860#issuecomment-821848462 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[GitHub] [hadoop] hadoop-yetus commented on pull request #2923: HDFS-15989. Split TestBalancer and De-flake testMaxIterationTime()

2021-04-17 Thread GitBox
hadoop-yetus commented on pull request #2923: URL: https://github.com/apache/hadoop/pull/2923#issuecomment-821861425 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

[GitHub] [hadoop] hadoop-yetus commented on pull request #2923: HDFS-15989. Split TestBalancer and De-flake testMaxIterationTime()

2021-04-17 Thread GitBox
hadoop-yetus commented on pull request #2923: URL: https://github.com/apache/hadoop/pull/2923#issuecomment-821874183 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok

<    1   2   3   4   5   6   7   8   9   10   >