[jira] [Work logged] (HDFS-16623) IllegalArgumentException in LifelineSender
[
https://issues.apache.org/jira/browse/HDFS-16623?focusedWorklogId=779798&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-779798
]
ASF GitHub Bot logged work on HDFS-16623:
-
Author: ASF GitHub Bot
Created on: 09/Jun/22 07:48
Start Date: 09/Jun/22 07:48
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on PR #4409:
URL: https://github.com/apache/hadoop/pull/4409#issuecomment-1150788913
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 38s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 37m 35s | | trunk passed |
| +1 :green_heart: | compile | 1m 43s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 1m 34s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 24s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 47s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 24s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 45s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 40s | | trunk passed |
| +1 :green_heart: | shadedclient | 22m 57s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 22s | | the patch passed |
| +1 :green_heart: | compile | 1m 26s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 1m 26s | | the patch passed |
| +1 :green_heart: | compile | 1m 20s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 1m 20s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 1s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 25s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 58s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 30s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 19s | | the patch passed |
| +1 :green_heart: | shadedclient | 22m 30s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 254m 46s | | hadoop-hdfs in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 23s | | The patch does not
generate ASF License warnings. |
| | | 363m 58s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4409/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4409 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 36b5aa7a5a87 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6
11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / eb2f7cfc0f4092e782aaacf13b0cd2fc5de5bccd |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4409/2/testReport/ |
| Max. process+thread count | 3100 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4409/2/console |
| versions |
[jira] [Created] (HDFS-16628) RBF: kerbose user remove Non-default namespace data failed
Xiping Zhang created HDFS-16628: --- Summary: RBF: kerbose user remove Non-default namespace data failed Key: HDFS-16628 URL: https://issues.apache.org/jira/browse/HDFS-16628 Project: Hadoop HDFS Issue Type: Bug Components: rbf Reporter: Xiping Zhang -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16563) Namenode WebUI prints sensitive information on Token Expiry
[ https://issues.apache.org/jira/browse/HDFS-16563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17552173#comment-17552173 ] Steve Loughran commented on HDFS-16563: --- causes YARN-11172 > Namenode WebUI prints sensitive information on Token Expiry > --- > > Key: HDFS-16563 > URL: https://issues.apache.org/jira/browse/HDFS-16563 > Project: Hadoop HDFS > Issue Type: Bug > Components: namanode, security, webhdfs >Affects Versions: 3.3.3 >Reporter: Renukaprasad C >Assignee: Renukaprasad C >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.4 > > Attachments: image-2022-04-27-23-01-16-033.png, > image-2022-04-27-23-28-40-568.png > > Time Spent: 3h > Remaining Estimate: 0h > > Login to Namenode WebUI. > Wait for token to expire. (Or modify the Token refresh time > dfs.namenode.delegation.token.renew/update-interval to lower value) > Refresh the WebUI after the Token expiry. > Full token information gets printed in WebUI. > > !image-2022-04-27-23-01-16-033.png! -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-16563) Namenode WebUI prints sensitive information on Token Expiry
[ https://issues.apache.org/jira/browse/HDFS-16563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HDFS-16563: -- Description: Login to Namenode WebUI. Wait for token to expire. (Or modify the Token refresh time dfs.namenode.delegation.token.renew/update-interval to lower value) Refresh the WebUI after the Token expiry. Full token information gets printed in WebUI. !image-2022-04-27-23-01-16-033.png! causes YARN-11172; all branches with this patch need that fix in too was: Login to Namenode WebUI. Wait for token to expire. (Or modify the Token refresh time dfs.namenode.delegation.token.renew/update-interval to lower value) Refresh the WebUI after the Token expiry. Full token information gets printed in WebUI. !image-2022-04-27-23-01-16-033.png! > Namenode WebUI prints sensitive information on Token Expiry > --- > > Key: HDFS-16563 > URL: https://issues.apache.org/jira/browse/HDFS-16563 > Project: Hadoop HDFS > Issue Type: Bug > Components: namanode, security, webhdfs >Affects Versions: 3.3.3 >Reporter: Renukaprasad C >Assignee: Renukaprasad C >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.4 > > Attachments: image-2022-04-27-23-01-16-033.png, > image-2022-04-27-23-28-40-568.png > > Time Spent: 3h > Remaining Estimate: 0h > > Login to Namenode WebUI. > Wait for token to expire. (Or modify the Token refresh time > dfs.namenode.delegation.token.renew/update-interval to lower value) > Refresh the WebUI after the Token expiry. > Full token information gets printed in WebUI. > > !image-2022-04-27-23-01-16-033.png! > causes YARN-11172; all branches with this patch need that fix in too -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16626) Under replicated blocks in dfsadmin report should contain pendingReconstruction‘s blocks
[
https://issues.apache.org/jira/browse/HDFS-16626?focusedWorklogId=779945&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-779945
]
ASF GitHub Bot logged work on HDFS-16626:
-
Author: ASF GitHub Bot
Created on: 09/Jun/22 13:20
Start Date: 09/Jun/22 13:20
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on PR #4416:
URL: https://github.com/apache/hadoop/pull/4416#issuecomment-115223
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 51s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| -1 :x: | mvninstall | 39m 5s |
[/branch-mvninstall-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4416/2/artifact/out/branch-mvninstall-root.txt)
| root in trunk failed. |
| +1 :green_heart: | compile | 1m 40s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 1m 32s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 20s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 39s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 18s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 40s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 42s | | trunk passed |
| +1 :green_heart: | shadedclient | 25m 37s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 24s | | the patch passed |
| +1 :green_heart: | compile | 1m 29s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 1m 29s | | the patch passed |
| +1 :green_heart: | compile | 1m 18s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 1m 18s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 0s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 28s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 58s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 31s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 31s | | the patch passed |
| +1 :green_heart: | shadedclient | 25m 30s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 374m 30s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4416/2/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 0m 58s | | The patch does not
generate ASF License warnings. |
| | | 489m 58s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.hdfs.server.mover.TestMover |
| | hadoop.hdfs.server.datanode.TestDataNodeLifeline |
| | hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks |
| | hadoop.hdfs.server.namenode.TestReconstructStripedBlocks |
| | hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics |
| | hadoop.hdfs.server.diskbalancer.command.TestDiskBalancerCommand |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4416/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4416 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux fd19872945ee 4.15.0-175-generic #184-Ubuntu SMP Thu Mar 24
17:48:36 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven
[jira] [Work logged] (HDFS-16622) addRDBI in IncrementalBlockReportManager may remove the block with bigger GS.
[
https://issues.apache.org/jira/browse/HDFS-16622?focusedWorklogId=779952&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-779952
]
ASF GitHub Bot logged work on HDFS-16622:
-
Author: ASF GitHub Bot
Created on: 09/Jun/22 13:35
Start Date: 09/Jun/22 13:35
Worklog Time Spent: 10m
Work Description: Hexiaoqiao commented on code in PR #4407:
URL: https://github.com/apache/hadoop/pull/4407#discussion_r893514408
##
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/IncrementalBlockReportManager.java:
##
@@ -251,12 +251,20 @@ synchronized void addRDBI(ReceivedDeletedBlockInfo rdbi,
DatanodeStorage storage) {
// Make sure another entry for the same block is first removed.
// There may only be one such entry.
+ReceivedDeletedBlockInfo removedInfo = null;
for (PerStorageIBR perStorage : pendingIBRs.values()) {
- if (perStorage.remove(rdbi.getBlock()) != null) {
+ removedInfo = perStorage.remove(rdbi.getBlock());
+ if (removedInfo != null) {
break;
}
}
-getPerStorageIBR(storage).put(rdbi);
+if (removedInfo != null &&
Review Comment:
@ZanderXu Thanks for the detailed information. It is an interesting case.
IMO, this improvement makes sense to me. Would you mind to add unit test to
cover this case?
Issue Time Tracking
---
Worklog Id: (was: 779952)
Time Spent: 1h (was: 50m)
> addRDBI in IncrementalBlockReportManager may remove the block with bigger GS.
> -
>
> Key: HDFS-16622
> URL: https://issues.apache.org/jira/browse/HDFS-16622
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: ZanderXu
>Assignee: ZanderXu
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> In our production environment, there is a strange missing block, according
> to the log, I suspect there is a bug in function
> addRDBI(ReceivedDeletedBlockInfo rdbi,DatanodeStorage storage)(line 250).
> Bug code in the for loop:
> {code:java}
> synchronized void addRDBI(ReceivedDeletedBlockInfo rdbi,
> DatanodeStorage storage) {
> // Make sure another entry for the same block is first removed.
> // There may only be one such entry.
> for (PerStorageIBR perStorage : pendingIBRs.values()) {
> if (perStorage.remove(rdbi.getBlock()) != null) {
> break;
> }
> }
> getPerStorageIBR(storage).put(rdbi);
> }
> {code}
> Removed the GS of the Block in ReceivedDeletedBlockInfo may be greater than
> the GS of the Block in rdbi. And NN will invalidate the Replicate will small
> GS when complete one block.
> So If there is only one replicate for one block, there is a possibility of
> missingblock because of this wrong logic.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16576) Remove unused Imports in Hadoop HDFS project
[ https://issues.apache.org/jira/browse/HDFS-16576?focusedWorklogId=779955&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-779955 ] ASF GitHub Bot logged work on HDFS-16576: - Author: ASF GitHub Bot Created on: 09/Jun/22 13:39 Start Date: 09/Jun/22 13:39 Worklog Time Spent: 10m Work Description: aajisaka merged PR #4389: URL: https://github.com/apache/hadoop/pull/4389 Issue Time Tracking --- Worklog Id: (was: 779955) Time Spent: 1h (was: 50m) > Remove unused Imports in Hadoop HDFS project > > > Key: HDFS-16576 > URL: https://issues.apache.org/jira/browse/HDFS-16576 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Ashutosh Gupta >Assignee: Ashutosh Gupta >Priority: Minor > Labels: pull-request-available > Time Spent: 1h > Remaining Estimate: 0h > > h3. Optimize Imports to keep code clean > # Remove any unused imports -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16576) Remove unused Imports in Hadoop HDFS project
[ https://issues.apache.org/jira/browse/HDFS-16576?focusedWorklogId=779956&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-779956 ] ASF GitHub Bot logged work on HDFS-16576: - Author: ASF GitHub Bot Created on: 09/Jun/22 13:39 Start Date: 09/Jun/22 13:39 Worklog Time Spent: 10m Work Description: aajisaka commented on PR #4389: URL: https://github.com/apache/hadoop/pull/4389#issuecomment-1151133768 Merged. Thank you @ashutoshcipher and @tomscut Issue Time Tracking --- Worklog Id: (was: 779956) Time Spent: 1h 10m (was: 1h) > Remove unused Imports in Hadoop HDFS project > > > Key: HDFS-16576 > URL: https://issues.apache.org/jira/browse/HDFS-16576 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Ashutosh Gupta >Assignee: Ashutosh Gupta >Priority: Minor > Labels: pull-request-available > Time Spent: 1h 10m > Remaining Estimate: 0h > > h3. Optimize Imports to keep code clean > # Remove any unused imports -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Resolved] (HDFS-16576) Remove unused Imports in Hadoop HDFS project
[ https://issues.apache.org/jira/browse/HDFS-16576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka resolved HDFS-16576. -- Fix Version/s: 3.4.0 Resolution: Fixed Committed to trunk. > Remove unused Imports in Hadoop HDFS project > > > Key: HDFS-16576 > URL: https://issues.apache.org/jira/browse/HDFS-16576 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Ashutosh Gupta >Assignee: Ashutosh Gupta >Priority: Minor > Labels: pull-request-available > Fix For: 3.4.0 > > Time Spent: 1h 10m > Remaining Estimate: 0h > > h3. Optimize Imports to keep code clean > # Remove any unused imports -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-16576) Remove unused imports in HDFS project
[ https://issues.apache.org/jira/browse/HDFS-16576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated HDFS-16576: - Summary: Remove unused imports in HDFS project (was: Remove unused Imports in Hadoop HDFS project) > Remove unused imports in HDFS project > - > > Key: HDFS-16576 > URL: https://issues.apache.org/jira/browse/HDFS-16576 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Ashutosh Gupta >Assignee: Ashutosh Gupta >Priority: Minor > Labels: pull-request-available > Fix For: 3.4.0 > > Time Spent: 1h 10m > Remaining Estimate: 0h > > h3. Optimize Imports to keep code clean > # Remove any unused imports -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16576) Remove unused imports in HDFS project
[ https://issues.apache.org/jira/browse/HDFS-16576?focusedWorklogId=779957&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-779957 ] ASF GitHub Bot logged work on HDFS-16576: - Author: ASF GitHub Bot Created on: 09/Jun/22 13:41 Start Date: 09/Jun/22 13:41 Worklog Time Spent: 10m Work Description: ashutoshcipher commented on PR #4389: URL: https://github.com/apache/hadoop/pull/4389#issuecomment-1151136011 Thanks @aajisaka and @tomscut. Issue Time Tracking --- Worklog Id: (was: 779957) Time Spent: 1h 20m (was: 1h 10m) > Remove unused imports in HDFS project > - > > Key: HDFS-16576 > URL: https://issues.apache.org/jira/browse/HDFS-16576 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Ashutosh Gupta >Assignee: Ashutosh Gupta >Priority: Minor > Labels: pull-request-available > Fix For: 3.4.0 > > Time Spent: 1h 20m > Remaining Estimate: 0h > > h3. Optimize Imports to keep code clean > # Remove any unused imports -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-16576) Remove unused imports in HDFS project
[ https://issues.apache.org/jira/browse/HDFS-16576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated HDFS-16576: - Fix Version/s: 3.3.4 Backported to branch-3.3. > Remove unused imports in HDFS project > - > > Key: HDFS-16576 > URL: https://issues.apache.org/jira/browse/HDFS-16576 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Ashutosh Gupta >Assignee: Ashutosh Gupta >Priority: Minor > Labels: pull-request-available > Fix For: 3.4.0, 3.3.4 > > Time Spent: 1h 20m > Remaining Estimate: 0h > > h3. Optimize Imports to keep code clean > # Remove any unused imports -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16626) Under replicated blocks in dfsadmin report should contain pendingReconstruction‘s blocks
[
https://issues.apache.org/jira/browse/HDFS-16626?focusedWorklogId=779964&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-779964
]
ASF GitHub Bot logged work on HDFS-16626:
-
Author: ASF GitHub Bot
Created on: 09/Jun/22 13:56
Start Date: 09/Jun/22 13:56
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on PR #4420:
URL: https://github.com/apache/hadoop/pull/4420#issuecomment-1151153833
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 56s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 40m 3s | | trunk passed |
| +1 :green_heart: | compile | 1m 42s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 1m 31s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 19s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 41s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 21s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 47s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 47s | | trunk passed |
| +1 :green_heart: | shadedclient | 26m 11s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 24s | | the patch passed |
| +1 :green_heart: | compile | 1m 27s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 1m 27s | | the patch passed |
| +1 :green_heart: | compile | 1m 19s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 1m 19s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 2s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 25s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 0s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 34s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 33s | | the patch passed |
| +1 :green_heart: | shadedclient | 25m 57s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 372m 0s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4420/1/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 0m 59s | | The patch does not
generate ASF License warnings. |
| | | 489m 39s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests |
hadoop.hdfs.server.namenode.TestReconstructStripedBlocks |
| | hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics |
| | hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4420/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4420 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 6d1b43af65c1 4.15.0-166-generic #174-Ubuntu SMP Wed Dec 8
19:07:44 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 8c07b5d97819c4532795f5643ec380953e597d0a |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
/usr/lib/j
[jira] [Work logged] (HDFS-16601) Failed to replace a bad datanode on the existing pipeline due to no more good datanodes being available to try
[
https://issues.apache.org/jira/browse/HDFS-16601?focusedWorklogId=779967&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-779967
]
ASF GitHub Bot logged work on HDFS-16601:
-
Author: ASF GitHub Bot
Created on: 09/Jun/22 14:01
Start Date: 09/Jun/22 14:01
Worklog Time Spent: 10m
Work Description: Hexiaoqiao commented on PR #4369:
URL: https://github.com/apache/hadoop/pull/4369#issuecomment-1151160510
Thanks for starting this proposal. I think there are still many issues for
data transfer for pipeline recovery from my practice, which includes both basic
function and performance. IIRC, there are only timeout exception and one no
explicit meaning exception, thus client has no helpful information (such src
node or target node meet issue, or other exceptions) to make decision.
Back to this PR, I totally agree to throw exception from datanode to client
first(but I am not sure if it is enough at this PR, maybe we need more
information) then add more fault-tolerant logic at client side.
IMO, we should file one new JIRA to design/refactor fault-tolerant for data
transfer of pipeline recovery. Just my own suggestion, not blocker.
Issue Time Tracking
---
Worklog Id: (was: 779967)
Time Spent: 1h (was: 50m)
> Failed to replace a bad datanode on the existing pipeline due to no more good
> datanodes being available to try
> --
>
> Key: HDFS-16601
> URL: https://issues.apache.org/jira/browse/HDFS-16601
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: ZanderXu
>Assignee: ZanderXu
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> In our production environment, we found a bug and stack like:
> {code:java}
> java.io.IOException: Failed to replace a bad datanode on the existing
> pipeline due to no more good datanodes being available to try. (Nodes:
> current=[DatanodeInfoWithStorage[127.0.0.1:59687,DS-b803febc-7b22-4144-9b39-7bf521cdaa8d,DISK],
>
> DatanodeInfoWithStorage[127.0.0.1:59670,DS-0d652bc2-1784-430d-961f-750f80a290f1,DISK]],
>
> original=[DatanodeInfoWithStorage[127.0.0.1:59670,DS-0d652bc2-1784-430d-961f-750f80a290f1,DISK],
>
> DatanodeInfoWithStorage[127.0.0.1:59687,DS-b803febc-7b22-4144-9b39-7bf521cdaa8d,DISK]]).
> The current failed datanode replacement policy is DEFAULT, and a client may
> configure this via
> 'dfs.client.block.write.replace-datanode-on-failure.policy' in its
> configuration.
> at
> org.apache.hadoop.hdfs.DataStreamer.findNewDatanode(DataStreamer.java:1418)
> at
> org.apache.hadoop.hdfs.DataStreamer.addDatanode2ExistingPipeline(DataStreamer.java:1478)
> at
> org.apache.hadoop.hdfs.DataStreamer.handleDatanodeReplacement(DataStreamer.java:1704)
> at
> org.apache.hadoop.hdfs.DataStreamer.setupPipelineInternal(DataStreamer.java:1605)
> at
> org.apache.hadoop.hdfs.DataStreamer.setupPipelineForAppendOrRecovery(DataStreamer.java:1587)
> at
> org.apache.hadoop.hdfs.DataStreamer.processDatanodeOrExternalError(DataStreamer.java:1371)
> at org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:674)
> {code}
> And the root cause is that DFSClient cannot perceive the exception of
> TransferBlock during PipelineRecovery. If failed during TransferBlock, the
> DFSClient will retry all datanodes in the cluster and then failed.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16600) Deadlock on DataNode
[
https://issues.apache.org/jira/browse/HDFS-16600?focusedWorklogId=779971&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-779971
]
ASF GitHub Bot logged work on HDFS-16600:
-
Author: ASF GitHub Bot
Created on: 09/Jun/22 14:07
Start Date: 09/Jun/22 14:07
Worklog Time Spent: 10m
Work Description: Hexiaoqiao commented on PR #4367:
URL: https://github.com/apache/hadoop/pull/4367#issuecomment-1151166917
Thanks all for the further discussion. About the unit test, I did not
retrieve this one,
`org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.testSynchronousEviction`.
Anything I missed? @ZanderXu Would mind to check if this test really located
at hadoop-hdfs module now? Please correct me if I am wrong. Thanks again.
Issue Time Tracking
---
Worklog Id: (was: 779971)
Time Spent: 3h (was: 2h 50m)
> Deadlock on DataNode
>
>
> Key: HDFS-16600
> URL: https://issues.apache.org/jira/browse/HDFS-16600
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: ZanderXu
>Assignee: ZanderXu
>Priority: Major
> Labels: pull-request-available
> Time Spent: 3h
> Remaining Estimate: 0h
>
> The UT
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.testSynchronousEviction
> failed, because happened deadlock, which is introduced by
> [HDFS-16534|https://issues.apache.org/jira/browse/HDFS-16534].
> DeadLock:
> {code:java}
> // org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.createRbw line 1588
> need a read lock
> try (AutoCloseableLock lock = lockManager.readLock(LockLevel.BLOCK_POOl,
> b.getBlockPoolId()))
> // org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.evictBlocks line
> 3526 need a write lock
> try (AutoCloseableLock lock = lockManager.writeLock(LockLevel.BLOCK_POOl,
> bpid))
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16627) improve BPServiceActor#register Log Add NN Addr
[
https://issues.apache.org/jira/browse/HDFS-16627?focusedWorklogId=779979&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-779979
]
ASF GitHub Bot logged work on HDFS-16627:
-
Author: ASF GitHub Bot
Created on: 09/Jun/22 14:24
Start Date: 09/Jun/22 14:24
Worklog Time Spent: 10m
Work Description: Hexiaoqiao commented on code in PR #4419:
URL: https://github.com/apache/hadoop/pull/4419#discussion_r893570312
##
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BPServiceActor.java:
##
@@ -816,7 +816,7 @@ void register(NamespaceInfo nsInfo) throws IOException {
// off disk - so update the bpRegistration object from that info
DatanodeRegistration newBpRegistration = bpos.createRegistration();
-LOG.info(this + " beginning handshake with NN");
+LOG.info("{} beginning handshake with NN:{}", this, nnAddr);
Review Comment:
Please keep the same format of log, such as delete the blank at end of a
sentence, leave one blank between two words. such as,
L819: `LOG.info("{} beginning handshake with NN: {}", this, nnAddr);`
L831: `LOG.info("Problem connecting to server: {}", nnAddr);`
Issue Time Tracking
---
Worklog Id: (was: 779979)
Time Spent: 0.5h (was: 20m)
> improve BPServiceActor#register Log Add NN Addr
> ---
>
> Key: HDFS-16627
> URL: https://issues.apache.org/jira/browse/HDFS-16627
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: hdfs
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Minor
> Labels: pull-request-available
> Fix For: 3.4.0
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> When I read the log, I think the Addr information of NN should be added to
> make the log information more complete.
> The log is as follows:
> {code:java}
> 2022-06-06 06:15:32,715 [BP-1990954485-172.17.0.2-1654496132136 heartbeating
> to localhost/127.0.0.1:42811] INFO datanode.DataNode
> (BPServiceActor.java:register(819)) - Block pool
> BP-1990954485-172.17.0.2-1654496132136 (Datanode Uuid
> 7d4b5459-6f2b-4203-bf6f-d31bfb9b6c3f) service to localhost/127.0.0.1:42811
> beginning handshake with NN.
> 2022-06-06 06:15:32,717 [BP-1990954485-172.17.0.2-1654496132136 heartbeating
> to localhost/127.0.0.1:42811] INFO datanode.DataNode
> (BPServiceActor.java:register(847)) - Block pool
> BP-1990954485-172.17.0.2-1654496132136 (Datanode Uuid
> 7d4b5459-6f2b-4203-bf6f-d31bfb9b6c3f) service to localhost/127.0.0.1:42811
> successfully registered with NN. {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16600) Deadlock on DataNode
[
https://issues.apache.org/jira/browse/HDFS-16600?focusedWorklogId=779985&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-779985
]
ASF GitHub Bot logged work on HDFS-16600:
-
Author: ASF GitHub Bot
Created on: 09/Jun/22 14:35
Start Date: 09/Jun/22 14:35
Worklog Time Spent: 10m
Work Description: ZanderXu commented on PR #4367:
URL: https://github.com/apache/hadoop/pull/4367#issuecomment-1151202018
Oh, I'm sorry, the failed UT is
`org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaPlacement.testSynchronousEviction`.
Issue Time Tracking
---
Worklog Id: (was: 779985)
Time Spent: 3h 10m (was: 3h)
> Deadlock on DataNode
>
>
> Key: HDFS-16600
> URL: https://issues.apache.org/jira/browse/HDFS-16600
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: ZanderXu
>Assignee: ZanderXu
>Priority: Major
> Labels: pull-request-available
> Time Spent: 3h 10m
> Remaining Estimate: 0h
>
> The UT
> org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.testSynchronousEviction
> failed, because happened deadlock, which is introduced by
> [HDFS-16534|https://issues.apache.org/jira/browse/HDFS-16534].
> DeadLock:
> {code:java}
> // org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.createRbw line 1588
> need a read lock
> try (AutoCloseableLock lock = lockManager.readLock(LockLevel.BLOCK_POOl,
> b.getBlockPoolId()))
> // org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.evictBlocks line
> 3526 need a write lock
> try (AutoCloseableLock lock = lockManager.writeLock(LockLevel.BLOCK_POOl,
> bpid))
> {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16627) improve BPServiceActor#register Log Add NN Addr
[
https://issues.apache.org/jira/browse/HDFS-16627?focusedWorklogId=779991&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-779991
]
ASF GitHub Bot logged work on HDFS-16627:
-
Author: ASF GitHub Bot
Created on: 09/Jun/22 14:44
Start Date: 09/Jun/22 14:44
Worklog Time Spent: 10m
Work Description: slfan1989 commented on code in PR #4419:
URL: https://github.com/apache/hadoop/pull/4419#discussion_r893598028
##
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BPServiceActor.java:
##
@@ -816,7 +816,7 @@ void register(NamespaceInfo nsInfo) throws IOException {
// off disk - so update the bpRegistration object from that info
DatanodeRegistration newBpRegistration = bpos.createRegistration();
-LOG.info(this + " beginning handshake with NN");
+LOG.info("{} beginning handshake with NN:{}", this, nnAddr);
Review Comment:
@Hexiaoqiao Thanks for your help reviewing the code, I will fix it.
Issue Time Tracking
---
Worklog Id: (was: 779991)
Time Spent: 40m (was: 0.5h)
> improve BPServiceActor#register Log Add NN Addr
> ---
>
> Key: HDFS-16627
> URL: https://issues.apache.org/jira/browse/HDFS-16627
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: hdfs
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Minor
> Labels: pull-request-available
> Fix For: 3.4.0
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> When I read the log, I think the Addr information of NN should be added to
> make the log information more complete.
> The log is as follows:
> {code:java}
> 2022-06-06 06:15:32,715 [BP-1990954485-172.17.0.2-1654496132136 heartbeating
> to localhost/127.0.0.1:42811] INFO datanode.DataNode
> (BPServiceActor.java:register(819)) - Block pool
> BP-1990954485-172.17.0.2-1654496132136 (Datanode Uuid
> 7d4b5459-6f2b-4203-bf6f-d31bfb9b6c3f) service to localhost/127.0.0.1:42811
> beginning handshake with NN.
> 2022-06-06 06:15:32,717 [BP-1990954485-172.17.0.2-1654496132136 heartbeating
> to localhost/127.0.0.1:42811] INFO datanode.DataNode
> (BPServiceActor.java:register(847)) - Block pool
> BP-1990954485-172.17.0.2-1654496132136 (Datanode Uuid
> 7d4b5459-6f2b-4203-bf6f-d31bfb9b6c3f) service to localhost/127.0.0.1:42811
> successfully registered with NN. {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16627) improve BPServiceActor#register Log Add NN Addr
[
https://issues.apache.org/jira/browse/HDFS-16627?focusedWorklogId=779992&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-779992
]
ASF GitHub Bot logged work on HDFS-16627:
-
Author: ASF GitHub Bot
Created on: 09/Jun/22 14:45
Start Date: 09/Jun/22 14:45
Worklog Time Spent: 10m
Work Description: slfan1989 commented on PR #4419:
URL: https://github.com/apache/hadoop/pull/4419#issuecomment-1151214937
> LGTM. Thanks @slfan1989
Thank you for help to review the code!
Issue Time Tracking
---
Worklog Id: (was: 779992)
Time Spent: 50m (was: 40m)
> improve BPServiceActor#register Log Add NN Addr
> ---
>
> Key: HDFS-16627
> URL: https://issues.apache.org/jira/browse/HDFS-16627
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: hdfs
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Minor
> Labels: pull-request-available
> Fix For: 3.4.0
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> When I read the log, I think the Addr information of NN should be added to
> make the log information more complete.
> The log is as follows:
> {code:java}
> 2022-06-06 06:15:32,715 [BP-1990954485-172.17.0.2-1654496132136 heartbeating
> to localhost/127.0.0.1:42811] INFO datanode.DataNode
> (BPServiceActor.java:register(819)) - Block pool
> BP-1990954485-172.17.0.2-1654496132136 (Datanode Uuid
> 7d4b5459-6f2b-4203-bf6f-d31bfb9b6c3f) service to localhost/127.0.0.1:42811
> beginning handshake with NN.
> 2022-06-06 06:15:32,717 [BP-1990954485-172.17.0.2-1654496132136 heartbeating
> to localhost/127.0.0.1:42811] INFO datanode.DataNode
> (BPServiceActor.java:register(847)) - Block pool
> BP-1990954485-172.17.0.2-1654496132136 (Datanode Uuid
> 7d4b5459-6f2b-4203-bf6f-d31bfb9b6c3f) service to localhost/127.0.0.1:42811
> successfully registered with NN. {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16601) Failed to replace a bad datanode on the existing pipeline due to no more good datanodes being available to try
[
https://issues.apache.org/jira/browse/HDFS-16601?focusedWorklogId=779997&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-779997
]
ASF GitHub Bot logged work on HDFS-16601:
-
Author: ASF GitHub Bot
Created on: 09/Jun/22 14:56
Start Date: 09/Jun/22 14:56
Worklog Time Spent: 10m
Work Description: ZanderXu commented on PR #4369:
URL: https://github.com/apache/hadoop/pull/4369#issuecomment-1151228050
Thanks @Hexiaoqiao for your suggestion. Yeah, your are right, we need more
failed information for client, like transfer source failed or transfer target
failed. If client have more information about failed transfer, It can
accurately and efficiently remove abnormal nodes. But this would be a big
feature.
Fortunately, at present, as long as failed exception throw to client, the
client defaults to thinking that the new dn is abnormal, and will exclude it
and retry transfer. During retrying transfer, Client will chose new source dn
and new target dn. Therefor, the source and target dn in the previous failed
transfer round will be replaced.
If it is target dn caused failed, excluded the target dn will be ok.
If it is source dn caused failed, it will be removed when building the new
pipeline.
So I think simple process is just throw failed exception to client, and
client can find and remove the real abnormal datanode.
Issue Time Tracking
---
Worklog Id: (was: 779997)
Time Spent: 1h 10m (was: 1h)
> Failed to replace a bad datanode on the existing pipeline due to no more good
> datanodes being available to try
> --
>
> Key: HDFS-16601
> URL: https://issues.apache.org/jira/browse/HDFS-16601
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: ZanderXu
>Assignee: ZanderXu
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> In our production environment, we found a bug and stack like:
> {code:java}
> java.io.IOException: Failed to replace a bad datanode on the existing
> pipeline due to no more good datanodes being available to try. (Nodes:
> current=[DatanodeInfoWithStorage[127.0.0.1:59687,DS-b803febc-7b22-4144-9b39-7bf521cdaa8d,DISK],
>
> DatanodeInfoWithStorage[127.0.0.1:59670,DS-0d652bc2-1784-430d-961f-750f80a290f1,DISK]],
>
> original=[DatanodeInfoWithStorage[127.0.0.1:59670,DS-0d652bc2-1784-430d-961f-750f80a290f1,DISK],
>
> DatanodeInfoWithStorage[127.0.0.1:59687,DS-b803febc-7b22-4144-9b39-7bf521cdaa8d,DISK]]).
> The current failed datanode replacement policy is DEFAULT, and a client may
> configure this via
> 'dfs.client.block.write.replace-datanode-on-failure.policy' in its
> configuration.
> at
> org.apache.hadoop.hdfs.DataStreamer.findNewDatanode(DataStreamer.java:1418)
> at
> org.apache.hadoop.hdfs.DataStreamer.addDatanode2ExistingPipeline(DataStreamer.java:1478)
> at
> org.apache.hadoop.hdfs.DataStreamer.handleDatanodeReplacement(DataStreamer.java:1704)
> at
> org.apache.hadoop.hdfs.DataStreamer.setupPipelineInternal(DataStreamer.java:1605)
> at
> org.apache.hadoop.hdfs.DataStreamer.setupPipelineForAppendOrRecovery(DataStreamer.java:1587)
> at
> org.apache.hadoop.hdfs.DataStreamer.processDatanodeOrExternalError(DataStreamer.java:1371)
> at org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:674)
> {code}
> And the root cause is that DFSClient cannot perceive the exception of
> TransferBlock during PipelineRecovery. If failed during TransferBlock, the
> DFSClient will retry all datanodes in the cluster and then failed.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16598) All datanodes [DatanodeInfoWithStorage[127.0.0.1:57448,DS-1b5f7e33-a2bf-4edc-9122-a74c995a99f5,DISK]] are bad. Aborting...
[
https://issues.apache.org/jira/browse/HDFS-16598?focusedWorklogId=77&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-77
]
ASF GitHub Bot logged work on HDFS-16598:
-
Author: ASF GitHub Bot
Created on: 09/Jun/22 15:01
Start Date: 09/Jun/22 15:01
Worklog Time Spent: 10m
Work Description: ZanderXu commented on PR #4366:
URL: https://github.com/apache/hadoop/pull/4366#issuecomment-1151236331
> Just suggest to improve them together in one PR.
@Hexiaoqiao do you mean to change all getReplicaInfo(ExtendedBlock b) to
getReplicaInfo(String bpid, long blkid) in fine-grained lock in this PR?
Issue Time Tracking
---
Worklog Id: (was: 77)
Time Spent: 2h 10m (was: 2h)
> All datanodes
> [DatanodeInfoWithStorage[127.0.0.1:57448,DS-1b5f7e33-a2bf-4edc-9122-a74c995a99f5,DISK]]
> are bad. Aborting...
> --
>
> Key: HDFS-16598
> URL: https://issues.apache.org/jira/browse/HDFS-16598
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: ZanderXu
>Assignee: ZanderXu
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> org.apache.hadoop.hdfs.testPipelineRecoveryOnRestartFailure failed with the
> stack like:
> {code:java}
> java.io.IOException: All datanodes
> [DatanodeInfoWithStorage[127.0.0.1:57448,DS-1b5f7e33-a2bf-4edc-9122-a74c995a99f5,DISK]]
> are bad. Aborting...
> at
> org.apache.hadoop.hdfs.DataStreamer.handleBadDatanode(DataStreamer.java:1667)
> at
> org.apache.hadoop.hdfs.DataStreamer.setupPipelineInternal(DataStreamer.java:1601)
> at
> org.apache.hadoop.hdfs.DataStreamer.setupPipelineForAppendOrRecovery(DataStreamer.java:1587)
> at
> org.apache.hadoop.hdfs.DataStreamer.processDatanodeOrExternalError(DataStreamer.java:1371)
> at org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:674)
> {code}
> After tracing the root cause, this bug was introduced by
> [HDFS-16534|https://issues.apache.org/jira/browse/HDFS-16534]. Because the
> block GS of client may be smaller than DN when pipeline recovery failed.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16605) Improve Code With Lambda in hadoop-hdfs-rbf moudle
[ https://issues.apache.org/jira/browse/HDFS-16605?focusedWorklogId=780021&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-780021 ] ASF GitHub Bot logged work on HDFS-16605: - Author: ASF GitHub Bot Created on: 09/Jun/22 15:52 Start Date: 09/Jun/22 15:52 Worklog Time Spent: 10m Work Description: slfan1989 commented on PR #4375: URL: https://github.com/apache/hadoop/pull/4375#issuecomment-1151312523 @goiri Please help me review the code again. Issue Time Tracking --- Worklog Id: (was: 780021) Time Spent: 1h 40m (was: 1.5h) > Improve Code With Lambda in hadoop-hdfs-rbf moudle > -- > > Key: HDFS-16605 > URL: https://issues.apache.org/jira/browse/HDFS-16605 > Project: Hadoop HDFS > Issue Type: Improvement > Components: rbf >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Minor > Labels: pull-request-available > Time Spent: 1h 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16627) improve BPServiceActor#register Log Add NN Addr
[
https://issues.apache.org/jira/browse/HDFS-16627?focusedWorklogId=780118&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-780118
]
ASF GitHub Bot logged work on HDFS-16627:
-
Author: ASF GitHub Bot
Created on: 09/Jun/22 21:45
Start Date: 09/Jun/22 21:45
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on PR #4419:
URL: https://github.com/apache/hadoop/pull/4419#issuecomment-1151655696
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 37s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 37m 10s | | trunk passed |
| +1 :green_heart: | compile | 1m 41s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 1m 39s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 23s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 43s | | trunk passed |
| -1 :x: | javadoc | 1m 24s |
[/branch-javadoc-hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4419/3/artifact/out/branch-javadoc-hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt)
| hadoop-hdfs in trunk failed with JDK Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1. |
| +1 :green_heart: | javadoc | 1m 49s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 41s | | trunk passed |
| +1 :green_heart: | shadedclient | 22m 58s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 25s | | the patch passed |
| +1 :green_heart: | compile | 1m 26s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 1m 26s | | the patch passed |
| +1 :green_heart: | compile | 1m 21s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 1m 21s | | the patch passed |
| +1 :green_heart: | blanks | 0m 1s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 0s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 27s | | the patch passed |
| -1 :x: | javadoc | 0m 57s |
[/patch-javadoc-hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4419/3/artifact/out/patch-javadoc-hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt)
| hadoop-hdfs in the patch failed with JDK Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1. |
| +1 :green_heart: | javadoc | 1m 29s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 23s | | the patch passed |
| +1 :green_heart: | shadedclient | 22m 25s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 251m 3s | | hadoop-hdfs in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 14s | | The patch does not
generate ASF License warnings. |
| | | 359m 36s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4419/3/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4419 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux d837a0f53be9 4.15.0-175-generic #184-Ubuntu SMP Thu Mar 24
17:48:36 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 1e3aa224e5aa4fb3f19dcb580eafd9d9b4f7af11 |
| Default Java | Private Build-1.8.0_312-8u312-b07-
[jira] [Work logged] (HDFS-16627) improve BPServiceActor#register Log Add NN Addr
[
https://issues.apache.org/jira/browse/HDFS-16627?focusedWorklogId=780117&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-780117
]
ASF GitHub Bot logged work on HDFS-16627:
-
Author: ASF GitHub Bot
Created on: 09/Jun/22 21:45
Start Date: 09/Jun/22 21:45
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on PR #4419:
URL: https://github.com/apache/hadoop/pull/4419#issuecomment-1151655335
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 46s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 37m 48s | | trunk passed |
| +1 :green_heart: | compile | 1m 44s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 1m 33s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 26s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 44s | | trunk passed |
| -1 :x: | javadoc | 1m 27s |
[/branch-javadoc-hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4419/2/artifact/out/branch-javadoc-hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt)
| hadoop-hdfs in trunk failed with JDK Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1. |
| +1 :green_heart: | javadoc | 1m 42s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 39s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 7s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 23s | | the patch passed |
| +1 :green_heart: | compile | 1m 29s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 1m 29s | | the patch passed |
| +1 :green_heart: | compile | 1m 18s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 1m 18s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 1s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 26s | | the patch passed |
| -1 :x: | javadoc | 1m 0s |
[/patch-javadoc-hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4419/2/artifact/out/patch-javadoc-hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt)
| hadoop-hdfs in the patch failed with JDK Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1. |
| +1 :green_heart: | javadoc | 1m 32s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 26s | | the patch passed |
| +1 :green_heart: | shadedclient | 22m 57s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 251m 44s | | hadoop-hdfs in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 13s | | The patch does not
generate ASF License warnings. |
| | | 361m 43s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4419/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4419 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 61d5321e2f89 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 1e3aa224e5aa4fb3f19dcb580eafd9d9b4f7af11 |
| Default Java | Private Build-1.8.0_312-8u312-b07-0
[jira] [Work logged] (HDFS-16619) impove HttpHeaders.Values And HttpHeaders.Names With recommended Class
[
https://issues.apache.org/jira/browse/HDFS-16619?focusedWorklogId=780119&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-780119
]
ASF GitHub Bot logged work on HDFS-16619:
-
Author: ASF GitHub Bot
Created on: 09/Jun/22 21:48
Start Date: 09/Jun/22 21:48
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on PR #4406:
URL: https://github.com/apache/hadoop/pull/4406#issuecomment-1151657257
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 50s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 37m 11s | | trunk passed |
| +1 :green_heart: | compile | 1m 44s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 1m 37s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 24s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 47s | | trunk passed |
| -1 :x: | javadoc | 1m 25s |
[/branch-javadoc-hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4406/3/artifact/out/branch-javadoc-hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt)
| hadoop-hdfs in trunk failed with JDK Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1. |
| +1 :green_heart: | javadoc | 1m 49s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 45s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 43s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 22s | | the patch passed |
| +1 :green_heart: | compile | 1m 28s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 1m 28s | |
hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 generated 0 new +
911 unchanged - 26 fixed = 911 total (was 937) |
| +1 :green_heart: | compile | 1m 18s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 1m 18s | |
hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 0 new
+ 890 unchanged - 26 fixed = 890 total (was 916) |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 1s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 24s | | the patch passed |
| -1 :x: | javadoc | 0m 55s |
[/patch-javadoc-hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4406/3/artifact/out/patch-javadoc-hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt)
| hadoop-hdfs in the patch failed with JDK Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1. |
| +1 :green_heart: | javadoc | 1m 32s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 18s | | the patch passed |
| +1 :green_heart: | shadedclient | 22m 28s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 247m 16s | | hadoop-hdfs in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 14s | | The patch does not
generate ASF License warnings. |
| | | 356m 51s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4406/3/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4406 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsi
[jira] [Work logged] (HDFS-16609) Fix Flakes Junit Tests that often report timeouts
[
https://issues.apache.org/jira/browse/HDFS-16609?focusedWorklogId=780133&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-780133
]
ASF GitHub Bot logged work on HDFS-16609:
-
Author: ASF GitHub Bot
Created on: 09/Jun/22 23:37
Start Date: 09/Jun/22 23:37
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on PR #4382:
URL: https://github.com/apache/hadoop/pull/4382#issuecomment-1151713943
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 56s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 42m 12s | | trunk passed |
| +1 :green_heart: | compile | 1m 43s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 1m 33s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 21s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 47s | | trunk passed |
| -1 :x: | javadoc | 1m 24s |
[/branch-javadoc-hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4382/4/artifact/out/branch-javadoc-hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt)
| hadoop-hdfs in trunk failed with JDK Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1. |
| +1 :green_heart: | javadoc | 1m 39s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 4m 7s | | trunk passed |
| +1 :green_heart: | shadedclient | 26m 51s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 23s | | the patch passed |
| +1 :green_heart: | compile | 1m 31s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 1m 31s | | the patch passed |
| +1 :green_heart: | compile | 1m 24s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 1m 24s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 8s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 35s | | the patch passed |
| -1 :x: | javadoc | 1m 3s |
[/patch-javadoc-hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4382/4/artifact/out/patch-javadoc-hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt)
| hadoop-hdfs in the patch failed with JDK Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1. |
| +1 :green_heart: | javadoc | 1m 35s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 45s | | the patch passed |
| +1 :green_heart: | shadedclient | 26m 8s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 346m 9s | | hadoop-hdfs in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 58s | | The patch does not
generate ASF License warnings. |
| | | 467m 30s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4382/4/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4382 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux d4cd432f2daf 4.15.0-175-generic #184-Ubuntu SMP Thu Mar 24
17:48:36 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 20fe8be4eada6d583a1a3c29272fdf535d7b5365 |
| Default Java | Private Build-1.8.0_312-8u312-b07-
[jira] [Work logged] (HDFS-16609) Fix Flakes Junit Tests that often report timeouts
[
https://issues.apache.org/jira/browse/HDFS-16609?focusedWorklogId=780136&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-780136
]
ASF GitHub Bot logged work on HDFS-16609:
-
Author: ASF GitHub Bot
Created on: 09/Jun/22 23:46
Start Date: 09/Jun/22 23:46
Worklog Time Spent: 10m
Work Description: slfan1989 commented on PR #4382:
URL: https://github.com/apache/hadoop/pull/4382#issuecomment-1151717102
@tomscut @Hexiaoqiao Thanks for helping to review the code, the junit test
has passed, the java doc problem can be ignored, I will submit pr separately to
fix the java doc problem.
Issue Time Tracking
---
Worklog Id: (was: 780136)
Time Spent: 1h 40m (was: 1.5h)
> Fix Flakes Junit Tests that often report timeouts
> -
>
> Key: HDFS-16609
> URL: https://issues.apache.org/jira/browse/HDFS-16609
> Project: Hadoop HDFS
> Issue Type: Improvement
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0
>
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> When I was dealing with HDFS-16590 JIRA, Junit Tests often reported errors, I
> found that one type of problem is TimeOut problem, these problems can be
> avoided by adjusting TimeOut time.
> The modified method is as follows:
> 1.org.apache.hadoop.hdfs.TestFileCreation#testServerDefaultsWithMinimalCaching
> {code:java}
> [ERROR]
> testServerDefaultsWithMinimalCaching(org.apache.hadoop.hdfs.TestFileCreation)
> Time elapsed: 7.136 s <<< ERROR!
> java.util.concurrent.TimeoutException:
> Timed out waiting for condition.
> Thread diagnostics:
> [WARNING]
> org.apache.hadoop.hdfs.TestFileCreation.testServerDefaultsWithMinimalCaching(org.apache.hadoop.hdfs.TestFileCreation)
> [ERROR] Run 1: TestFileCreation.testServerDefaultsWithMinimalCaching:277
> Timeout Timed out ...
> [INFO] Run 2: PASS{code}
> 2.org.apache.hadoop.hdfs.TestDFSShell#testFilePermissions
> {code:java}
> [ERROR] testFilePermissions(org.apache.hadoop.hdfs.TestDFSShell) Time
> elapsed: 30.022 s <<< ERROR!
> org.junit.runners.model.TestTimedOutException: test timed out after 3
> milliseconds
> at java.lang.Thread.dumpThreads(Native Method)
> at java.lang.Thread.getStackTrace(Thread.java:1549)
> at
> org.junit.internal.runners.statements.FailOnTimeout.createTimeoutException(FailOnTimeout.java:182)
> at
> org.junit.internal.runners.statements.FailOnTimeout.getResult(FailOnTimeout.java:177)
> at
> org.junit.internal.runners.statements.FailOnTimeout.evaluate(FailOnTimeout.java:128)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.lang.Thread.run(Thread.java:748)
> [WARNING]
> org.apache.hadoop.hdfs.TestDFSShell.testFilePermissions(org.apache.hadoop.hdfs.TestDFSShell)
> [ERROR] Run 1: TestDFSShell.testFilePermissions TestTimedOut test timed out
> after 3 mil...
> [INFO] Run 2: PASS {code}
> 3.org.apache.hadoop.hdfs.server.sps.TestExternalStoragePolicySatisfier#testSPSWhenFileHasExcessRedundancyBlocks
> {code:java}
> [ERROR]
> testSPSWhenFileHasExcessRedundancyBlocks(org.apache.hadoop.hdfs.server.sps.TestExternalStoragePolicySatisfier)
> Time elapsed: 67.904 s <<< ERROR!
> java.util.concurrent.TimeoutException:
> Timed out waiting for condition.
> [WARNING]
> org.apache.hadoop.hdfs.server.sps.TestExternalStoragePolicySatisfier.testSPSWhenFileHasExcessRedundancyBlocks(org.apache.hadoop.hdfs.server.sps.TestExternalStoragePolicySatisfier)
> [ERROR] Run 1:
> TestExternalStoragePolicySatisfier.testSPSWhenFileHasExcessRedundancyBlocks:1379
> Timeout
> [ERROR] Run 2:
> TestExternalStoragePolicySatisfier.testSPSWhenFileHasExcessRedundancyBlocks:1379
> Timeout
> [INFO] Run 3: PASS {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16619) impove HttpHeaders.Values And HttpHeaders.Names With recommended Class
[ https://issues.apache.org/jira/browse/HDFS-16619?focusedWorklogId=780138&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-780138 ] ASF GitHub Bot logged work on HDFS-16619: - Author: ASF GitHub Bot Created on: 09/Jun/22 23:49 Start Date: 09/Jun/22 23:49 Worklog Time Spent: 10m Work Description: slfan1989 commented on PR #4406: URL: https://github.com/apache/hadoop/pull/4406#issuecomment-1151719908 @tomscut Please help me review the code, this pr is to replace the deprecated import. There should be no code risk and will not cause junit to fail. Issue Time Tracking --- Worklog Id: (was: 780138) Time Spent: 1h (was: 50m) > impove HttpHeaders.Values And HttpHeaders.Names With recommended Class > -- > > Key: HDFS-16619 > URL: https://issues.apache.org/jira/browse/HDFS-16619 > Project: Hadoop HDFS > Issue Type: Improvement >Affects Versions: 3.4.0 >Reporter: fanshilun >Assignee: fanshilun >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > Time Spent: 1h > Remaining Estimate: 0h > > HttpHeaders.Values and HttpHeaders.Names are deprecated, use > HttpHeaderValues and HttpHeaderNames instead. -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16627) improve BPServiceActor#register Log Add NN Addr
[
https://issues.apache.org/jira/browse/HDFS-16627?focusedWorklogId=780140&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-780140
]
ASF GitHub Bot logged work on HDFS-16627:
-
Author: ASF GitHub Bot
Created on: 09/Jun/22 23:50
Start Date: 09/Jun/22 23:50
Worklog Time Spent: 10m
Work Description: slfan1989 commented on PR #4419:
URL: https://github.com/apache/hadoop/pull/4419#issuecomment-1151721248
@Hexiaoqiao Please help me review the code again!
Issue Time Tracking
---
Worklog Id: (was: 780140)
Time Spent: 1h 20m (was: 1h 10m)
> improve BPServiceActor#register Log Add NN Addr
> ---
>
> Key: HDFS-16627
> URL: https://issues.apache.org/jira/browse/HDFS-16627
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: hdfs
>Affects Versions: 3.4.0
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Minor
> Labels: pull-request-available
> Fix For: 3.4.0
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> When I read the log, I think the Addr information of NN should be added to
> make the log information more complete.
> The log is as follows:
> {code:java}
> 2022-06-06 06:15:32,715 [BP-1990954485-172.17.0.2-1654496132136 heartbeating
> to localhost/127.0.0.1:42811] INFO datanode.DataNode
> (BPServiceActor.java:register(819)) - Block pool
> BP-1990954485-172.17.0.2-1654496132136 (Datanode Uuid
> 7d4b5459-6f2b-4203-bf6f-d31bfb9b6c3f) service to localhost/127.0.0.1:42811
> beginning handshake with NN.
> 2022-06-06 06:15:32,717 [BP-1990954485-172.17.0.2-1654496132136 heartbeating
> to localhost/127.0.0.1:42811] INFO datanode.DataNode
> (BPServiceActor.java:register(847)) - Block pool
> BP-1990954485-172.17.0.2-1654496132136 (Datanode Uuid
> 7d4b5459-6f2b-4203-bf6f-d31bfb9b6c3f) service to localhost/127.0.0.1:42811
> successfully registered with NN. {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15506) [JDK 11] Fix javadoc errors in hadoop-hdfs module
[
https://issues.apache.org/jira/browse/HDFS-15506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17552478#comment-17552478
]
fanshilun commented on HDFS-15506:
--
Hi, [~aajisaka] I found some new java doc compilation failures in jdk11, I will
fix it.
> [JDK 11] Fix javadoc errors in hadoop-hdfs module
> -
>
> Key: HDFS-15506
> URL: https://issues.apache.org/jira/browse/HDFS-15506
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: documentation
>Reporter: Akira Ajisaka
>Assignee: Xieming Li
>Priority: Major
> Labels: newbie
> Fix For: 3.3.1, 3.4.0
>
> Attachments: HDFS-15506.001.patch, HDFS-15506.002.patch
>
>
> {noformat}
> [ERROR]
> /Users/aajisaka/git/hadoop/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeAdminDefaultMonitor.java:43:
> error: self-closing element not allowed
> [ERROR] *
> [ERROR]^
> [ERROR]
> /Users/aajisaka/git/hadoop/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java:682:
> error: malformed HTML
> [ERROR]* a NameNode per second. Values <= 0 disable throttling. This
> affects
> [ERROR]^
> [ERROR]
> /Users/aajisaka/git/hadoop/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSUtil.java:1780:
> error: exception not thrown: java.io.FileNotFoundException
> [ERROR]* @throws FileNotFoundException
> [ERROR] ^
> [ERROR]
> /Users/aajisaka/git/hadoop/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/DirectorySnapshottableFeature.java:176:
> error: @param name not found
> [ERROR]* @param mtime The snapshot creation time set by Time.now().
> [ERROR] ^
> [ERROR]
> /Users/aajisaka/git/hadoop/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSAdmin.java:2187:
> error: exception not thrown: java.lang.Exception
> [ERROR]* @exception Exception if the filesystem does not exist.
> [ERROR] ^
> {noformat}
> Full error log:
> https://gist.github.com/aajisaka/a0c16f0408a623e798dd7df29fbddf82
> How to reproduce the failure:
> * Remove {{true}} from pom.xml
> * Run {{mvn process-sources javadoc:javadoc-no-fork}}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-16629) [JDK 11] Fix javadoc warnings in hadoop-hdfs module
fanshilun created HDFS-16629: Summary: [JDK 11] Fix javadoc warnings in hadoop-hdfs module Key: HDFS-16629 URL: https://issues.apache.org/jira/browse/HDFS-16629 Project: Hadoop HDFS Issue Type: Improvement Components: hdfs Affects Versions: 3.4.0, 3.3.4 Reporter: fanshilun Assignee: fanshilun Fix For: 3.4.0, 3.3.4 During compilation of the most recently committed code, a java doc waring appeared and I will fix it. -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work started] (HDFS-16629) [JDK 11] Fix javadoc warnings in hadoop-hdfs module
[ https://issues.apache.org/jira/browse/HDFS-16629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-16629 started by fanshilun. > [JDK 11] Fix javadoc warnings in hadoop-hdfs module > > > Key: HDFS-16629 > URL: https://issues.apache.org/jira/browse/HDFS-16629 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs >Affects Versions: 3.4.0, 3.3.4 >Reporter: fanshilun >Assignee: fanshilun >Priority: Minor > Fix For: 3.4.0, 3.3.4 > > > During compilation of the most recently committed code, a java doc waring > appeared and I will fix it. -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-16629) [JDK 11] Fix javadoc warnings in hadoop-hdfs module
[
https://issues.apache.org/jira/browse/HDFS-16629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
fanshilun updated HDFS-16629:
-
Description:
During compilation of the most recently committed code, a java doc waring
appeared and I will fix it.
{code:java}
1 error
100 warnings
[INFO]
[INFO] BUILD FAILURE
[INFO]
[INFO] Total time: 37.132 s
[INFO] Finished at: 2022-06-09T17:07:12Z
[INFO]
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-javadoc-plugin:3.0.1:javadoc-no-fork
(default-cli) on project hadoop-hdfs: An error has occurred in Javadoc report
generation:
[ERROR] Exit code: 1 - javadoc: warning - You have specified the HTML version
as HTML 4.01 by using the -html4 option.
[ERROR] The default is currently HTML5 and the support for HTML 4.01 will be
removed
[ERROR] in a future release. To suppress this warning, please ensure that any
HTML constructs {code}
was:During compilation of the most recently committed code, a java doc waring
appeared and I will fix it.
> [JDK 11] Fix javadoc warnings in hadoop-hdfs module
>
>
> Key: HDFS-16629
> URL: https://issues.apache.org/jira/browse/HDFS-16629
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: hdfs
>Affects Versions: 3.4.0, 3.3.4
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Minor
> Fix For: 3.4.0, 3.3.4
>
>
> During compilation of the most recently committed code, a java doc waring
> appeared and I will fix it.
> {code:java}
> 1 error
> 100 warnings
> [INFO]
>
> [INFO] BUILD FAILURE
> [INFO]
>
> [INFO] Total time: 37.132 s
> [INFO] Finished at: 2022-06-09T17:07:12Z
> [INFO]
>
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-javadoc-plugin:3.0.1:javadoc-no-fork
> (default-cli) on project hadoop-hdfs: An error has occurred in Javadoc report
> generation:
> [ERROR] Exit code: 1 - javadoc: warning - You have specified the HTML version
> as HTML 4.01 by using the -html4 option.
> [ERROR] The default is currently HTML5 and the support for HTML 4.01 will be
> removed
> [ERROR] in a future release. To suppress this warning, please ensure that any
> HTML constructs {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16629) [JDK 11] Fix javadoc warnings in hadoop-hdfs module
[
https://issues.apache.org/jira/browse/HDFS-16629?focusedWorklogId=780149&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-780149
]
ASF GitHub Bot logged work on HDFS-16629:
-
Author: ASF GitHub Bot
Created on: 10/Jun/22 00:20
Start Date: 10/Jun/22 00:20
Worklog Time Spent: 10m
Work Description: slfan1989 opened a new pull request, #4423:
URL: https://github.com/apache/hadoop/pull/4423
JIRA: HDFS-16629. [JDK 11] Fix javadoc warnings in hadoop-hdfs module.
During compilation of the most recently committed code, a java doc waring
appeared and I will fix it.
```
1 error
100 warnings
[INFO]
[INFO] BUILD FAILURE
[INFO]
[INFO] Total time: 37.132 s
[INFO] Finished at: 2022-06-09T17:07:12Z
[INFO]
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-javadoc-plugin:3.0.1:javadoc-no-fork
(default-cli) on project hadoop-hdfs: An error has occurred in Javadoc report
generation:
[ERROR] Exit code: 1 - javadoc: warning - You have specified the HTML
version as HTML 4.01 by using the -html4 option.
[ERROR] The default is currently HTML5 and the support for HTML 4.01 will be
removed
[ERROR] in a future release. To suppress this warning, please ensure that
any HTML constructs
```
Issue Time Tracking
---
Worklog Id: (was: 780149)
Remaining Estimate: 0h
Time Spent: 10m
> [JDK 11] Fix javadoc warnings in hadoop-hdfs module
>
>
> Key: HDFS-16629
> URL: https://issues.apache.org/jira/browse/HDFS-16629
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: hdfs
>Affects Versions: 3.4.0, 3.3.4
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Minor
> Fix For: 3.4.0, 3.3.4
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> During compilation of the most recently committed code, a java doc waring
> appeared and I will fix it.
> {code:java}
> 1 error
> 100 warnings
> [INFO]
>
> [INFO] BUILD FAILURE
> [INFO]
>
> [INFO] Total time: 37.132 s
> [INFO] Finished at: 2022-06-09T17:07:12Z
> [INFO]
>
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-javadoc-plugin:3.0.1:javadoc-no-fork
> (default-cli) on project hadoop-hdfs: An error has occurred in Javadoc report
> generation:
> [ERROR] Exit code: 1 - javadoc: warning - You have specified the HTML version
> as HTML 4.01 by using the -html4 option.
> [ERROR] The default is currently HTML5 and the support for HTML 4.01 will be
> removed
> [ERROR] in a future release. To suppress this warning, please ensure that any
> HTML constructs {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-16629) [JDK 11] Fix javadoc warnings in hadoop-hdfs module
[
https://issues.apache.org/jira/browse/HDFS-16629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HDFS-16629:
--
Labels: pull-request-available (was: )
> [JDK 11] Fix javadoc warnings in hadoop-hdfs module
>
>
> Key: HDFS-16629
> URL: https://issues.apache.org/jira/browse/HDFS-16629
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: hdfs
>Affects Versions: 3.4.0, 3.3.4
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Minor
> Labels: pull-request-available
> Fix For: 3.4.0, 3.3.4
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> During compilation of the most recently committed code, a java doc waring
> appeared and I will fix it.
> {code:java}
> 1 error
> 100 warnings
> [INFO]
>
> [INFO] BUILD FAILURE
> [INFO]
>
> [INFO] Total time: 37.132 s
> [INFO] Finished at: 2022-06-09T17:07:12Z
> [INFO]
>
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-javadoc-plugin:3.0.1:javadoc-no-fork
> (default-cli) on project hadoop-hdfs: An error has occurred in Javadoc report
> generation:
> [ERROR] Exit code: 1 - javadoc: warning - You have specified the HTML version
> as HTML 4.01 by using the -html4 option.
> [ERROR] The default is currently HTML5 and the support for HTML 4.01 will be
> removed
> [ERROR] in a future release. To suppress this warning, please ensure that any
> HTML constructs {code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16557) BootstrapStandby failed because of checking gap for inprogress EditLogInputStream
[
https://issues.apache.org/jira/browse/HDFS-16557?focusedWorklogId=780179&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-780179
]
ASF GitHub Bot logged work on HDFS-16557:
-
Author: ASF GitHub Bot
Created on: 10/Jun/22 02:29
Start Date: 10/Jun/22 02:29
Worklog Time Spent: 10m
Work Description: tomscut commented on PR #4219:
URL: https://github.com/apache/hadoop/pull/4219#issuecomment-1151852402
> Thanks @tomscut , after tracing the code, I think we cannot add
`elis.isInProgress()`.
>
> And I will explain my ideas trough questions and answers. **Question one:
Why was INVALID_TXID considered in the original code?**
>
> * CheckForGaps method is used to check whether streams contains continuous
TXids from fromTxId to toAtLeastTxid
> * LastTxId equals INVALID_TXID means the stream is in progress
> * toAtLeastTxid maybe abnormal value, like Long.MaxValue. So the
CheckForGaps method only need to cover the latest inprogress segment.
>
> **Question two: What is the difference between INVALID_TXID and is
InProgress()？**
>
> * Before introducing [SBN READ], LastTxId equals INVALID_TXID means the
stream is in progress. And stream is in progress means it's lastTxId is
INVALID_TXID.
> * But after introducing [SBN READ], LastTxId equals INVALID_TXID means the
stream is in progress. But stream is in progress cannot mean it's lastTxId is
INVALID_TXID. Because introducing getJournaledEdits.
> * So if we add `elis.isInProgress()` in CheckForGaps, it cannot cover the
last writing segments which actual contains latest edit.
>
> Please correct me if anything is wrong.
Thanks @ZanderXu for your comment. Please refer to the stack.
When we set `dfs.ha.tail-edits.in-progress=true`, the txID can be read by
getJournaledEdits (there is no gap actually) . But there is an GAP exception
thrown.
Issue Time Tracking
---
Worklog Id: (was: 780179)
Time Spent: 3h 10m (was: 3h)
> BootstrapStandby failed because of checking gap for inprogress
> EditLogInputStream
> -
>
> Key: HDFS-16557
> URL: https://issues.apache.org/jira/browse/HDFS-16557
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Tao Li
>Assignee: Tao Li
>Priority: Major
> Labels: pull-request-available
> Attachments: image-2022-04-22-17-17-14-577.png,
> image-2022-04-22-17-17-14-618.png, image-2022-04-22-17-17-23-113.png,
> image-2022-04-22-17-17-32-487.png
>
> Time Spent: 3h 10m
> Remaining Estimate: 0h
>
> The lastTxId of an inprogress EditLogInputStream lastTxId isn't necessarily
> HdfsServerConstants.INVALID_TXID. We can determine its status directly by
> EditLogInputStream#isInProgress.
> We introduced [SBN READ], and set
> {color:#ff}{{dfs.ha.tail-edits.in-progress=true}}{color}. Then
> bootstrapStandby, the EditLogInputStream of inProgress is misjudged,
> resulting in a gap check failure, which causes bootstrapStandby to fail.
> hdfs namenode -bootstrapStandby
> !image-2022-04-22-17-17-32-487.png|width=766,height=161!
> !image-2022-04-22-17-17-14-577.png|width=598,height=187!
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-16628) RBF: kerbose user remove Non-default namespace data failed
[ https://issues.apache.org/jira/browse/HDFS-16628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDFS-16628: -- Labels: pull-request-available (was: ) > RBF: kerbose user remove Non-default namespace data failed > -- > > Key: HDFS-16628 > URL: https://issues.apache.org/jira/browse/HDFS-16628 > Project: Hadoop HDFS > Issue Type: Bug > Components: rbf >Reporter: Xiping Zhang >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16628) RBF: kerbose user remove Non-default namespace data failed
[ https://issues.apache.org/jira/browse/HDFS-16628?focusedWorklogId=780180&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-780180 ] ASF GitHub Bot logged work on HDFS-16628: - Author: ASF GitHub Bot Created on: 10/Jun/22 02:48 Start Date: 10/Jun/22 02:48 Worklog Time Spent: 10m Work Description: zhangxiping1 opened a new pull request, #4424: URL: https://github.com/apache/hadoop/pull/4424 HDFS-16628 RBF: kerbose user remove Non-default namespace data failed Issue Time Tracking --- Worklog Id: (was: 780180) Remaining Estimate: 0h Time Spent: 10m > RBF: kerbose user remove Non-default namespace data failed > -- > > Key: HDFS-16628 > URL: https://issues.apache.org/jira/browse/HDFS-16628 > Project: Hadoop HDFS > Issue Type: Bug > Components: rbf >Reporter: Xiping Zhang >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16628) RBF: kerbose user remove Non-default namespace data failed
[
https://issues.apache.org/jira/browse/HDFS-16628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17552509#comment-17552509
]
Xiping Zhang commented on HDFS-16628:
-
MountTableResolver.java
{code:java}
//代码占位符
@VisibleForTesting
public static String getTrashRoot() throws IOException {
// Gets the Trash directory for the current user.
return FileSystem.USER_HOME_PREFIX + "/" +
RouterRpcServer.getRemoteUser().getUserName() + "/" + // should get
shortUserName
FileSystem.TRASH_PREFIX;
} {code}
Should be consistent with FileSytem.java
{code:java}
//代码占位符
/** Return the current user's home directory in this FileSystem.
* The default implementation returns {@code "/user/$USER/"}.
*/
public Path getHomeDirectory() {
String username;
try {
username = UserGroupInformation.getCurrentUser().getShortUserName();
} catch(IOException ex) {
LOGGER.warn("Unable to get user name. Fall back to system property " +
"user.name", ex);
username = System.getProperty("user.name");
}
return this.makeQualified(
new Path(USER_HOME_PREFIX + "/" + username));
} {code}
> RBF: kerbose user remove Non-default namespace data failed
> --
>
> Key: HDFS-16628
> URL: https://issues.apache.org/jira/browse/HDFS-16628
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: rbf
>Reporter: Xiping Zhang
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-16628) RBF: kerbose user remove Non-default namespace data failed
[ https://issues.apache.org/jira/browse/HDFS-16628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiping Zhang updated HDFS-16628: Fix Version/s: 3.4.0 Affects Version/s: 3.4.0 > RBF: kerbose user remove Non-default namespace data failed > -- > > Key: HDFS-16628 > URL: https://issues.apache.org/jira/browse/HDFS-16628 > Project: Hadoop HDFS > Issue Type: Bug > Components: rbf >Affects Versions: 3.4.0 >Reporter: Xiping Zhang >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16557) BootstrapStandby failed because of checking gap for inprogress EditLogInputStream
[
https://issues.apache.org/jira/browse/HDFS-16557?focusedWorklogId=780183&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-780183
]
ASF GitHub Bot logged work on HDFS-16557:
-
Author: ASF GitHub Bot
Created on: 10/Jun/22 03:07
Start Date: 10/Jun/22 03:07
Worklog Time Spent: 10m
Work Description: ZanderXu commented on PR #4219:
URL: https://github.com/apache/hadoop/pull/4219#issuecomment-1151877997
OK, back to BootstrapStandby GAP.
Form this stack information, I got that it try to get streams from
1049842441 to 1050196644. But cannot get the txid 1049842441 from the result
streams.
So I think we should to trace the root cause, why can't we find txid
1049842441 in the return result of `selectInputStreams(streams, 1049842441,
true, true)`?
Please correct me if anything is wrong.
Issue Time Tracking
---
Worklog Id: (was: 780183)
Time Spent: 3h 20m (was: 3h 10m)
> BootstrapStandby failed because of checking gap for inprogress
> EditLogInputStream
> -
>
> Key: HDFS-16557
> URL: https://issues.apache.org/jira/browse/HDFS-16557
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Tao Li
>Assignee: Tao Li
>Priority: Major
> Labels: pull-request-available
> Attachments: image-2022-04-22-17-17-14-577.png,
> image-2022-04-22-17-17-14-618.png, image-2022-04-22-17-17-23-113.png,
> image-2022-04-22-17-17-32-487.png
>
> Time Spent: 3h 20m
> Remaining Estimate: 0h
>
> The lastTxId of an inprogress EditLogInputStream lastTxId isn't necessarily
> HdfsServerConstants.INVALID_TXID. We can determine its status directly by
> EditLogInputStream#isInProgress.
> We introduced [SBN READ], and set
> {color:#ff}{{dfs.ha.tail-edits.in-progress=true}}{color}. Then
> bootstrapStandby, the EditLogInputStream of inProgress is misjudged,
> resulting in a gap check failure, which causes bootstrapStandby to fail.
> hdfs namenode -bootstrapStandby
> !image-2022-04-22-17-17-32-487.png|width=766,height=161!
> !image-2022-04-22-17-17-14-577.png|width=598,height=187!
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-16628) RBF: kerbose user remove Non-default namespace data failed
[ https://issues.apache.org/jira/browse/HDFS-16628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiping Zhang updated HDFS-16628: Description: remove data from the router will fail using such a user username/d...@hadoop.com > RBF: kerbose user remove Non-default namespace data failed > -- > > Key: HDFS-16628 > URL: https://issues.apache.org/jira/browse/HDFS-16628 > Project: Hadoop HDFS > Issue Type: Bug > Components: rbf >Affects Versions: 3.4.0 >Reporter: Xiping Zhang >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > Time Spent: 10m > Remaining Estimate: 0h > > remove data from the router will fail using such a user > username/d...@hadoop.com -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16557) BootstrapStandby failed because of checking gap for inprogress EditLogInputStream
[
https://issues.apache.org/jira/browse/HDFS-16557?focusedWorklogId=780186&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-780186
]
ASF GitHub Bot logged work on HDFS-16557:
-
Author: ASF GitHub Bot
Created on: 10/Jun/22 03:14
Start Date: 10/Jun/22 03:14
Worklog Time Spent: 10m
Work Description: tomscut commented on PR #4219:
URL: https://github.com/apache/hadoop/pull/4219#issuecomment-1151882369
> OK, back to BootstrapStandby GAP. Form this stack information, I got that
it try to get streams from 1049842441 to 1050196644. But cannot get the txid
1049842441 from the result streams. So I think we should to trace the root
cause, why can't we find txid 1049842441 in the return result of
`selectInputStreams(streams, 1049842441, true, true)`?
>
> Please correct me if anything is wrong.
Please refer to the discussion with @xkrogen above.
The root cause is the` if` condition (`if(next ==
HdfsServerConstants.INVALID_TXID)`) that does not enter properly.
Issue Time Tracking
---
Worklog Id: (was: 780186)
Time Spent: 3.5h (was: 3h 20m)
> BootstrapStandby failed because of checking gap for inprogress
> EditLogInputStream
> -
>
> Key: HDFS-16557
> URL: https://issues.apache.org/jira/browse/HDFS-16557
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Tao Li
>Assignee: Tao Li
>Priority: Major
> Labels: pull-request-available
> Attachments: image-2022-04-22-17-17-14-577.png,
> image-2022-04-22-17-17-14-618.png, image-2022-04-22-17-17-23-113.png,
> image-2022-04-22-17-17-32-487.png
>
> Time Spent: 3.5h
> Remaining Estimate: 0h
>
> The lastTxId of an inprogress EditLogInputStream lastTxId isn't necessarily
> HdfsServerConstants.INVALID_TXID. We can determine its status directly by
> EditLogInputStream#isInProgress.
> We introduced [SBN READ], and set
> {color:#ff}{{dfs.ha.tail-edits.in-progress=true}}{color}. Then
> bootstrapStandby, the EditLogInputStream of inProgress is misjudged,
> resulting in a gap check failure, which causes bootstrapStandby to fail.
> hdfs namenode -bootstrapStandby
> !image-2022-04-22-17-17-32-487.png|width=766,height=161!
> !image-2022-04-22-17-17-14-577.png|width=598,height=187!
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16557) BootstrapStandby failed because of checking gap for inprogress EditLogInputStream
[
https://issues.apache.org/jira/browse/HDFS-16557?focusedWorklogId=780187&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-780187
]
ASF GitHub Bot logged work on HDFS-16557:
-
Author: ASF GitHub Bot
Created on: 10/Jun/22 03:18
Start Date: 10/Jun/22 03:18
Worklog Time Spent: 10m
Work Description: ZanderXu commented on PR #4219:
URL: https://github.com/apache/hadoop/pull/4219#issuecomment-1151885068
Oh, i know, the root cause is that getJournaledEdits returns up to 5000
txids by default. And 1049842441 - 1049837441 = 5000.
I can't reached to 1050196644, so checkForGaps failed.
Issue Time Tracking
---
Worklog Id: (was: 780187)
Time Spent: 3h 40m (was: 3.5h)
> BootstrapStandby failed because of checking gap for inprogress
> EditLogInputStream
> -
>
> Key: HDFS-16557
> URL: https://issues.apache.org/jira/browse/HDFS-16557
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Tao Li
>Assignee: Tao Li
>Priority: Major
> Labels: pull-request-available
> Attachments: image-2022-04-22-17-17-14-577.png,
> image-2022-04-22-17-17-14-618.png, image-2022-04-22-17-17-23-113.png,
> image-2022-04-22-17-17-32-487.png
>
> Time Spent: 3h 40m
> Remaining Estimate: 0h
>
> The lastTxId of an inprogress EditLogInputStream lastTxId isn't necessarily
> HdfsServerConstants.INVALID_TXID. We can determine its status directly by
> EditLogInputStream#isInProgress.
> We introduced [SBN READ], and set
> {color:#ff}{{dfs.ha.tail-edits.in-progress=true}}{color}. Then
> bootstrapStandby, the EditLogInputStream of inProgress is misjudged,
> resulting in a gap check failure, which causes bootstrapStandby to fail.
> hdfs namenode -bootstrapStandby
> !image-2022-04-22-17-17-32-487.png|width=766,height=161!
> !image-2022-04-22-17-17-14-577.png|width=598,height=187!
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16557) BootstrapStandby failed because of checking gap for inprogress EditLogInputStream
[
https://issues.apache.org/jira/browse/HDFS-16557?focusedWorklogId=780189&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-780189
]
ASF GitHub Bot logged work on HDFS-16557:
-
Author: ASF GitHub Bot
Created on: 10/Jun/22 03:22
Start Date: 10/Jun/22 03:22
Worklog Time Spent: 10m
Work Description: ZanderXu commented on PR #4219:
URL: https://github.com/apache/hadoop/pull/4219#issuecomment-1151887486
So in this case, we should change bootstrap logic.
Solution one: set DFS_HA_TAILEDITS_INPROGRESS_KEY to false.
Solution two: call getJournaledEdits multiple times until get the latest
txid, and then go to checkgap
Issue Time Tracking
---
Worklog Id: (was: 780189)
Time Spent: 3h 50m (was: 3h 40m)
> BootstrapStandby failed because of checking gap for inprogress
> EditLogInputStream
> -
>
> Key: HDFS-16557
> URL: https://issues.apache.org/jira/browse/HDFS-16557
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Tao Li
>Assignee: Tao Li
>Priority: Major
> Labels: pull-request-available
> Attachments: image-2022-04-22-17-17-14-577.png,
> image-2022-04-22-17-17-14-618.png, image-2022-04-22-17-17-23-113.png,
> image-2022-04-22-17-17-32-487.png
>
> Time Spent: 3h 50m
> Remaining Estimate: 0h
>
> The lastTxId of an inprogress EditLogInputStream lastTxId isn't necessarily
> HdfsServerConstants.INVALID_TXID. We can determine its status directly by
> EditLogInputStream#isInProgress.
> We introduced [SBN READ], and set
> {color:#ff}{{dfs.ha.tail-edits.in-progress=true}}{color}. Then
> bootstrapStandby, the EditLogInputStream of inProgress is misjudged,
> resulting in a gap check failure, which causes bootstrapStandby to fail.
> hdfs namenode -bootstrapStandby
> !image-2022-04-22-17-17-32-487.png|width=766,height=161!
> !image-2022-04-22-17-17-14-577.png|width=598,height=187!
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16557) BootstrapStandby failed because of checking gap for inprogress EditLogInputStream
[
https://issues.apache.org/jira/browse/HDFS-16557?focusedWorklogId=780191&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-780191
]
ASF GitHub Bot logged work on HDFS-16557:
-
Author: ASF GitHub Bot
Created on: 10/Jun/22 03:28
Start Date: 10/Jun/22 03:28
Worklog Time Spent: 10m
Work Description: ZanderXu commented on PR #4219:
URL: https://github.com/apache/hadoop/pull/4219#issuecomment-1151890746
As I explained above, change to `if (next ==
HdfsServerConstants.INVALID_TXID || elis.isInProgress())` maybe change the
original semantics of the `checkgap` method.
About my explain, do you have any questions？😁 Discuss together and become
more familiar with the relevant logic.
Issue Time Tracking
---
Worklog Id: (was: 780191)
Time Spent: 4h (was: 3h 50m)
> BootstrapStandby failed because of checking gap for inprogress
> EditLogInputStream
> -
>
> Key: HDFS-16557
> URL: https://issues.apache.org/jira/browse/HDFS-16557
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Tao Li
>Assignee: Tao Li
>Priority: Major
> Labels: pull-request-available
> Attachments: image-2022-04-22-17-17-14-577.png,
> image-2022-04-22-17-17-14-618.png, image-2022-04-22-17-17-23-113.png,
> image-2022-04-22-17-17-32-487.png
>
> Time Spent: 4h
> Remaining Estimate: 0h
>
> The lastTxId of an inprogress EditLogInputStream lastTxId isn't necessarily
> HdfsServerConstants.INVALID_TXID. We can determine its status directly by
> EditLogInputStream#isInProgress.
> We introduced [SBN READ], and set
> {color:#ff}{{dfs.ha.tail-edits.in-progress=true}}{color}. Then
> bootstrapStandby, the EditLogInputStream of inProgress is misjudged,
> resulting in a gap check failure, which causes bootstrapStandby to fail.
> hdfs namenode -bootstrapStandby
> !image-2022-04-22-17-17-32-487.png|width=766,height=161!
> !image-2022-04-22-17-17-14-577.png|width=598,height=187!
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16570) RBF: The router using MultipleDestinationMountTableResolver remove Multiple subcluster data under the mount point failed
[
https://issues.apache.org/jira/browse/HDFS-16570?focusedWorklogId=780192&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-780192
]
ASF GitHub Bot logged work on HDFS-16570:
-
Author: ASF GitHub Bot
Created on: 10/Jun/22 03:29
Start Date: 10/Jun/22 03:29
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on PR #4269:
URL: https://github.com/apache/hadoop/pull/4269#issuecomment-1151891187
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 40s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 38m 1s | | trunk passed |
| +1 :green_heart: | compile | 1m 1s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 0m 57s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 0m 44s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 55s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 8s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 15s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 48s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 19s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 41s | | the patch passed |
| +1 :green_heart: | compile | 0m 44s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 0m 44s | | the patch passed |
| +1 :green_heart: | compile | 0m 39s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 0m 39s | | the patch passed |
| +1 :green_heart: | blanks | 0m 1s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 27s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 43s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 39s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 0m 58s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 28s | | the patch passed |
| +1 :green_heart: | shadedclient | 20m 24s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 21m 56s | | hadoop-hdfs-rbf in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 52s | | The patch does not
generate ASF License warnings. |
| | | 119m 4s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4269/3/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4269 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 5dc8306caacd 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 94273aed12c914e3cd5bec3f27390aa9876e519c |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4269/3/testReport/ |
| Max. process+thread count | 2662 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-rbf U:
hadoop-hdfs-project/hadoop-hdfs-rbf |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4269/3/console |
[jira] [Work logged] (HDFS-16557) BootstrapStandby failed because of checking gap for inprogress EditLogInputStream
[
https://issues.apache.org/jira/browse/HDFS-16557?focusedWorklogId=780193&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-780193
]
ASF GitHub Bot logged work on HDFS-16557:
-
Author: ASF GitHub Bot
Created on: 10/Jun/22 03:46
Start Date: 10/Jun/22 03:46
Worklog Time Spent: 10m
Work Description: ZanderXu commented on PR #4219:
URL: https://github.com/apache/hadoop/pull/4219#issuecomment-1151901548
> When we set dfs.ha.tail-edits.in-progress=true, the edits can be read by
getJournaledEdits (there is no gap actually) . But there is an GAP exception
thrown.
I think there is a gap here because bootstrap expects to get 1050196644
txid, but can't find it in the result. So throwing GAP Exception is ok.
Issue Time Tracking
---
Worklog Id: (was: 780193)
Time Spent: 4h 10m (was: 4h)
> BootstrapStandby failed because of checking gap for inprogress
> EditLogInputStream
> -
>
> Key: HDFS-16557
> URL: https://issues.apache.org/jira/browse/HDFS-16557
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Tao Li
>Assignee: Tao Li
>Priority: Major
> Labels: pull-request-available
> Attachments: image-2022-04-22-17-17-14-577.png,
> image-2022-04-22-17-17-14-618.png, image-2022-04-22-17-17-23-113.png,
> image-2022-04-22-17-17-32-487.png
>
> Time Spent: 4h 10m
> Remaining Estimate: 0h
>
> The lastTxId of an inprogress EditLogInputStream lastTxId isn't necessarily
> HdfsServerConstants.INVALID_TXID. We can determine its status directly by
> EditLogInputStream#isInProgress.
> We introduced [SBN READ], and set
> {color:#ff}{{dfs.ha.tail-edits.in-progress=true}}{color}. Then
> bootstrapStandby, the EditLogInputStream of inProgress is misjudged,
> resulting in a gap check failure, which causes bootstrapStandby to fail.
> hdfs namenode -bootstrapStandby
> !image-2022-04-22-17-17-32-487.png|width=766,height=161!
> !image-2022-04-22-17-17-14-577.png|width=598,height=187!
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16463) Make dirent cross platform compatible
[ https://issues.apache.org/jira/browse/HDFS-16463?focusedWorklogId=780197&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-780197 ] ASF GitHub Bot logged work on HDFS-16463: - Author: ASF GitHub Bot Created on: 10/Jun/22 04:29 Start Date: 10/Jun/22 04:29 Worklog Time Spent: 10m Work Description: GauthamBanasandra merged PR #4370: URL: https://github.com/apache/hadoop/pull/4370 Issue Time Tracking --- Worklog Id: (was: 780197) Time Spent: 6h 20m (was: 6h 10m) > Make dirent cross platform compatible > - > > Key: HDFS-16463 > URL: https://issues.apache.org/jira/browse/HDFS-16463 > Project: Hadoop HDFS > Issue Type: Improvement > Components: libhdfs++ >Affects Versions: 3.4.0 > Environment: Windows 10 >Reporter: Gautham Banasandra >Assignee: Gautham Banasandra >Priority: Major > Labels: libhdfscpp, pull-request-available > Time Spent: 6h 20m > Remaining Estimate: 0h > > [jnihelper.c|https://github.com/apache/hadoop/blob/1fed18bb2d8ac3dbaecc3feddded30bed918d556/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs/jni_helper.c#L28] > in HDFS native client uses *dirent.h*. This header file isn't available on > Windows. Thus, we need to replace this with a cross platform compatible > implementation for dirent. -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16628) RBF: kerbose user remove Non-default namespace data failed
[
https://issues.apache.org/jira/browse/HDFS-16628?focusedWorklogId=780202&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-780202
]
ASF GitHub Bot logged work on HDFS-16628:
-
Author: ASF GitHub Bot
Created on: 10/Jun/22 04:48
Start Date: 10/Jun/22 04:48
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on PR #4424:
URL: https://github.com/apache/hadoop/pull/4424#issuecomment-1151935757
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 41s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 37m 30s | | trunk passed |
| +1 :green_heart: | compile | 1m 1s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 0m 57s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 0m 53s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 2s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 7s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 19s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 48s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 5s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 41s | | the patch passed |
| +1 :green_heart: | compile | 0m 44s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 0m 44s | | the patch passed |
| +1 :green_heart: | compile | 0m 40s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 0m 40s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 27s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 41s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 42s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 0m 57s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 28s | | the patch passed |
| +1 :green_heart: | shadedclient | 20m 32s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 21m 41s | | hadoop-hdfs-rbf in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 54s | | The patch does not
generate ASF License warnings. |
| | | 118m 57s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4424/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4424 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux a003555ec08e 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6
11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 61dbbdce1617dedabd4e5d9e8b35009ed13c0152 |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4424/1/testReport/ |
| Max. process+thread count | 2752 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-rbf U:
hadoop-hdfs-project/hadoop-hdfs-rbf |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4424/1/console |
[jira] [Work logged] (HDFS-16629) [JDK 11] Fix javadoc warnings in hadoop-hdfs module
[
https://issues.apache.org/jira/browse/HDFS-16629?focusedWorklogId=780223&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-780223
]
ASF GitHub Bot logged work on HDFS-16629:
-
Author: ASF GitHub Bot
Created on: 10/Jun/22 06:29
Start Date: 10/Jun/22 06:29
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on PR #4423:
URL: https://github.com/apache/hadoop/pull/4423#issuecomment-1152007711
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 36s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 37m 32s | | trunk passed |
| +1 :green_heart: | compile | 1m 44s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 1m 34s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 24s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 50s | | trunk passed |
| -1 :x: | javadoc | 1m 22s |
[/branch-javadoc-hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4423/1/artifact/out/branch-javadoc-hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt)
| hadoop-hdfs in trunk failed with JDK Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1. |
| +1 :green_heart: | javadoc | 1m 46s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 53s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 33s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 24s | | the patch passed |
| +1 :green_heart: | compile | 1m 33s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 1m 33s | | the patch passed |
| +1 :green_heart: | compile | 1m 49s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 1m 49s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 59s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 28s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 54s | |
hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 generated 0 new + 99
unchanged - 1 fixed = 99 total (was 100) |
| +1 :green_heart: | javadoc | 1m 26s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 29s | | the patch passed |
| +1 :green_heart: | shadedclient | 24m 20s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 256m 14s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4423/1/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 1m 5s | | The patch does not
generate ASF License warnings. |
| | | 367m 34s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests |
hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaPlacement |
| | hadoop.hdfs.TestRollingUpgrade |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4423/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4423 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
[jira] [Work logged] (HDFS-16621) Remove unused JNStorage#getCurrentDir()
[ https://issues.apache.org/jira/browse/HDFS-16621?focusedWorklogId=780225&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-780225 ] ASF GitHub Bot logged work on HDFS-16621: - Author: ASF GitHub Bot Created on: 10/Jun/22 06:42 Start Date: 10/Jun/22 06:42 Worklog Time Spent: 10m Work Description: ayushtkn merged PR #4404: URL: https://github.com/apache/hadoop/pull/4404 Issue Time Tracking --- Worklog Id: (was: 780225) Time Spent: 1.5h (was: 1h 20m) > Remove unused JNStorage#getCurrentDir() > --- > > Key: HDFS-16621 > URL: https://issues.apache.org/jira/browse/HDFS-16621 > Project: Hadoop HDFS > Issue Type: Improvement > Components: journal-node, qjm >Affects Versions: 3.3.0 >Reporter: JiangHua Zhu >Assignee: JiangHua Zhu >Priority: Minor > Labels: pull-request-available > Time Spent: 1.5h > Remaining Estimate: 0h > > There is no use of getCurrentDir() anywhere in JNStorage, we should remove it. -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16621) Remove unused JNStorage#getCurrentDir()
[ https://issues.apache.org/jira/browse/HDFS-16621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17552586#comment-17552586 ] Ayush Saxena commented on HDFS-16621: - Committed to trunk. Thanx [~jianghuazhu] for the contribution!!! > Remove unused JNStorage#getCurrentDir() > --- > > Key: HDFS-16621 > URL: https://issues.apache.org/jira/browse/HDFS-16621 > Project: Hadoop HDFS > Issue Type: Improvement > Components: journal-node, qjm >Affects Versions: 3.3.0 >Reporter: JiangHua Zhu >Assignee: JiangHua Zhu >Priority: Minor > Labels: pull-request-available > Time Spent: 1.5h > Remaining Estimate: 0h > > There is no use of getCurrentDir() anywhere in JNStorage, we should remove it. -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Resolved] (HDFS-16621) Remove unused JNStorage#getCurrentDir()
[ https://issues.apache.org/jira/browse/HDFS-16621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena resolved HDFS-16621. - Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed > Remove unused JNStorage#getCurrentDir() > --- > > Key: HDFS-16621 > URL: https://issues.apache.org/jira/browse/HDFS-16621 > Project: Hadoop HDFS > Issue Type: Improvement > Components: journal-node, qjm >Affects Versions: 3.3.0 >Reporter: JiangHua Zhu >Assignee: JiangHua Zhu >Priority: Minor > Labels: pull-request-available > Fix For: 3.4.0 > > Time Spent: 1.5h > Remaining Estimate: 0h > > There is no use of getCurrentDir() anywhere in JNStorage, we should remove it. -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
