[jira] [Commented] (HDFS-16802) Print options when accessing ClientProtocol#rename2()
[
https://issues.apache.org/jira/browse/HDFS-16802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17617492#comment-17617492
]
ASF GitHub Bot commented on HDFS-16802:
---
jianghuazhu commented on code in PR #5013:
URL: https://github.com/apache/hadoop/pull/5013#discussion_r995373375
##
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirRenameOp.java:
##
@@ -249,7 +249,7 @@ static RenameResult renameToInt(
String dst = dstArg;
if (NameNode.stateChangeLog.isDebugEnabled()) {
NameNode.stateChangeLog.debug("DIR* NameSystem.renameTo: with options -"
+
- " " + src + " to " + dst);
+ " {} to {}, options={}", src, dst, Arrays.toString(options));
Review Comment:
`
if (NameNode.stateChangeLog.isDebugEnabled()) {
..
}
`
In Hadoop, this phenomenon seems to be a lot.
> Print options when accessing ClientProtocol#rename2()
> -
>
> Key: HDFS-16802
> URL: https://issues.apache.org/jira/browse/HDFS-16802
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
>Affects Versions: 3.3.4
>Reporter: JiangHua Zhu
>Assignee: JiangHua Zhu
>Priority: Minor
> Labels: pull-request-available
>
> When accessing ClientProtocol#rename2(), the carried options cannot be seen
> in the log. Here is some log information:
> {code:java}
> 2022-10-13 10:21:10,727 [Listener at localhost/59732] DEBUG hdfs.StateChange
> (FSDirRenameOp.java:renameToInt(255)) - DIR* NameSystem.renameTo: with
> options - /testNamenodeRetryCache/testRename2/src to
> /testNamenodeRetryCache/testRename2/target
> {code}
> We should improve this, maybe printing options would be better.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16802) Print options when accessing ClientProtocol#rename2()
[
https://issues.apache.org/jira/browse/HDFS-16802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17617491#comment-17617491
]
ASF GitHub Bot commented on HDFS-16802:
---
jianghuazhu commented on code in PR #5013:
URL: https://github.com/apache/hadoop/pull/5013#discussion_r995372224
##
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirRenameOp.java:
##
@@ -249,7 +249,7 @@ static RenameResult renameToInt(
String dst = dstArg;
if (NameNode.stateChangeLog.isDebugEnabled()) {
NameNode.stateChangeLog.debug("DIR* NameSystem.renameTo: with options -"
+
- " " + src + " to " + dst);
+ " {} to {}, options={}", src, dst, Arrays.toString(options));
Review Comment:
Done, waiting for UT.
> Print options when accessing ClientProtocol#rename2()
> -
>
> Key: HDFS-16802
> URL: https://issues.apache.org/jira/browse/HDFS-16802
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
>Affects Versions: 3.3.4
>Reporter: JiangHua Zhu
>Assignee: JiangHua Zhu
>Priority: Minor
> Labels: pull-request-available
>
> When accessing ClientProtocol#rename2(), the carried options cannot be seen
> in the log. Here is some log information:
> {code:java}
> 2022-10-13 10:21:10,727 [Listener at localhost/59732] DEBUG hdfs.StateChange
> (FSDirRenameOp.java:renameToInt(255)) - DIR* NameSystem.renameTo: with
> options - /testNamenodeRetryCache/testRename2/src to
> /testNamenodeRetryCache/testRename2/target
> {code}
> We should improve this, maybe printing options would be better.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16802) Print options when accessing ClientProtocol#rename2()
[
https://issues.apache.org/jira/browse/HDFS-16802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17617472#comment-17617472
]
ASF GitHub Bot commented on HDFS-16802:
---
ZanderXu commented on code in PR #5013:
URL: https://github.com/apache/hadoop/pull/5013#discussion_r995344427
##
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirRenameOp.java:
##
@@ -249,7 +249,7 @@ static RenameResult renameToInt(
String dst = dstArg;
if (NameNode.stateChangeLog.isDebugEnabled()) {
NameNode.stateChangeLog.debug("DIR* NameSystem.renameTo: with options -"
+
- " " + src + " to " + dst);
+ " {} to {}, options={}", src, dst, Arrays.toString(options));
Review Comment:
Can remove `if (NameNode.stateChangeLog.isDebugEnabled()) {`
> Print options when accessing ClientProtocol#rename2()
> -
>
> Key: HDFS-16802
> URL: https://issues.apache.org/jira/browse/HDFS-16802
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
>Affects Versions: 3.3.4
>Reporter: JiangHua Zhu
>Assignee: JiangHua Zhu
>Priority: Minor
> Labels: pull-request-available
>
> When accessing ClientProtocol#rename2(), the carried options cannot be seen
> in the log. Here is some log information:
> {code:java}
> 2022-10-13 10:21:10,727 [Listener at localhost/59732] DEBUG hdfs.StateChange
> (FSDirRenameOp.java:renameToInt(255)) - DIR* NameSystem.renameTo: with
> options - /testNamenodeRetryCache/testRename2/src to
> /testNamenodeRetryCache/testRename2/target
> {code}
> We should improve this, maybe printing options would be better.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16703) Enable RPC Timeout for some protocols of NameNode.
[ https://issues.apache.org/jira/browse/HDFS-16703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17617449#comment-17617449 ] ASF GitHub Bot commented on HDFS-16703: --- ZanderXu commented on PR #4660: URL: https://github.com/apache/hadoop/pull/4660#issuecomment-1278453491 @slfan1989 Sir, about this PR, if you thinks it's difficult to configure it, how about just enabling a configurable timeout for NamenodeProtocolPB? Because we encountered many times this problem in our prod environment that RBF can not sense the crashed namenode in time, because the `NamenodeHeartbeatService` are blocked to waiting for the response from namenode for a long time. > Enable RPC Timeout for some protocols of NameNode. > -- > > Key: HDFS-16703 > URL: https://issues.apache.org/jira/browse/HDFS-16703 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: ZanderXu >Assignee: ZanderXu >Priority: Major > Labels: pull-request-available > Time Spent: 1h 40m > Remaining Estimate: 0h > > When I read some code about protocol, I found that only > ClientNamenodeProtocolPB proxy with RPC timeout, other protocolPB proxy > without RPC timeout, such as RefreshAuthorizationPolicyProtocolPB, > RefreshUserMappingsProtocolPB, RefreshCallQueueProtocolPB, > GetUserMappingsProtocolPB and NamenodeProtocolPB. > > If proxy without rpc timeout, it will be blocked for a long time if the NN > machine crash or bad network during writing or reading with NN. > > So I feel that we should enable RPC timeout for all ProtocolPB. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16802) Print options when accessing ClientProtocol#rename2()
[
https://issues.apache.org/jira/browse/HDFS-16802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17617434#comment-17617434
]
ASF GitHub Bot commented on HDFS-16802:
---
ZanderXu commented on code in PR #5013:
URL: https://github.com/apache/hadoop/pull/5013#discussion_r995309524
##
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirRenameOp.java:
##
@@ -249,7 +249,7 @@ static RenameResult renameToInt(
String dst = dstArg;
if (NameNode.stateChangeLog.isDebugEnabled()) {
NameNode.stateChangeLog.debug("DIR* NameSystem.renameTo: with options -"
+
- " " + src + " to " + dst);
+ " {} to {}, options={}", src, dst, Arrays.toString(options));
Review Comment:
ok
> Print options when accessing ClientProtocol#rename2()
> -
>
> Key: HDFS-16802
> URL: https://issues.apache.org/jira/browse/HDFS-16802
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
>Affects Versions: 3.3.4
>Reporter: JiangHua Zhu
>Assignee: JiangHua Zhu
>Priority: Minor
> Labels: pull-request-available
>
> When accessing ClientProtocol#rename2(), the carried options cannot be seen
> in the log. Here is some log information:
> {code:java}
> 2022-10-13 10:21:10,727 [Listener at localhost/59732] DEBUG hdfs.StateChange
> (FSDirRenameOp.java:renameToInt(255)) - DIR* NameSystem.renameTo: with
> options - /testNamenodeRetryCache/testRename2/src to
> /testNamenodeRetryCache/testRename2/target
> {code}
> We should improve this, maybe printing options would be better.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16802) Print options when accessing ClientProtocol#rename2()
[
https://issues.apache.org/jira/browse/HDFS-16802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17617426#comment-17617426
]
ASF GitHub Bot commented on HDFS-16802:
---
jianghuazhu commented on code in PR #5013:
URL: https://github.com/apache/hadoop/pull/5013#discussion_r995305422
##
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirRenameOp.java:
##
@@ -249,7 +249,7 @@ static RenameResult renameToInt(
String dst = dstArg;
if (NameNode.stateChangeLog.isDebugEnabled()) {
NameNode.stateChangeLog.debug("DIR* NameSystem.renameTo: with options -"
+
- " " + src + " to " + dst);
+ " {} to {}, options={}", src, dst, Arrays.toString(options));
Review Comment:
I agree.
Here are some audit logs:
`
2022-10-14 11:16:30,825 [Listener at localhost/51113] INFO
FSNamesystem.audit (FSNamesystem.java:logAuditMessage(8853)) - allowed=true
ugi=hdfs (auth:SIMPLE) ip=null cmd=rename ( options=[NONE])
src=/testNamenodeRetryCache/testRename2/src
dst=/testNamenodeRetryCache/testRename2/target perm=hdfs:supergroup:rwxrwxrwx
proto=null
`
Should we keep a consistent norm?
New format:
`
2022-10-14 11:20:18,813 [Listener at localhost/58086] DEBUG
hdfs.StateChange(FSDirRenameOp.java:renameToInt(256)) - DIR*
NameSystem.renameTo: with options=[NONE]
/testNamenodeRetryCache/testRename2/src to /testNamenodeRetryCache
/testRename2/target
`
> Print options when accessing ClientProtocol#rename2()
> -
>
> Key: HDFS-16802
> URL: https://issues.apache.org/jira/browse/HDFS-16802
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
>Affects Versions: 3.3.4
>Reporter: JiangHua Zhu
>Assignee: JiangHua Zhu
>Priority: Minor
> Labels: pull-request-available
>
> When accessing ClientProtocol#rename2(), the carried options cannot be seen
> in the log. Here is some log information:
> {code:java}
> 2022-10-13 10:21:10,727 [Listener at localhost/59732] DEBUG hdfs.StateChange
> (FSDirRenameOp.java:renameToInt(255)) - DIR* NameSystem.renameTo: with
> options - /testNamenodeRetryCache/testRename2/src to
> /testNamenodeRetryCache/testRename2/target
> {code}
> We should improve this, maybe printing options would be better.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16802) Print options when accessing ClientProtocol#rename2()
[
https://issues.apache.org/jira/browse/HDFS-16802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17617417#comment-17617417
]
ASF GitHub Bot commented on HDFS-16802:
---
ZanderXu commented on code in PR #5013:
URL: https://github.com/apache/hadoop/pull/5013#discussion_r995286333
##
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirRenameOp.java:
##
@@ -249,7 +249,7 @@ static RenameResult renameToInt(
String dst = dstArg;
if (NameNode.stateChangeLog.isDebugEnabled()) {
NameNode.stateChangeLog.debug("DIR* NameSystem.renameTo: with options -"
+
- " " + src + " to " + dst);
+ " {} to {}, options={}", src, dst, Arrays.toString(options));
Review Comment:
How about changing the code as bellow?
```
NameNode.stateChangeLog.debug("DIR* NameSystem.renameTo: with options {} -
{} to {}",
Arrays.toString(options), src, dst);
```
> Print options when accessing ClientProtocol#rename2()
> -
>
> Key: HDFS-16802
> URL: https://issues.apache.org/jira/browse/HDFS-16802
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
>Affects Versions: 3.3.4
>Reporter: JiangHua Zhu
>Assignee: JiangHua Zhu
>Priority: Minor
> Labels: pull-request-available
>
> When accessing ClientProtocol#rename2(), the carried options cannot be seen
> in the log. Here is some log information:
> {code:java}
> 2022-10-13 10:21:10,727 [Listener at localhost/59732] DEBUG hdfs.StateChange
> (FSDirRenameOp.java:renameToInt(255)) - DIR* NameSystem.renameTo: with
> options - /testNamenodeRetryCache/testRename2/src to
> /testNamenodeRetryCache/testRename2/target
> {code}
> We should improve this, maybe printing options would be better.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16802) Print options when accessing ClientProtocol#rename2()
[
https://issues.apache.org/jira/browse/HDFS-16802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17617398#comment-17617398
]
ASF GitHub Bot commented on HDFS-16802:
---
jianghuazhu commented on PR #5013:
URL: https://github.com/apache/hadoop/pull/5013#issuecomment-1278366433
@ZanderXu , can you help review this pr again?
Thanks.
> Print options when accessing ClientProtocol#rename2()
> -
>
> Key: HDFS-16802
> URL: https://issues.apache.org/jira/browse/HDFS-16802
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
>Affects Versions: 3.3.4
>Reporter: JiangHua Zhu
>Assignee: JiangHua Zhu
>Priority: Minor
> Labels: pull-request-available
>
> When accessing ClientProtocol#rename2(), the carried options cannot be seen
> in the log. Here is some log information:
> {code:java}
> 2022-10-13 10:21:10,727 [Listener at localhost/59732] DEBUG hdfs.StateChange
> (FSDirRenameOp.java:renameToInt(255)) - DIR* NameSystem.renameTo: with
> options - /testNamenodeRetryCache/testRename2/src to
> /testNamenodeRetryCache/testRename2/target
> {code}
> We should improve this, maybe printing options would be better.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16753) WebHDFSHandler should reject non-compliant requests
[
https://issues.apache.org/jira/browse/HDFS-16753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17617290#comment-17617290
]
ASF GitHub Bot commented on HDFS-16753:
---
hadoop-yetus commented on PR #4834:
URL: https://github.com/apache/hadoop/pull/4834#issuecomment-1278127660
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 40s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 39m 21s | | trunk passed |
| +1 :green_heart: | compile | 1m 35s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 1m 40s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 14s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 40s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 23s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 39s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 43s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 27s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 23s | | the patch passed |
| +1 :green_heart: | compile | 1m 23s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 1m 23s | | the patch passed |
| +1 :green_heart: | compile | 1m 18s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 1m 18s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 0s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 25s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 58s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 36s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 21s | | the patch passed |
| +1 :green_heart: | shadedclient | 22m 38s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 243m 51s | | hadoop-hdfs in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 56s | | The patch does not
generate ASF License warnings. |
| | | 354m 22s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4834/3/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4834 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 366deeb46b5e 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 75b7e3155c86023126b17644b4924cd4863ed206 |
| Default Java | Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4834/3/testReport/ |
| Max. process+thread count | 3311 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4834/3/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.
[jira] [Commented] (HDFS-16802) Print options when accessing ClientProtocol#rename2()
[
https://issues.apache.org/jira/browse/HDFS-16802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17617287#comment-17617287
]
ASF GitHub Bot commented on HDFS-16802:
---
hadoop-yetus commented on PR #5013:
URL: https://github.com/apache/hadoop/pull/5013#issuecomment-1278124975
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 2m 21s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 41m 56s | | trunk passed |
| +1 :green_heart: | compile | 1m 42s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 1m 30s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 16s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 40s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 17s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 42s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 47s | | trunk passed |
| +1 :green_heart: | shadedclient | 26m 36s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 25s | | the patch passed |
| +1 :green_heart: | compile | 1m 29s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 1m 29s | | the patch passed |
| +1 :green_heart: | compile | 1m 18s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 1m 18s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 0s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 26s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 58s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 30s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 32s | | the patch passed |
| +1 :green_heart: | shadedclient | 25m 36s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 337m 28s | | hadoop-hdfs in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 0s | | The patch does not
generate ASF License warnings. |
| | | 458m 7s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5013/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5013 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 77f3c1f97e20 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 327a6bf0b2b9ebccfa63ae0fea35445015194f2a |
| Default Java | Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5013/2/testReport/ |
| Max. process+thread count | 1936 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5013/2/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.
[jira] [Commented] (HDFS-16795) Use secure XML parser utils in hdfs classes
[ https://issues.apache.org/jira/browse/HDFS-16795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17617216#comment-17617216 ] ASF GitHub Bot commented on HDFS-16795: --- steveloughran opened a new pull request, #5016: URL: https://github.com/apache/hadoop/pull/5016 Move construction of XML parsers in HDFS modules to using the locked-down parser factory of HADOOP-18469. Contributed by P J Fanning Change-Id: I9e21228eeebff699ebd22f46a99722cb9efb0cf4 ### Description of PR ### How was this patch tested? ### For code changes: - [ ] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')? - [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files? > Use secure XML parser utils in hdfs classes > --- > > Key: HDFS-16795 > URL: https://issues.apache.org/jira/browse/HDFS-16795 > Project: Hadoop HDFS > Issue Type: Improvement >Affects Versions: 3.3.4 >Reporter: PJ Fanning >Priority: Major > Labels: pull-request-available > > Uptakes HADOOP-18469 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-16766) hdfs ec command loads (administrator provided) erasure code policy files without disabling xml entity expansion
[
https://issues.apache.org/jira/browse/HDFS-16766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated HDFS-16766:
--
Description:
XML External Entity (XXE) attacks can occur when an XML parser supports XML
entities while processing XML received from an untrusted source. The attack
resides in XML input containing references to an external entity an is parsed
by the weakly configured javax.xml.parsers.DocumentBuilder XML parser.
https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/util/ECPolicyLoader.java#L93
h3. sonatype-2022-5732
If anyone is landing on this page following the sonatype-2022-5732 alert
# the xml expansion only happens on the command line of the {{hdfs ec)) command
https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HDFSErasureCoding.html#Administrative_commands
# the outcome of entity expansion will be the command failing/running out of
memory
# if a cluster admin is loading erasure policies from untrusted sources, there
are fundamental process issues to worry about beyond xml references
was:
XML External Entity (XXE) attacks can occur when an XML parser supports XML
entities while processing XML received from an untrusted source. The attack
resides in XML input containing references to an external entity an is parsed
by the weakly configured javax.xml.parsers.DocumentBuilder XML parser.
https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/util/ECPolicyLoader.java#L93
h3. sonatype-2022-5732
If anyone is landing on this page following the sonatype-2022-5732 alert
# the xml expansion only happens on the command line of the {{hdfs ec)) command
https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HDFSErasureCoding.html#Administrative_commands
# the outcome of entity expansion will be the command failing/running out of
memory
# if you cluster admin is loading erasure policies from untrusted sources,
there are fundamental process issues to worry about beyond xml references
> hdfs ec command loads (administrator provided) erasure code policy files
> without disabling xml entity expansion
> ---
>
> Key: HDFS-16766
> URL: https://issues.apache.org/jira/browse/HDFS-16766
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: security
>Affects Versions: 3.3.4
>Reporter: Jing
>Assignee: Ashutosh Gupta
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0, 3.3.5, 3.2.5
>
>
> XML External Entity (XXE) attacks can occur when an XML parser supports XML
> entities while processing XML received from an untrusted source. The attack
> resides in XML input containing references to an external entity an is parsed
> by the weakly configured javax.xml.parsers.DocumentBuilder XML parser.
>
> https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/util/ECPolicyLoader.java#L93
> h3. sonatype-2022-5732
> If anyone is landing on this page following the sonatype-2022-5732 alert
> # the xml expansion only happens on the command line of the {{hdfs ec))
> command
> https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HDFSErasureCoding.html#Administrative_commands
> # the outcome of entity expansion will be the command failing/running out of
> memory
> # if a cluster admin is loading erasure policies from untrusted sources,
> there are fundamental process issues to worry about beyond xml references
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-16766) hdfs ec command loads (administrator provided) erasure code policy files without disabling xml entity expansion
[
https://issues.apache.org/jira/browse/HDFS-16766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated HDFS-16766:
--
Summary: hdfs ec command loads (administrator provided) erasure code policy
files without disabling xml entity expansion (was: XML External Entity (XXE)
attacks can occur while processing XML received from an untrusted source)
> hdfs ec command loads (administrator provided) erasure code policy files
> without disabling xml entity expansion
> ---
>
> Key: HDFS-16766
> URL: https://issues.apache.org/jira/browse/HDFS-16766
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: security
>Affects Versions: 3.3.4
>Reporter: Jing
>Assignee: Ashutosh Gupta
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0, 3.3.5, 3.2.5
>
>
> XML External Entity (XXE) attacks can occur when an XML parser supports XML
> entities while processing XML received from an untrusted source. The attack
> resides in XML input containing references to an external entity an is parsed
> by the weakly configured javax.xml.parsers.DocumentBuilder XML parser.
>
> https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/util/ECPolicyLoader.java#L93
> h3. sonatype-2022-5732
> If anyone is landing on this page following the sonatype-2022-5732 alert
> # the xml expansion only happens on the command line of the {{hdfs ec))
> command
> https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HDFSErasureCoding.html#Administrative_commands
> # the outcome of entity expansion will be the command failing/running out of
> memory
> # if you cluster admin is loading erasure policies from untrusted sources,
> there are fundamental process issues to worry about beyond xml references
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-16766) XML External Entity (XXE) attacks can occur while processing XML received from an untrusted source
[
https://issues.apache.org/jira/browse/HDFS-16766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated HDFS-16766:
--
Description:
XML External Entity (XXE) attacks can occur when an XML parser supports XML
entities while processing XML received from an untrusted source. The attack
resides in XML input containing references to an external entity an is parsed
by the weakly configured javax.xml.parsers.DocumentBuilder XML parser.
https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/util/ECPolicyLoader.java#L93
h3. sonatype-2022-5732
If anyone is landing on this page following the sonatype-2022-5732 alert
# the xml expansion only happens on the command line of the {{hdfs ec)) command
https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HDFSErasureCoding.html#Administrative_commands
# the outcome of entity expansion will be the command failing/running out of
memory
# if you cluster admin is loading erasure policies from untrusted sources,
there are fundamental process issues to worry about beyond xml references
was:
XML External Entity (XXE) attacks can occur when an XML parser supports XML
entities while processing XML received from an untrusted source. The attack
resides in XML input containing references to an external entity an is parsed
by the weakly configured javax.xml.parsers.DocumentBuilder XML parser.
https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/util/ECPolicyLoader.java#L93
> XML External Entity (XXE) attacks can occur while processing XML received
> from an untrusted source
> --
>
> Key: HDFS-16766
> URL: https://issues.apache.org/jira/browse/HDFS-16766
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: security
>Affects Versions: 3.3.4
>Reporter: Jing
>Assignee: Ashutosh Gupta
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0, 3.3.5, 3.2.5
>
>
> XML External Entity (XXE) attacks can occur when an XML parser supports XML
> entities while processing XML received from an untrusted source. The attack
> resides in XML input containing references to an external entity an is parsed
> by the weakly configured javax.xml.parsers.DocumentBuilder XML parser.
>
> https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/util/ECPolicyLoader.java#L93
> h3. sonatype-2022-5732
> If anyone is landing on this page following the sonatype-2022-5732 alert
> # the xml expansion only happens on the command line of the {{hdfs ec))
> command
> https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HDFSErasureCoding.html#Administrative_commands
> # the outcome of entity expansion will be the command failing/running out of
> memory
> # if you cluster admin is loading erasure policies from untrusted sources,
> there are fundamental process issues to worry about beyond xml references
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16802) Print options when accessing ClientProtocol#rename2()
[
https://issues.apache.org/jira/browse/HDFS-16802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17617103#comment-17617103
]
ASF GitHub Bot commented on HDFS-16802:
---
hadoop-yetus commented on PR #5013:
URL: https://github.com/apache/hadoop/pull/5013#issuecomment-1277718334
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 44s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 38m 50s | | trunk passed |
| +1 :green_heart: | compile | 1m 42s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 1m 33s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 14s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 38s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 24s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 45s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 32s | | trunk passed |
| +1 :green_heart: | shadedclient | 22m 56s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 23s | | the patch passed |
| +1 :green_heart: | compile | 1m 24s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 1m 24s | | the patch passed |
| +1 :green_heart: | compile | 1m 16s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 1m 16s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 56s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 26s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 57s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 28s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 21s | | the patch passed |
| +1 :green_heart: | shadedclient | 22m 49s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 240m 59s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5013/1/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 1m 2s | | The patch does not
generate ASF License warnings. |
| | | 350m 40s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.hdfs.TestRollingUpgrade |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5013/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5013 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 78907c5d79cb 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 829c9aa5af5bf09b4016d3a6d03d15a7cd311428 |
| Default Java | Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5013/1/testReport/ |
| Max. process+thread count | 2954 (vs. ulimit of 5500)
[jira] [Commented] (HDFS-16793) ObserverNameNode fails to select streaming inputStream with a timeout exception
[
https://issues.apache.org/jira/browse/HDFS-16793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17617100#comment-17617100
]
ASF GitHub Bot commented on HDFS-16793:
---
hadoop-yetus commented on PR #4971:
URL: https://github.com/apache/hadoop/pull/4971#issuecomment-1277713878
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 35s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 42m 42s | | trunk passed |
| +1 :green_heart: | compile | 1m 57s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 1m 33s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 20s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 48s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 27s | | trunk passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 38s | | trunk passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 58s | | trunk passed |
| +1 :green_heart: | shadedclient | 28m 47s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 39s | | the patch passed |
| +1 :green_heart: | compile | 1m 41s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 1m 41s | | the patch passed |
| +1 :green_heart: | compile | 1m 28s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 1m 28s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 6s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 38s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 2s | | the patch passed with JDK
Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 32s | | the patch passed with JDK
Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 4m 3s | | the patch passed |
| +1 :green_heart: | shadedclient | 27m 45s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 364m 36s | | hadoop-hdfs in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 10s | | The patch does not
generate ASF License warnings. |
| | | 491m 20s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4971/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4971 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 84ac63d4e941 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 5f1de83d50d8245b92ebdbeb2dd20a5d462286c1 |
| Default Java | Private Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.16+8-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_342-8u342-b07-0ubuntu1~20.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4971/2/testReport/ |
| Max. process+thread count | 2088 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4971/2/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.
[jira] [Commented] (HDFS-16802) Print options when accessing ClientProtocol#rename2()
[
https://issues.apache.org/jira/browse/HDFS-16802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616909#comment-17616909
]
ASF GitHub Bot commented on HDFS-16802:
---
jianghuazhu commented on code in PR #5013:
URL: https://github.com/apache/hadoop/pull/5013#discussion_r994397572
##
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirRenameOp.java:
##
@@ -249,7 +249,7 @@ static RenameResult renameToInt(
String dst = dstArg;
if (NameNode.stateChangeLog.isDebugEnabled()) {
NameNode.stateChangeLog.debug("DIR* NameSystem.renameTo: with options -"
+
- " " + src + " to " + dst);
+ " " + src + " to " + dst + ", options=" + Arrays.toString(options));
Review Comment:
Thanks @ZanderXu , I'll update later.
> Print options when accessing ClientProtocol#rename2()
> -
>
> Key: HDFS-16802
> URL: https://issues.apache.org/jira/browse/HDFS-16802
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
>Affects Versions: 3.3.4
>Reporter: JiangHua Zhu
>Assignee: JiangHua Zhu
>Priority: Minor
> Labels: pull-request-available
>
> When accessing ClientProtocol#rename2(), the carried options cannot be seen
> in the log. Here is some log information:
> {code:java}
> 2022-10-13 10:21:10,727 [Listener at localhost/59732] DEBUG hdfs.StateChange
> (FSDirRenameOp.java:renameToInt(255)) - DIR* NameSystem.renameTo: with
> options - /testNamenodeRetryCache/testRename2/src to
> /testNamenodeRetryCache/testRename2/target
> {code}
> We should improve this, maybe printing options would be better.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16802) Print options when accessing ClientProtocol#rename2()
[
https://issues.apache.org/jira/browse/HDFS-16802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616888#comment-17616888
]
ASF GitHub Bot commented on HDFS-16802:
---
ZanderXu commented on code in PR #5013:
URL: https://github.com/apache/hadoop/pull/5013#discussion_r994359343
##
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirRenameOp.java:
##
@@ -249,7 +249,7 @@ static RenameResult renameToInt(
String dst = dstArg;
if (NameNode.stateChangeLog.isDebugEnabled()) {
NameNode.stateChangeLog.debug("DIR* NameSystem.renameTo: with options -"
+
- " " + src + " to " + dst);
+ " " + src + " to " + dst + ", options=" + Arrays.toString(options));
Review Comment:
@jianghuazhu Can use `{}` formatted this log?
> Print options when accessing ClientProtocol#rename2()
> -
>
> Key: HDFS-16802
> URL: https://issues.apache.org/jira/browse/HDFS-16802
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
>Affects Versions: 3.3.4
>Reporter: JiangHua Zhu
>Assignee: JiangHua Zhu
>Priority: Minor
> Labels: pull-request-available
>
> When accessing ClientProtocol#rename2(), the carried options cannot be seen
> in the log. Here is some log information:
> {code:java}
> 2022-10-13 10:21:10,727 [Listener at localhost/59732] DEBUG hdfs.StateChange
> (FSDirRenameOp.java:renameToInt(255)) - DIR* NameSystem.renameTo: with
> options - /testNamenodeRetryCache/testRename2/src to
> /testNamenodeRetryCache/testRename2/target
> {code}
> We should improve this, maybe printing options would be better.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-16802) Print options when accessing ClientProtocol#rename2()
[
https://issues.apache.org/jira/browse/HDFS-16802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HDFS-16802:
--
Labels: pull-request-available (was: )
> Print options when accessing ClientProtocol#rename2()
> -
>
> Key: HDFS-16802
> URL: https://issues.apache.org/jira/browse/HDFS-16802
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
>Affects Versions: 3.3.4
>Reporter: JiangHua Zhu
>Assignee: JiangHua Zhu
>Priority: Minor
> Labels: pull-request-available
>
> When accessing ClientProtocol#rename2(), the carried options cannot be seen
> in the log. Here is some log information:
> {code:java}
> 2022-10-13 10:21:10,727 [Listener at localhost/59732] DEBUG hdfs.StateChange
> (FSDirRenameOp.java:renameToInt(255)) - DIR* NameSystem.renameTo: with
> options - /testNamenodeRetryCache/testRename2/src to
> /testNamenodeRetryCache/testRename2/target
> {code}
> We should improve this, maybe printing options would be better.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16802) Print options when accessing ClientProtocol#rename2()
[
https://issues.apache.org/jira/browse/HDFS-16802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616880#comment-17616880
]
ASF GitHub Bot commented on HDFS-16802:
---
jianghuazhu opened a new pull request, #5013:
URL: https://github.com/apache/hadoop/pull/5013
### Description of PR
When accessing ClientProtocol#rename2(), we cannot see options in the log.
The purpose of this PR is to improve it as much as possible.
Details: HDFS-16802
### How was this patch tested?
Not too stressful for testing.
> Print options when accessing ClientProtocol#rename2()
> -
>
> Key: HDFS-16802
> URL: https://issues.apache.org/jira/browse/HDFS-16802
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
>Affects Versions: 3.3.4
>Reporter: JiangHua Zhu
>Assignee: JiangHua Zhu
>Priority: Minor
>
> When accessing ClientProtocol#rename2(), the carried options cannot be seen
> in the log. Here is some log information:
> {code:java}
> 2022-10-13 10:21:10,727 [Listener at localhost/59732] DEBUG hdfs.StateChange
> (FSDirRenameOp.java:renameToInt(255)) - DIR* NameSystem.renameTo: with
> options - /testNamenodeRetryCache/testRename2/src to
> /testNamenodeRetryCache/testRename2/target
> {code}
> We should improve this, maybe printing options would be better.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16802) Print options when accessing ClientProtocol#rename2()
[
https://issues.apache.org/jira/browse/HDFS-16802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616873#comment-17616873
]
JiangHua Zhu commented on HDFS-16802:
-
New log format:
{code:java}
2022-10-13 12:11:38,813 [Listener at localhost/58086] DEBUG hdfs.StateChange
(FSDirRenameOp.java:renameToInt(256)) - DIR* NameSystem.renameTo: with options
- /testNamenodeRetryCache/testRename2/src to /testNamenodeRetryCache
/testRename2/target, options=[NONE]
{code}
> Print options when accessing ClientProtocol#rename2()
> -
>
> Key: HDFS-16802
> URL: https://issues.apache.org/jira/browse/HDFS-16802
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
>Affects Versions: 3.3.4
>Reporter: JiangHua Zhu
>Assignee: JiangHua Zhu
>Priority: Minor
>
> When accessing ClientProtocol#rename2(), the carried options cannot be seen
> in the log. Here is some log information:
> {code:java}
> 2022-10-13 10:21:10,727 [Listener at localhost/59732] DEBUG hdfs.StateChange
> (FSDirRenameOp.java:renameToInt(255)) - DIR* NameSystem.renameTo: with
> options - /testNamenodeRetryCache/testRename2/src to
> /testNamenodeRetryCache/testRename2/target
> {code}
> We should improve this, maybe printing options would be better.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Assigned] (HDFS-16802) Print options when accessing ClientProtocol#rename2()
[
https://issues.apache.org/jira/browse/HDFS-16802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
JiangHua Zhu reassigned HDFS-16802:
---
Assignee: JiangHua Zhu
> Print options when accessing ClientProtocol#rename2()
> -
>
> Key: HDFS-16802
> URL: https://issues.apache.org/jira/browse/HDFS-16802
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
>Affects Versions: 3.3.4
>Reporter: JiangHua Zhu
>Assignee: JiangHua Zhu
>Priority: Minor
>
> When accessing ClientProtocol#rename2(), the carried options cannot be seen
> in the log. Here is some log information:
> {code:java}
> 2022-10-13 10:21:10,727 [Listener at localhost/59732] DEBUG hdfs.StateChange
> (FSDirRenameOp.java:renameToInt(255)) - DIR* NameSystem.renameTo: with
> options - /testNamenodeRetryCache/testRename2/src to
> /testNamenodeRetryCache/testRename2/target
> {code}
> We should improve this, maybe printing options would be better.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-16802) Print options when accessing ClientProtocol#rename2()
JiangHua Zhu created HDFS-16802:
---
Summary: Print options when accessing ClientProtocol#rename2()
Key: HDFS-16802
URL: https://issues.apache.org/jira/browse/HDFS-16802
Project: Hadoop HDFS
Issue Type: Improvement
Components: namenode
Affects Versions: 3.3.4
Reporter: JiangHua Zhu
When accessing ClientProtocol#rename2(), the carried options cannot be seen in
the log. Here is some log information:
{code:java}
2022-10-13 10:21:10,727 [Listener at localhost/59732] DEBUG hdfs.StateChange
(FSDirRenameOp.java:renameToInt(255)) - DIR* NameSystem.renameTo: with options
- /testNamenodeRetryCache/testRename2/src to
/testNamenodeRetryCache/testRename2/target
{code}
We should improve this, maybe printing options would be better.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13507) RBF: Remove update functionality from routeradmin's add cmd
[
https://issues.apache.org/jira/browse/HDFS-13507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616823#comment-17616823
]
ASF GitHub Bot commented on HDFS-13507:
---
ZanderXu commented on code in PR #4990:
URL: https://github.com/apache/hadoop/pull/4990#discussion_r994235078
##
hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/tools/federation/RouterAdmin.java:
##
@@ -147,13 +147,13 @@ private String getUsage(String cmd) {
return usage.toString();
}
if (cmd.equals("-add")) {
- return "\t[-add
"
+ return "\t[-add
"
+ "[-readonly] [-faulttolerant] "
+ "[-order HASH|LOCAL|RANDOM|HASH_ALL|SPACE] "
+ "-owner -group -mode ]";
} else if (cmd.equals("-update")) {
return "\t[-update "
- + " [ ] "
+ + " [ ] "
Review Comment:
Got. I will add a detailed description.
> RBF: Remove update functionality from routeradmin's add cmd
> ---
>
> Key: HDFS-13507
> URL: https://issues.apache.org/jira/browse/HDFS-13507
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Wei Yan
>Assignee: Gang Li
>Priority: Minor
> Labels: incompatible, pull-request-available
> Attachments: HDFS-13507-HDFS-13891.003.patch,
> HDFS-13507-HDFS-13891.004.patch, HDFS-13507.000.patch, HDFS-13507.001.patch,
> HDFS-13507.002.patch, HDFS-13507.003.patch
>
>
> Follow up the discussion in HDFS-13326. We should remove the "update"
> functionality from routeradmin's add cmd, to make it consistent with RPC
> calls.
> Note that: this is an incompatible change.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13507) RBF: Remove update functionality from routeradmin's add cmd
[
https://issues.apache.org/jira/browse/HDFS-13507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17616821#comment-17616821
]
ASF GitHub Bot commented on HDFS-13507:
---
ayushtkn commented on code in PR #4990:
URL: https://github.com/apache/hadoop/pull/4990#discussion_r994232772
##
hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/tools/federation/RouterAdmin.java:
##
@@ -147,13 +147,13 @@ private String getUsage(String cmd) {
return usage.toString();
}
if (cmd.equals("-add")) {
- return "\t[-add
"
+ return "\t[-add
"
+ "[-readonly] [-faulttolerant] "
+ "[-order HASH|LOCAL|RANDOM|HASH_ALL|SPACE] "
+ "-owner -group -mode ]";
} else if (cmd.equals("-update")) {
return "\t[-update "
- + " [ ] "
+ + " [ ] "
Review Comment:
Cool, makes sense, will go through that once more. From the description of
the command it appeared that it doesn’t
> RBF: Remove update functionality from routeradmin's add cmd
> ---
>
> Key: HDFS-13507
> URL: https://issues.apache.org/jira/browse/HDFS-13507
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Wei Yan
>Assignee: Gang Li
>Priority: Minor
> Labels: incompatible, pull-request-available
> Attachments: HDFS-13507-HDFS-13891.003.patch,
> HDFS-13507-HDFS-13891.004.patch, HDFS-13507.000.patch, HDFS-13507.001.patch,
> HDFS-13507.002.patch, HDFS-13507.003.patch
>
>
> Follow up the discussion in HDFS-13326. We should remove the "update"
> functionality from routeradmin's add cmd, to make it consistent with RPC
> calls.
> Note that: this is an incompatible change.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
