[jira] [Commented] (HDFS-16207) Remove NN logs stack trace for non-existent xattr query
[ https://issues.apache.org/jira/browse/HDFS-16207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17412705#comment-17412705 ] Hadoop QA commented on HDFS-16207: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Logfile || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 9m 41s{color} | {color:blue}{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || || | {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m 0s{color} | {color:green}{color} | {color:green} No case conflicting files found. {color} | | {color:green}+1{color} | {color:green} {color} | {color:green} 0m 0s{color} | {color:green}test4tests{color} | {color:green} The patch appears to include 2 new or modified test files. {color} | || || || || {color:brown} branch-2.10 Compile Tests {color} || || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m 28s{color} | {color:blue}{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 12m 25s{color} | {color:green}{color} | {color:green} branch-2.10 passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 0s{color} | {color:green}{color} | {color:green} branch-2.10 passed with JDK Azul Systems, Inc.-1.7.0_262-b10 {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 41s{color} | {color:green}{color} | {color:green} branch-2.10 passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~16.04.1-b10 {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 50s{color} | {color:green}{color} | {color:green} branch-2.10 passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 44s{color} | {color:green}{color} | {color:green} branch-2.10 passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 50s{color} | {color:green}{color} | {color:green} branch-2.10 passed with JDK Azul Systems, Inc.-1.7.0_262-b10 {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 15s{color} | {color:green}{color} | {color:green} branch-2.10 passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~16.04.1-b10 {color} | | {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 10m 1s{color} | {color:blue}{color} | {color:blue} Both FindBugs and SpotBugs are enabled, using SpotBugs. {color} | | {color:red}-1{color} | {color:red} spotbugs {color} | {color:red} 2m 24s{color} | {color:red}https://ci-hadoop.apache.org/job/PreCommit-HDFS-Build/713/artifact/out/branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html{color} | {color:red} hadoop-hdfs-project/hadoop-hdfs in branch-2.10 has 1 extant spotbugs warnings. {color} | || || || || {color:brown} Patch Compile Tests {color} || || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 21s{color} | {color:blue}{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 27s{color} | {color:green}{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 56s{color} | {color:green}{color} | {color:green} the patch passed with JDK Azul Systems, Inc.-1.7.0_262-b10 {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 56s{color} | {color:green}{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 39s{color} | {color:green}{color} | {color:green} the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~16.04.1-b10 {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 39s{color} | {color:green}{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 44s{color} | {color:green}{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 31s{color} | {color:green}{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green}{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 40s{color} | {color:green}{color} | {color:green} the patch passed with JDK Azul Systems, Inc.-1.7.0_262-b10 {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 9s{color} | {color:green}{color} | {color:green} the patch passed with JDK Private
[jira] [Commented] (HDFS-16207) Remove NN logs stack trace for non-existent xattr query
[ https://issues.apache.org/jira/browse/HDFS-16207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17412574#comment-17412574 ] Ahmed Hussein commented on HDFS-16207: -- Thanks [~cnauroth]! I will upload a separate patch for branch-2.10 shortly. Let me know if you prefer that I submit a separate Pull-request for that rather than attaching a patch to the jira. > Remove NN logs stack trace for non-existent xattr query > --- > > Key: HDFS-16207 > URL: https://issues.apache.org/jira/browse/HDFS-16207 > Project: Hadoop HDFS > Issue Type: Bug > Components: namenode >Affects Versions: 3.4.0, 2.10.2, 3.3.2, 3.2.4 >Reporter: Ahmed Hussein >Assignee: Ahmed Hussein >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > The NN logs a full stack trace every time a getXAttrs is called for a > non-existent xattr. The logging has zero value add. The increased logging > load may harm performance. Something is now probing for xattrs resulting in > many lines of: > {code:bash} > 2021-09-02 13:48:03,340 [IPC Server handler 5 on default port 59951] INFO > ipc.Server (Server.java:logException(3149)) - IPC Server handler 5 on default > port 59951, call Call#17 Retry#0 > org.apache.hadoop.hdfs.protocol.ClientProtocol.getXAttrs from 127.0.0.1:59961 > java.io.IOException: At least one of the attributes provided was not found. > at > org.apache.hadoop.hdfs.server.namenode.FSDirXAttrOp.getXAttrs(FSDirXAttrOp.java:134) > at > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getXAttrs(FSNamesystem.java:8472) > at > org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getXAttrs(NameNodeRpcServer.java:2317) > at > org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getXAttrs(ClientNamenodeProtocolServerSideTranslatorPB.java:1745) > at > org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) > at > org.apache.hadoop.ipc.ProtobufRpcEngine2$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine2.java:604) > at > org.apache.hadoop.ipc.ProtobufRpcEngine2$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine2.java:572) > at > org.apache.hadoop.ipc.ProtobufRpcEngine2$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine2.java:556) > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1093) > at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:1155) > at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:1083) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1900) > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:3088) > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16207) Remove NN logs stack trace for non-existent xattr query
[ https://issues.apache.org/jira/browse/HDFS-16207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17412370#comment-17412370 ] Chris Nauroth commented on HDFS-16207: -- [~ahussein], thank you. I merged this to trunk, branch-3.3 and branch-3.2, resolving some trivial merge conflicts along the way. From the Affects Version/s, it sounds like you also wanted to merge to branch-2.10. Trying to merge to that branch fails with this compilation error: {code} ERROR] /home/cnauroth/hadoop/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java:[524,9] cannot find symbol symbol: class XAttrNotFoundException location: class org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer {code} I stopped there. If you still want to merge to branch-2.10, please provide a separate patch compatible with the branch. > Remove NN logs stack trace for non-existent xattr query > --- > > Key: HDFS-16207 > URL: https://issues.apache.org/jira/browse/HDFS-16207 > Project: Hadoop HDFS > Issue Type: Bug > Components: namenode >Affects Versions: 3.4.0, 2.10.2, 3.3.2, 3.2.4 >Reporter: Ahmed Hussein >Assignee: Ahmed Hussein >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > The NN logs a full stack trace every time a getXAttrs is called for a > non-existent xattr. The logging has zero value add. The increased logging > load may harm performance. Something is now probing for xattrs resulting in > many lines of: > {code:bash} > 2021-09-02 13:48:03,340 [IPC Server handler 5 on default port 59951] INFO > ipc.Server (Server.java:logException(3149)) - IPC Server handler 5 on default > port 59951, call Call#17 Retry#0 > org.apache.hadoop.hdfs.protocol.ClientProtocol.getXAttrs from 127.0.0.1:59961 > java.io.IOException: At least one of the attributes provided was not found. > at > org.apache.hadoop.hdfs.server.namenode.FSDirXAttrOp.getXAttrs(FSDirXAttrOp.java:134) > at > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getXAttrs(FSNamesystem.java:8472) > at > org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getXAttrs(NameNodeRpcServer.java:2317) > at > org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getXAttrs(ClientNamenodeProtocolServerSideTranslatorPB.java:1745) > at > org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) > at > org.apache.hadoop.ipc.ProtobufRpcEngine2$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine2.java:604) > at > org.apache.hadoop.ipc.ProtobufRpcEngine2$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine2.java:572) > at > org.apache.hadoop.ipc.ProtobufRpcEngine2$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine2.java:556) > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1093) > at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:1155) > at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:1083) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1900) > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:3088) > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16207) Remove NN logs stack trace for non-existent xattr query
[ https://issues.apache.org/jira/browse/HDFS-16207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17412120#comment-17412120 ] Ahmed Hussein commented on HDFS-16207: -- I am fine with merging the changes. Thanks [~cnauroth] for the review! > Remove NN logs stack trace for non-existent xattr query > --- > > Key: HDFS-16207 > URL: https://issues.apache.org/jira/browse/HDFS-16207 > Project: Hadoop HDFS > Issue Type: Bug > Components: namenode >Affects Versions: 3.4.0, 2.10.2, 3.3.2, 3.2.4 >Reporter: Ahmed Hussein >Assignee: Ahmed Hussein >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > The NN logs a full stack trace every time a getXAttrs is called for a > non-existent xattr. The logging has zero value add. The increased logging > load may harm performance. Something is now probing for xattrs resulting in > many lines of: > {code:bash} > 2021-09-02 13:48:03,340 [IPC Server handler 5 on default port 59951] INFO > ipc.Server (Server.java:logException(3149)) - IPC Server handler 5 on default > port 59951, call Call#17 Retry#0 > org.apache.hadoop.hdfs.protocol.ClientProtocol.getXAttrs from 127.0.0.1:59961 > java.io.IOException: At least one of the attributes provided was not found. > at > org.apache.hadoop.hdfs.server.namenode.FSDirXAttrOp.getXAttrs(FSDirXAttrOp.java:134) > at > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getXAttrs(FSNamesystem.java:8472) > at > org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getXAttrs(NameNodeRpcServer.java:2317) > at > org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getXAttrs(ClientNamenodeProtocolServerSideTranslatorPB.java:1745) > at > org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) > at > org.apache.hadoop.ipc.ProtobufRpcEngine2$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine2.java:604) > at > org.apache.hadoop.ipc.ProtobufRpcEngine2$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine2.java:572) > at > org.apache.hadoop.ipc.ProtobufRpcEngine2$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine2.java:556) > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1093) > at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:1155) > at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:1083) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1900) > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:3088) > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16207) Remove NN logs stack trace for non-existent xattr query
[ https://issues.apache.org/jira/browse/HDFS-16207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17412086#comment-17412086 ] Chris Nauroth commented on HDFS-16207: -- +1 Thank you for the contribution, Ahmed. I see you asked Kihwal to review, so I'll wait a day before merging in case you really wanted Kihwal's opinion specifically. > Remove NN logs stack trace for non-existent xattr query > --- > > Key: HDFS-16207 > URL: https://issues.apache.org/jira/browse/HDFS-16207 > Project: Hadoop HDFS > Issue Type: Bug > Components: namenode >Affects Versions: 3.4.0, 2.10.2, 3.3.2, 3.2.4 >Reporter: Ahmed Hussein >Assignee: Ahmed Hussein >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > The NN logs a full stack trace every time a getXAttrs is called for a > non-existent xattr. The logging has zero value add. The increased logging > load may harm performance. Something is now probing for xattrs resulting in > many lines of: > {code:bash} > 2021-09-02 13:48:03,340 [IPC Server handler 5 on default port 59951] INFO > ipc.Server (Server.java:logException(3149)) - IPC Server handler 5 on default > port 59951, call Call#17 Retry#0 > org.apache.hadoop.hdfs.protocol.ClientProtocol.getXAttrs from 127.0.0.1:59961 > java.io.IOException: At least one of the attributes provided was not found. > at > org.apache.hadoop.hdfs.server.namenode.FSDirXAttrOp.getXAttrs(FSDirXAttrOp.java:134) > at > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getXAttrs(FSNamesystem.java:8472) > at > org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getXAttrs(NameNodeRpcServer.java:2317) > at > org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getXAttrs(ClientNamenodeProtocolServerSideTranslatorPB.java:1745) > at > org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) > at > org.apache.hadoop.ipc.ProtobufRpcEngine2$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine2.java:604) > at > org.apache.hadoop.ipc.ProtobufRpcEngine2$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine2.java:572) > at > org.apache.hadoop.ipc.ProtobufRpcEngine2$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine2.java:556) > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1093) > at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:1155) > at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:1083) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1900) > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:3088) > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16207) Remove NN logs stack trace for non-existent xattr query
[ https://issues.apache.org/jira/browse/HDFS-16207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17411293#comment-17411293 ] Ahmed Hussein commented on HDFS-16207: -- I submitted a PR to suppress the stack by: # created a new IOEXception class {{XAttrNotFoundException}} # Add {{XAttrNotFoundException}} to the list of "{{Tersed}}" Exceptions in the {{clientRpcServer}}. [~kihwal], Can you please take a look at the PR? > Remove NN logs stack trace for non-existent xattr query > --- > > Key: HDFS-16207 > URL: https://issues.apache.org/jira/browse/HDFS-16207 > Project: Hadoop HDFS > Issue Type: Bug > Components: namenode >Affects Versions: 3.4.0, 2.10.2, 3.3.2, 3.2.4 >Reporter: Ahmed Hussein >Assignee: Ahmed Hussein >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > The NN logs a full stack trace every time a getXAttrs is called for a > non-existent xattr. The logging has zero value add. The increased logging > load may harm performance. Something is now probing for xattrs resulting in > many lines of: > {code:bash} > 2021-09-02 13:48:03,340 [IPC Server handler 5 on default port 59951] INFO > ipc.Server (Server.java:logException(3149)) - IPC Server handler 5 on default > port 59951, call Call#17 Retry#0 > org.apache.hadoop.hdfs.protocol.ClientProtocol.getXAttrs from 127.0.0.1:59961 > java.io.IOException: At least one of the attributes provided was not found. > at > org.apache.hadoop.hdfs.server.namenode.FSDirXAttrOp.getXAttrs(FSDirXAttrOp.java:134) > at > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getXAttrs(FSNamesystem.java:8472) > at > org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getXAttrs(NameNodeRpcServer.java:2317) > at > org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getXAttrs(ClientNamenodeProtocolServerSideTranslatorPB.java:1745) > at > org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) > at > org.apache.hadoop.ipc.ProtobufRpcEngine2$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine2.java:604) > at > org.apache.hadoop.ipc.ProtobufRpcEngine2$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine2.java:572) > at > org.apache.hadoop.ipc.ProtobufRpcEngine2$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine2.java:556) > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1093) > at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:1155) > at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:1083) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1900) > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:3088) > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org