[jira] [Commented] (HADOOP-16453) Update how exceptions are handled in NetUtils.java
[
https://issues.apache.org/jira/browse/HADOOP-16453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16904791#comment-16904791
]
Lisheng Sun commented on HADOOP-16453:
--
[~elgoiri] Should we commit this patch to trunk? Thank you.
> Update how exceptions are handled in NetUtils.java
> --
>
> Key: HADOOP-16453
> URL: https://issues.apache.org/jira/browse/HADOOP-16453
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Lisheng Sun
>Assignee: Lisheng Sun
>Priority: Minor
> Attachments: HADOOP-16453.001.patch, HADOOP-16453.002.patch,
> HADOOP-16453.003.patch, HADOOP-16453.004.patch, HADOOP-16453.005.patch
>
>
> When there is no String Constructor for the exception, we Log a Trace
> Message. Given that log and throw is not a very good approach I think the
> right thing would be to just not log it at all as HADOOP-16431.
> {code:java}
> private static T wrapWithMessage(
> T exception, String msg) throws T {
> Class clazz = exception.getClass();
> try {
> Constructor ctor =
> clazz.getConstructor(String.class);
> Throwable t = ctor.newInstance(msg);
> return (T)(t.initCause(exception));
> } catch (Throwable e) {
> LOG.trace("Unable to wrap exception of type {}: it has no (String) "
> + "constructor", clazz, e);
> throw exception;
> }
> }
> {code}
> *exception stack：*
> {code:java}
> 19/07/12 11:23:45 INFO mapreduce.JobSubmitter: Executing with tokens: [Kind:
> HDFS_DELEGATION_TOKEN, Service: ha-hdfs:azorprc-xiaomi, Ident: (token for
> sql_prc: HDFS_DELEGATION_TOKEN owner=sql_prc/hadoop@XIAOMI.HADOOP,
> renewer=yarn_prc, realUser=, issueDate=1562901814007, maxDate=1594437814007,
> sequenceNumber=3349939, masterKeyId=1400)]
> 19/07/12 11:23:46 TRACE net.NetUtils: Unable to wrap exception of type class
> java.nio.channels.ClosedByInterruptException: it has no (String) constructor
> java.lang.NoSuchMethodException:
> java.nio.channels.ClosedByInterruptException.(java.lang.String)
> at java.lang.Class.getConstructor0(Class.java:3082)
> at java.lang.Class.getConstructor(Class.java:1825)
> at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:830)
> at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:806)
> at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1559)
> at org.apache.hadoop.ipc.Client.call(Client.java:1501)
> at org.apache.hadoop.ipc.Client.call(Client.java:1411)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:228)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116)
> at com.sun.proxy.$Proxy9.getFileInfo(Unknown Source)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:949)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.RequestHedgingProxyProvider$RequestHedgingInvocationHandler$1.call(RequestHedgingProxyProvider.java:143)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 WARN ipc.Client: Exception encountered while connecting to
> the server : java.io.InterruptedIOException: Interrupted while waiting for IO
> on channel java.nio.channels.SocketChannel[connected
> local=/10.118.30.48:34324 remote=/10.69.11.137:11200]. 6 millis timeout
> left.
> 19/07/12 11:23:48 INFO conf.Configuration: resource-types.xml not found
> 19/07/12 11:23:48 INFO
[jira] [Commented] (HADOOP-16453) Update how exceptions are handled in NetUtils.java
[
https://issues.apache.org/jira/browse/HADOOP-16453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16903142#comment-16903142
]
Íñigo Goiri commented on HADOOP-16453:
--
Thanks [~leosun08].
+1 from my side.
Anybody else wants to chime in?
I'll give it a couple days.
> Update how exceptions are handled in NetUtils.java
> --
>
> Key: HADOOP-16453
> URL: https://issues.apache.org/jira/browse/HADOOP-16453
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Lisheng Sun
>Assignee: Lisheng Sun
>Priority: Minor
> Attachments: HADOOP-16453.001.patch, HADOOP-16453.002.patch,
> HADOOP-16453.003.patch, HADOOP-16453.004.patch, HADOOP-16453.005.patch
>
>
> When there is no String Constructor for the exception, we Log a Trace
> Message. Given that log and throw is not a very good approach I think the
> right thing would be to just not log it at all as HADOOP-16431.
> {code:java}
> private static T wrapWithMessage(
> T exception, String msg) throws T {
> Class clazz = exception.getClass();
> try {
> Constructor ctor =
> clazz.getConstructor(String.class);
> Throwable t = ctor.newInstance(msg);
> return (T)(t.initCause(exception));
> } catch (Throwable e) {
> LOG.trace("Unable to wrap exception of type {}: it has no (String) "
> + "constructor", clazz, e);
> throw exception;
> }
> }
> {code}
> *exception stack：*
> {code:java}
> 19/07/12 11:23:45 INFO mapreduce.JobSubmitter: Executing with tokens: [Kind:
> HDFS_DELEGATION_TOKEN, Service: ha-hdfs:azorprc-xiaomi, Ident: (token for
> sql_prc: HDFS_DELEGATION_TOKEN owner=sql_prc/hadoop@XIAOMI.HADOOP,
> renewer=yarn_prc, realUser=, issueDate=1562901814007, maxDate=1594437814007,
> sequenceNumber=3349939, masterKeyId=1400)]
> 19/07/12 11:23:46 TRACE net.NetUtils: Unable to wrap exception of type class
> java.nio.channels.ClosedByInterruptException: it has no (String) constructor
> java.lang.NoSuchMethodException:
> java.nio.channels.ClosedByInterruptException.(java.lang.String)
> at java.lang.Class.getConstructor0(Class.java:3082)
> at java.lang.Class.getConstructor(Class.java:1825)
> at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:830)
> at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:806)
> at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1559)
> at org.apache.hadoop.ipc.Client.call(Client.java:1501)
> at org.apache.hadoop.ipc.Client.call(Client.java:1411)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:228)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116)
> at com.sun.proxy.$Proxy9.getFileInfo(Unknown Source)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:949)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.RequestHedgingProxyProvider$RequestHedgingInvocationHandler$1.call(RequestHedgingProxyProvider.java:143)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 WARN ipc.Client: Exception encountered while connecting to
> the server : java.io.InterruptedIOException: Interrupted while waiting for IO
> on channel java.nio.channels.SocketChannel[connected
> local=/10.118.30.48:34324 remote=/10.69.11.137:11200]. 6 millis timeout
> left.
> 19/07/12 11:23:48 INFO conf.Configuration:
[jira] [Commented] (HADOOP-16453) Update how exceptions are handled in NetUtils.java
[
https://issues.apache.org/jira/browse/HADOOP-16453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16902594#comment-16902594
]
Lisheng Sun commented on HADOOP-16453:
--
hi [~elgoiri] I run all UT in HDFS and YARN of my local and verify this patch
don't break anything.
> Update how exceptions are handled in NetUtils.java
> --
>
> Key: HADOOP-16453
> URL: https://issues.apache.org/jira/browse/HADOOP-16453
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Lisheng Sun
>Assignee: Lisheng Sun
>Priority: Minor
> Attachments: HADOOP-16453.001.patch, HADOOP-16453.002.patch,
> HADOOP-16453.003.patch, HADOOP-16453.004.patch, HADOOP-16453.005.patch
>
>
> When there is no String Constructor for the exception, we Log a Trace
> Message. Given that log and throw is not a very good approach I think the
> right thing would be to just not log it at all as HADOOP-16431.
> {code:java}
> private static T wrapWithMessage(
> T exception, String msg) throws T {
> Class clazz = exception.getClass();
> try {
> Constructor ctor =
> clazz.getConstructor(String.class);
> Throwable t = ctor.newInstance(msg);
> return (T)(t.initCause(exception));
> } catch (Throwable e) {
> LOG.trace("Unable to wrap exception of type {}: it has no (String) "
> + "constructor", clazz, e);
> throw exception;
> }
> }
> {code}
> *exception stack：*
> {code:java}
> 19/07/12 11:23:45 INFO mapreduce.JobSubmitter: Executing with tokens: [Kind:
> HDFS_DELEGATION_TOKEN, Service: ha-hdfs:azorprc-xiaomi, Ident: (token for
> sql_prc: HDFS_DELEGATION_TOKEN owner=sql_prc/hadoop@XIAOMI.HADOOP,
> renewer=yarn_prc, realUser=, issueDate=1562901814007, maxDate=1594437814007,
> sequenceNumber=3349939, masterKeyId=1400)]
> 19/07/12 11:23:46 TRACE net.NetUtils: Unable to wrap exception of type class
> java.nio.channels.ClosedByInterruptException: it has no (String) constructor
> java.lang.NoSuchMethodException:
> java.nio.channels.ClosedByInterruptException.(java.lang.String)
> at java.lang.Class.getConstructor0(Class.java:3082)
> at java.lang.Class.getConstructor(Class.java:1825)
> at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:830)
> at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:806)
> at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1559)
> at org.apache.hadoop.ipc.Client.call(Client.java:1501)
> at org.apache.hadoop.ipc.Client.call(Client.java:1411)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:228)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116)
> at com.sun.proxy.$Proxy9.getFileInfo(Unknown Source)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:949)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.RequestHedgingProxyProvider$RequestHedgingInvocationHandler$1.call(RequestHedgingProxyProvider.java:143)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 WARN ipc.Client: Exception encountered while connecting to
> the server : java.io.InterruptedIOException: Interrupted while waiting for IO
> on channel java.nio.channels.SocketChannel[connected
> local=/10.118.30.48:34324 remote=/10.69.11.137:11200]. 6 millis timeout
> left.
> 19/07/12 11:23:48 INFO conf.Configuration:
[jira] [Commented] (HADOOP-16453) Update how exceptions are handled in NetUtils.java
[
https://issues.apache.org/jira/browse/HADOOP-16453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16901293#comment-16901293
]
Íñigo Goiri commented on HADOOP-16453:
--
As this is the core for many other components, can you run HDFS and YARN tests
to verify we are not breaking anything here?
> Update how exceptions are handled in NetUtils.java
> --
>
> Key: HADOOP-16453
> URL: https://issues.apache.org/jira/browse/HADOOP-16453
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Lisheng Sun
>Assignee: Lisheng Sun
>Priority: Minor
> Attachments: HADOOP-16453.001.patch, HADOOP-16453.002.patch,
> HADOOP-16453.003.patch, HADOOP-16453.004.patch, HADOOP-16453.005.patch
>
>
> When there is no String Constructor for the exception, we Log a Trace
> Message. Given that log and throw is not a very good approach I think the
> right thing would be to just not log it at all as HADOOP-16431.
> {code:java}
> private static T wrapWithMessage(
> T exception, String msg) throws T {
> Class clazz = exception.getClass();
> try {
> Constructor ctor =
> clazz.getConstructor(String.class);
> Throwable t = ctor.newInstance(msg);
> return (T)(t.initCause(exception));
> } catch (Throwable e) {
> LOG.trace("Unable to wrap exception of type {}: it has no (String) "
> + "constructor", clazz, e);
> throw exception;
> }
> }
> {code}
> *exception stack：*
> {code:java}
> 19/07/12 11:23:45 INFO mapreduce.JobSubmitter: Executing with tokens: [Kind:
> HDFS_DELEGATION_TOKEN, Service: ha-hdfs:azorprc-xiaomi, Ident: (token for
> sql_prc: HDFS_DELEGATION_TOKEN owner=sql_prc/hadoop@XIAOMI.HADOOP,
> renewer=yarn_prc, realUser=, issueDate=1562901814007, maxDate=1594437814007,
> sequenceNumber=3349939, masterKeyId=1400)]
> 19/07/12 11:23:46 TRACE net.NetUtils: Unable to wrap exception of type class
> java.nio.channels.ClosedByInterruptException: it has no (String) constructor
> java.lang.NoSuchMethodException:
> java.nio.channels.ClosedByInterruptException.(java.lang.String)
> at java.lang.Class.getConstructor0(Class.java:3082)
> at java.lang.Class.getConstructor(Class.java:1825)
> at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:830)
> at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:806)
> at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1559)
> at org.apache.hadoop.ipc.Client.call(Client.java:1501)
> at org.apache.hadoop.ipc.Client.call(Client.java:1411)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:228)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116)
> at com.sun.proxy.$Proxy9.getFileInfo(Unknown Source)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:949)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.RequestHedgingProxyProvider$RequestHedgingInvocationHandler$1.call(RequestHedgingProxyProvider.java:143)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 WARN ipc.Client: Exception encountered while connecting to
> the server : java.io.InterruptedIOException: Interrupted while waiting for IO
> on channel java.nio.channels.SocketChannel[connected
> local=/10.118.30.48:34324 remote=/10.69.11.137:11200]. 6 millis timeout
> left.
> 19/07/12 11:23:48 INFO
[jira] [Commented] (HADOOP-16453) Update how exceptions are handled in NetUtils.java
[
https://issues.apache.org/jira/browse/HADOOP-16453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16900752#comment-16900752
]
Hadoop QA commented on HADOOP-16453:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
52s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 20m
10s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 17m
18s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
54s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
47s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 36s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
44s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 16m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 16s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
21s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 10m
22s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
11s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}106m 4s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.0 Server=19.03.0 Image:yetus/hadoop:bdbca0e53b4 |
| JIRA Issue | HADOOP-16453 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12976790/HADOOP-16453.005.patch
|
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 952ec86bdad2 4.15.0-52-generic #56-Ubuntu SMP Tue Jun 4
22:49:08 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 1127215 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_212 |
| findbugs | v3.1.0-RC1 |
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/16461/testReport/ |
| Max. process+thread count | 1495 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/16461/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> Update how exceptions are handled in NetUtils.java
> --
>
> Key: HADOOP-16453
>
[jira] [Commented] (HADOOP-16453) Update how exceptions are handled in NetUtils.java
[
https://issues.apache.org/jira/browse/HADOOP-16453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16900681#comment-16900681
]
Lisheng Sun commented on HADOOP-16453:
--
Fix UT in hadoop.ipc.TestIPC and upload the v5 patch. [~elgoiri] Could you
help reivew it ? Thank you.
> Update how exceptions are handled in NetUtils.java
> --
>
> Key: HADOOP-16453
> URL: https://issues.apache.org/jira/browse/HADOOP-16453
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Lisheng Sun
>Assignee: Lisheng Sun
>Priority: Minor
> Attachments: HADOOP-16453.001.patch, HADOOP-16453.002.patch,
> HADOOP-16453.003.patch, HADOOP-16453.004.patch, HADOOP-16453.005.patch
>
>
> When there is no String Constructor for the exception, we Log a Trace
> Message. Given that log and throw is not a very good approach I think the
> right thing would be to just not log it at all as HADOOP-16431.
> {code:java}
> private static T wrapWithMessage(
> T exception, String msg) throws T {
> Class clazz = exception.getClass();
> try {
> Constructor ctor =
> clazz.getConstructor(String.class);
> Throwable t = ctor.newInstance(msg);
> return (T)(t.initCause(exception));
> } catch (Throwable e) {
> LOG.trace("Unable to wrap exception of type {}: it has no (String) "
> + "constructor", clazz, e);
> throw exception;
> }
> }
> {code}
> *exception stack：*
> {code:java}
> 19/07/12 11:23:45 INFO mapreduce.JobSubmitter: Executing with tokens: [Kind:
> HDFS_DELEGATION_TOKEN, Service: ha-hdfs:azorprc-xiaomi, Ident: (token for
> sql_prc: HDFS_DELEGATION_TOKEN owner=sql_prc/hadoop@XIAOMI.HADOOP,
> renewer=yarn_prc, realUser=, issueDate=1562901814007, maxDate=1594437814007,
> sequenceNumber=3349939, masterKeyId=1400)]
> 19/07/12 11:23:46 TRACE net.NetUtils: Unable to wrap exception of type class
> java.nio.channels.ClosedByInterruptException: it has no (String) constructor
> java.lang.NoSuchMethodException:
> java.nio.channels.ClosedByInterruptException.(java.lang.String)
> at java.lang.Class.getConstructor0(Class.java:3082)
> at java.lang.Class.getConstructor(Class.java:1825)
> at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:830)
> at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:806)
> at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1559)
> at org.apache.hadoop.ipc.Client.call(Client.java:1501)
> at org.apache.hadoop.ipc.Client.call(Client.java:1411)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:228)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116)
> at com.sun.proxy.$Proxy9.getFileInfo(Unknown Source)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:949)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.RequestHedgingProxyProvider$RequestHedgingInvocationHandler$1.call(RequestHedgingProxyProvider.java:143)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 WARN ipc.Client: Exception encountered while connecting to
> the server : java.io.InterruptedIOException: Interrupted while waiting for IO
> on channel java.nio.channels.SocketChannel[connected
> local=/10.118.30.48:34324 remote=/10.69.11.137:11200]. 6 millis timeout
> left.
> 19/07/12 11:23:48 INFO conf.Configuration:
[jira] [Commented] (HADOOP-16453) Update how exceptions are handled in NetUtils.java
[
https://issues.apache.org/jira/browse/HADOOP-16453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16900660#comment-16900660
]
Hadoop QA commented on HADOOP-16453:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m
39s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 41m
1s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m
13s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
16s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 40s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
41s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
2s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 16m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 27s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 8m 17s{color}
| {color:red} hadoop-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
43s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}119m 33s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.ipc.TestIPC |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.1 Server=19.03.1 Image:yetus/hadoop:bdbca0e53b4 |
| JIRA Issue | HADOOP-16453 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12976782/HADOOP-16453.004.patch
|
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux a4eac5bb 4.15.0-52-generic #56-Ubuntu SMP Tue Jun 4
22:49:08 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 1127215 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_222 |
| findbugs | v3.1.0-RC1 |
| unit |
https://builds.apache.org/job/PreCommit-HADOOP-Build/16460/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/16460/testReport/ |
| Max. process+thread count | 1363 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/16460/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This
[jira] [Commented] (HADOOP-16453) Update how exceptions are handled in NetUtils.java
[
https://issues.apache.org/jira/browse/HADOOP-16453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16900592#comment-16900592
]
Lisheng Sun commented on HADOOP-16453:
--
Fix checkstyle issue and upload the v4 patch.
> Update how exceptions are handled in NetUtils.java
> --
>
> Key: HADOOP-16453
> URL: https://issues.apache.org/jira/browse/HADOOP-16453
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Lisheng Sun
>Assignee: Lisheng Sun
>Priority: Minor
> Attachments: HADOOP-16453.001.patch, HADOOP-16453.002.patch,
> HADOOP-16453.003.patch, HADOOP-16453.004.patch
>
>
> When there is no String Constructor for the exception, we Log a Trace
> Message. Given that log and throw is not a very good approach I think the
> right thing would be to just not log it at all as HADOOP-16431.
> {code:java}
> private static T wrapWithMessage(
> T exception, String msg) throws T {
> Class clazz = exception.getClass();
> try {
> Constructor ctor =
> clazz.getConstructor(String.class);
> Throwable t = ctor.newInstance(msg);
> return (T)(t.initCause(exception));
> } catch (Throwable e) {
> LOG.trace("Unable to wrap exception of type {}: it has no (String) "
> + "constructor", clazz, e);
> throw exception;
> }
> }
> {code}
> *exception stack：*
> {code:java}
> 19/07/12 11:23:45 INFO mapreduce.JobSubmitter: Executing with tokens: [Kind:
> HDFS_DELEGATION_TOKEN, Service: ha-hdfs:azorprc-xiaomi, Ident: (token for
> sql_prc: HDFS_DELEGATION_TOKEN owner=sql_prc/hadoop@XIAOMI.HADOOP,
> renewer=yarn_prc, realUser=, issueDate=1562901814007, maxDate=1594437814007,
> sequenceNumber=3349939, masterKeyId=1400)]
> 19/07/12 11:23:46 TRACE net.NetUtils: Unable to wrap exception of type class
> java.nio.channels.ClosedByInterruptException: it has no (String) constructor
> java.lang.NoSuchMethodException:
> java.nio.channels.ClosedByInterruptException.(java.lang.String)
> at java.lang.Class.getConstructor0(Class.java:3082)
> at java.lang.Class.getConstructor(Class.java:1825)
> at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:830)
> at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:806)
> at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1559)
> at org.apache.hadoop.ipc.Client.call(Client.java:1501)
> at org.apache.hadoop.ipc.Client.call(Client.java:1411)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:228)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116)
> at com.sun.proxy.$Proxy9.getFileInfo(Unknown Source)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:949)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.RequestHedgingProxyProvider$RequestHedgingInvocationHandler$1.call(RequestHedgingProxyProvider.java:143)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 WARN ipc.Client: Exception encountered while connecting to
> the server : java.io.InterruptedIOException: Interrupted while waiting for IO
> on channel java.nio.channels.SocketChannel[connected
> local=/10.118.30.48:34324 remote=/10.69.11.137:11200]. 6 millis timeout
> left.
> 19/07/12 11:23:48 INFO conf.Configuration: resource-types.xml not found
> 19/07/12 11:23:48 INFO resource.ResourceUtils: Unable to
[jira] [Commented] (HADOOP-16453) Update how exceptions are handled in NetUtils.java
[
https://issues.apache.org/jira/browse/HADOOP-16453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16900590#comment-16900590
]
Hadoop QA commented on HADOOP-16453:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
35s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 19m
44s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 17m
44s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
19s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 36s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
46s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
5s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 16m
44s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 46s{color} | {color:orange} hadoop-common-project/hadoop-common: The patch
generated 1 new + 53 unchanged - 0 fixed = 54 total (was 53) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 47s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 9m 46s{color}
| {color:red} hadoop-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
41s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}101m 26s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.ipc.TestIPC |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.1 Server=19.03.1 Image:yetus/hadoop:bdbca0e53b4 |
| JIRA Issue | HADOOP-16453 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12976766/HADOOP-16453.003.patch
|
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux c3f20fe82dbd 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24
10:55:24 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / d6697da |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_222 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HADOOP-Build/16459/artifact/out/diff-checkstyle-hadoop-common-project_hadoop-common.txt
|
| unit |
https://builds.apache.org/job/PreCommit-HADOOP-Build/16459/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/16459/testReport/ |
| Max. process+thread count | 1378 (vs. ulimit of 5500) |
| modules
[jira] [Commented] (HADOOP-16453) Update how exceptions are handled in NetUtils.java
[
https://issues.apache.org/jira/browse/HADOOP-16453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16900542#comment-16900542
]
Lisheng Sun commented on HADOOP-16453:
--
Thanx [~elgoiri] for your suggestions. I updated comments to all the places
where it was called.
Uploaded the v3 patch. Could you help review it? Thank you.
> Update how exceptions are handled in NetUtils.java
> --
>
> Key: HADOOP-16453
> URL: https://issues.apache.org/jira/browse/HADOOP-16453
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Lisheng Sun
>Assignee: Lisheng Sun
>Priority: Minor
> Attachments: HADOOP-16453.001.patch, HADOOP-16453.002.patch,
> HADOOP-16453.003.patch
>
>
> When there is no String Constructor for the exception, we Log a Trace
> Message. Given that log and throw is not a very good approach I think the
> right thing would be to just not log it at all as HADOOP-16431.
> {code:java}
> private static T wrapWithMessage(
> T exception, String msg) throws T {
> Class clazz = exception.getClass();
> try {
> Constructor ctor =
> clazz.getConstructor(String.class);
> Throwable t = ctor.newInstance(msg);
> return (T)(t.initCause(exception));
> } catch (Throwable e) {
> LOG.trace("Unable to wrap exception of type {}: it has no (String) "
> + "constructor", clazz, e);
> throw exception;
> }
> }
> {code}
> *exception stack：*
> {code:java}
> 19/07/12 11:23:45 INFO mapreduce.JobSubmitter: Executing with tokens: [Kind:
> HDFS_DELEGATION_TOKEN, Service: ha-hdfs:azorprc-xiaomi, Ident: (token for
> sql_prc: HDFS_DELEGATION_TOKEN owner=sql_prc/hadoop@XIAOMI.HADOOP,
> renewer=yarn_prc, realUser=, issueDate=1562901814007, maxDate=1594437814007,
> sequenceNumber=3349939, masterKeyId=1400)]
> 19/07/12 11:23:46 TRACE net.NetUtils: Unable to wrap exception of type class
> java.nio.channels.ClosedByInterruptException: it has no (String) constructor
> java.lang.NoSuchMethodException:
> java.nio.channels.ClosedByInterruptException.(java.lang.String)
> at java.lang.Class.getConstructor0(Class.java:3082)
> at java.lang.Class.getConstructor(Class.java:1825)
> at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:830)
> at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:806)
> at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1559)
> at org.apache.hadoop.ipc.Client.call(Client.java:1501)
> at org.apache.hadoop.ipc.Client.call(Client.java:1411)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:228)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116)
> at com.sun.proxy.$Proxy9.getFileInfo(Unknown Source)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:949)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.RequestHedgingProxyProvider$RequestHedgingInvocationHandler$1.call(RequestHedgingProxyProvider.java:143)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 WARN ipc.Client: Exception encountered while connecting to
> the server : java.io.InterruptedIOException: Interrupted while waiting for IO
> on channel java.nio.channels.SocketChannel[connected
> local=/10.118.30.48:34324 remote=/10.69.11.137:11200]. 6 millis timeout
> left.
> 19/07/12 11:23:48 INFO conf.Configuration:
[jira] [Commented] (HADOOP-16453) Update how exceptions are handled in NetUtils.java
[
https://issues.apache.org/jira/browse/HADOOP-16453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16900279#comment-16900279
]
Íñigo Goiri commented on HADOOP-16453:
--
As the unit test shows, we are changing the behavior a little bit.
I'm fine with this as far as we make sure that all the places that use this are
aware of this.
> Update how exceptions are handled in NetUtils.java
> --
>
> Key: HADOOP-16453
> URL: https://issues.apache.org/jira/browse/HADOOP-16453
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Lisheng Sun
>Assignee: Lisheng Sun
>Priority: Minor
> Attachments: HADOOP-16453.001.patch, HADOOP-16453.002.patch
>
>
> When there is no String Constructor for the exception, we Log a Trace
> Message. Given that log and throw is not a very good approach I think the
> right thing would be to just not log it at all as HADOOP-16431.
> {code:java}
> private static T wrapWithMessage(
> T exception, String msg) throws T {
> Class clazz = exception.getClass();
> try {
> Constructor ctor =
> clazz.getConstructor(String.class);
> Throwable t = ctor.newInstance(msg);
> return (T)(t.initCause(exception));
> } catch (Throwable e) {
> LOG.trace("Unable to wrap exception of type {}: it has no (String) "
> + "constructor", clazz, e);
> throw exception;
> }
> }
> {code}
> *exception stack：*
> {code:java}
> 19/07/12 11:23:45 INFO mapreduce.JobSubmitter: Executing with tokens: [Kind:
> HDFS_DELEGATION_TOKEN, Service: ha-hdfs:azorprc-xiaomi, Ident: (token for
> sql_prc: HDFS_DELEGATION_TOKEN owner=sql_prc/hadoop@XIAOMI.HADOOP,
> renewer=yarn_prc, realUser=, issueDate=1562901814007, maxDate=1594437814007,
> sequenceNumber=3349939, masterKeyId=1400)]
> 19/07/12 11:23:46 TRACE net.NetUtils: Unable to wrap exception of type class
> java.nio.channels.ClosedByInterruptException: it has no (String) constructor
> java.lang.NoSuchMethodException:
> java.nio.channels.ClosedByInterruptException.(java.lang.String)
> at java.lang.Class.getConstructor0(Class.java:3082)
> at java.lang.Class.getConstructor(Class.java:1825)
> at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:830)
> at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:806)
> at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1559)
> at org.apache.hadoop.ipc.Client.call(Client.java:1501)
> at org.apache.hadoop.ipc.Client.call(Client.java:1411)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:228)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116)
> at com.sun.proxy.$Proxy9.getFileInfo(Unknown Source)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:949)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.RequestHedgingProxyProvider$RequestHedgingInvocationHandler$1.call(RequestHedgingProxyProvider.java:143)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 WARN ipc.Client: Exception encountered while connecting to
> the server : java.io.InterruptedIOException: Interrupted while waiting for IO
> on channel java.nio.channels.SocketChannel[connected
> local=/10.118.30.48:34324 remote=/10.69.11.137:11200]. 6 millis timeout
> left.
> 19/07/12 11:23:48 INFO conf.Configuration: resource-types.xml not
[jira] [Commented] (HADOOP-16453) Update how exceptions are handled in NetUtils.java
[
https://issues.apache.org/jira/browse/HADOOP-16453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16899570#comment-16899570
]
He Xiaoqiao commented on HADOOP-16453:
--
+1 for [^HADOOP-16453.002.patch] from my side. Thanks [~leosun08].
> Update how exceptions are handled in NetUtils.java
> --
>
> Key: HADOOP-16453
> URL: https://issues.apache.org/jira/browse/HADOOP-16453
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Lisheng Sun
>Assignee: Lisheng Sun
>Priority: Minor
> Attachments: HADOOP-16453.001.patch, HADOOP-16453.002.patch
>
>
> When there is no String Constructor for the exception, we Log a Trace
> Message. Given that log and throw is not a very good approach I think the
> right thing would be to just not log it at all as HADOOP-16431.
> {code:java}
> private static T wrapWithMessage(
> T exception, String msg) throws T {
> Class clazz = exception.getClass();
> try {
> Constructor ctor =
> clazz.getConstructor(String.class);
> Throwable t = ctor.newInstance(msg);
> return (T)(t.initCause(exception));
> } catch (Throwable e) {
> LOG.trace("Unable to wrap exception of type {}: it has no (String) "
> + "constructor", clazz, e);
> throw exception;
> }
> }
> {code}
> *exception stack：*
> {code:java}
> 19/07/12 11:23:45 INFO mapreduce.JobSubmitter: Executing with tokens: [Kind:
> HDFS_DELEGATION_TOKEN, Service: ha-hdfs:azorprc-xiaomi, Ident: (token for
> sql_prc: HDFS_DELEGATION_TOKEN owner=sql_prc/hadoop@XIAOMI.HADOOP,
> renewer=yarn_prc, realUser=, issueDate=1562901814007, maxDate=1594437814007,
> sequenceNumber=3349939, masterKeyId=1400)]
> 19/07/12 11:23:46 TRACE net.NetUtils: Unable to wrap exception of type class
> java.nio.channels.ClosedByInterruptException: it has no (String) constructor
> java.lang.NoSuchMethodException:
> java.nio.channels.ClosedByInterruptException.(java.lang.String)
> at java.lang.Class.getConstructor0(Class.java:3082)
> at java.lang.Class.getConstructor(Class.java:1825)
> at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:830)
> at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:806)
> at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1559)
> at org.apache.hadoop.ipc.Client.call(Client.java:1501)
> at org.apache.hadoop.ipc.Client.call(Client.java:1411)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:228)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116)
> at com.sun.proxy.$Proxy9.getFileInfo(Unknown Source)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:949)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.RequestHedgingProxyProvider$RequestHedgingInvocationHandler$1.call(RequestHedgingProxyProvider.java:143)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 WARN ipc.Client: Exception encountered while connecting to
> the server : java.io.InterruptedIOException: Interrupted while waiting for IO
> on channel java.nio.channels.SocketChannel[connected
> local=/10.118.30.48:34324 remote=/10.69.11.137:11200]. 6 millis timeout
> left.
> 19/07/12 11:23:48 INFO conf.Configuration: resource-types.xml not found
> 19/07/12 11:23:48 INFO resource.ResourceUtils: Unable to find
> 'resource-types.xml'.
>
[jira] [Commented] (HADOOP-16453) Update how exceptions are handled in NetUtils.java
[
https://issues.apache.org/jira/browse/HADOOP-16453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16899567#comment-16899567
]
Lisheng Sun commented on HADOOP-16453:
--
hi [~elgoiri] Could you mind continue to review this patch? If it looks good
to you, you help +1 and commit it.
Thank you again.
> Update how exceptions are handled in NetUtils.java
> --
>
> Key: HADOOP-16453
> URL: https://issues.apache.org/jira/browse/HADOOP-16453
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Lisheng Sun
>Assignee: Lisheng Sun
>Priority: Minor
> Attachments: HADOOP-16453.001.patch, HADOOP-16453.002.patch
>
>
> When there is no String Constructor for the exception, we Log a Trace
> Message. Given that log and throw is not a very good approach I think the
> right thing would be to just not log it at all as HADOOP-16431.
> {code:java}
> private static T wrapWithMessage(
> T exception, String msg) throws T {
> Class clazz = exception.getClass();
> try {
> Constructor ctor =
> clazz.getConstructor(String.class);
> Throwable t = ctor.newInstance(msg);
> return (T)(t.initCause(exception));
> } catch (Throwable e) {
> LOG.trace("Unable to wrap exception of type {}: it has no (String) "
> + "constructor", clazz, e);
> throw exception;
> }
> }
> {code}
> *exception stack：*
> {code:java}
> 19/07/12 11:23:45 INFO mapreduce.JobSubmitter: Executing with tokens: [Kind:
> HDFS_DELEGATION_TOKEN, Service: ha-hdfs:azorprc-xiaomi, Ident: (token for
> sql_prc: HDFS_DELEGATION_TOKEN owner=sql_prc/hadoop@XIAOMI.HADOOP,
> renewer=yarn_prc, realUser=, issueDate=1562901814007, maxDate=1594437814007,
> sequenceNumber=3349939, masterKeyId=1400)]
> 19/07/12 11:23:46 TRACE net.NetUtils: Unable to wrap exception of type class
> java.nio.channels.ClosedByInterruptException: it has no (String) constructor
> java.lang.NoSuchMethodException:
> java.nio.channels.ClosedByInterruptException.(java.lang.String)
> at java.lang.Class.getConstructor0(Class.java:3082)
> at java.lang.Class.getConstructor(Class.java:1825)
> at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:830)
> at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:806)
> at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1559)
> at org.apache.hadoop.ipc.Client.call(Client.java:1501)
> at org.apache.hadoop.ipc.Client.call(Client.java:1411)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:228)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116)
> at com.sun.proxy.$Proxy9.getFileInfo(Unknown Source)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:949)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.RequestHedgingProxyProvider$RequestHedgingInvocationHandler$1.call(RequestHedgingProxyProvider.java:143)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 WARN ipc.Client: Exception encountered while connecting to
> the server : java.io.InterruptedIOException: Interrupted while waiting for IO
> on channel java.nio.channels.SocketChannel[connected
> local=/10.118.30.48:34324 remote=/10.69.11.137:11200]. 6 millis timeout
> left.
> 19/07/12 11:23:48 INFO conf.Configuration: resource-types.xml not found
> 19/07/12 11:23:48 INFO
[jira] [Commented] (HADOOP-16453) Update how exceptions are handled in NetUtils.java
[
https://issues.apache.org/jira/browse/HADOOP-16453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16898215#comment-16898215
]
Íñigo Goiri commented on HADOOP-16453:
--
{quote}
Íñigo Goiri Do you intent to mean that we remove the catch(Throwable ) and
replace it with the actual exceptions or add the NoMethodException separately
before, and then instead throwing exception, return the processed exception?
{quote}
Correct. Actually, [^HADOOP-16453.002.patch] looks much better.
> Update how exceptions are handled in NetUtils.java
> --
>
> Key: HADOOP-16453
> URL: https://issues.apache.org/jira/browse/HADOOP-16453
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Lisheng Sun
>Assignee: Lisheng Sun
>Priority: Minor
> Attachments: HADOOP-16453.001.patch, HADOOP-16453.002.patch
>
>
> When there is no String Constructor for the exception, we Log a Trace
> Message. Given that log and throw is not a very good approach I think the
> right thing would be to just not log it at all as HADOOP-16431.
> {code:java}
> private static T wrapWithMessage(
> T exception, String msg) throws T {
> Class clazz = exception.getClass();
> try {
> Constructor ctor =
> clazz.getConstructor(String.class);
> Throwable t = ctor.newInstance(msg);
> return (T)(t.initCause(exception));
> } catch (Throwable e) {
> LOG.trace("Unable to wrap exception of type {}: it has no (String) "
> + "constructor", clazz, e);
> throw exception;
> }
> }
> {code}
> *exception stack：*
> {code:java}
> 19/07/12 11:23:45 INFO mapreduce.JobSubmitter: Executing with tokens: [Kind:
> HDFS_DELEGATION_TOKEN, Service: ha-hdfs:azorprc-xiaomi, Ident: (token for
> sql_prc: HDFS_DELEGATION_TOKEN owner=sql_prc/hadoop@XIAOMI.HADOOP,
> renewer=yarn_prc, realUser=, issueDate=1562901814007, maxDate=1594437814007,
> sequenceNumber=3349939, masterKeyId=1400)]
> 19/07/12 11:23:46 TRACE net.NetUtils: Unable to wrap exception of type class
> java.nio.channels.ClosedByInterruptException: it has no (String) constructor
> java.lang.NoSuchMethodException:
> java.nio.channels.ClosedByInterruptException.(java.lang.String)
> at java.lang.Class.getConstructor0(Class.java:3082)
> at java.lang.Class.getConstructor(Class.java:1825)
> at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:830)
> at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:806)
> at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1559)
> at org.apache.hadoop.ipc.Client.call(Client.java:1501)
> at org.apache.hadoop.ipc.Client.call(Client.java:1411)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:228)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116)
> at com.sun.proxy.$Proxy9.getFileInfo(Unknown Source)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:949)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.RequestHedgingProxyProvider$RequestHedgingInvocationHandler$1.call(RequestHedgingProxyProvider.java:143)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 WARN ipc.Client: Exception encountered while connecting to
> the server : java.io.InterruptedIOException: Interrupted while waiting for IO
> on channel java.nio.channels.SocketChannel[connected
>
[jira] [Commented] (HADOOP-16453) Update how exceptions are handled in NetUtils.java
[
https://issues.apache.org/jira/browse/HADOOP-16453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16898139#comment-16898139
]
Hadoop QA commented on HADOOP-16453:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
56s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 25m
43s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 17m
4s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
47s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
19s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 33s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
41s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
4s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 14m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 14m
56s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 41s{color} | {color:orange} hadoop-common-project/hadoop-common: The patch
generated 1 new + 53 unchanged - 0 fixed = 54 total (was 53) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 31s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 8m 46s{color}
| {color:red} hadoop-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
42s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}103m 15s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.ipc.TestIPC |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.1 Server=19.03.1 Image:yetus/hadoop:bdbca0e53b4 |
| JIRA Issue | HADOOP-16453 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12976441/HADOOP-16453.002.patch
|
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux af0b63261c6e 4.15.0-52-generic #56-Ubuntu SMP Tue Jun 4
22:49:08 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / a7371a7 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_222 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HADOOP-Build/16446/artifact/out/diff-checkstyle-hadoop-common-project_hadoop-common.txt
|
| unit |
https://builds.apache.org/job/PreCommit-HADOOP-Build/16446/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/16446/testReport/ |
| Max. process+thread count | 1356 (vs. ulimit of 5500) |
| modules
[jira] [Commented] (HADOOP-16453) Update how exceptions are handled in NetUtils.java
[
https://issues.apache.org/jira/browse/HADOOP-16453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16898088#comment-16898088
]
Ayush Saxena commented on HADOOP-16453:
---
[~leosun08] We can't return exception just as it is, the wrap message catches
the exception and adds the message, If we just return the message will not be
added.
{code:java}
} catch (IOException ex) {
return (IOException) new IOException("Failed on local exception: "
+ exception + "; Host Details : "
+ getHostDetailsAsString(destHost, destPort, localHost))
.initCause(exception);
{code}
And we will be changing the exception, for the log issue which I think isn't
correct thing to do. We should make sure the exception returned is the same as
it is in the present scenario.
[~elgoiri] Do you intent to mean that we remove the catch(Throwable ) and
replace it with the actual exceptions or add the {{NoMethodException}}
separately before, and then instead throwing exception, return the processed
exception?
> Update how exceptions are handled in NetUtils.java
> --
>
> Key: HADOOP-16453
> URL: https://issues.apache.org/jira/browse/HADOOP-16453
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Lisheng Sun
>Assignee: Lisheng Sun
>Priority: Minor
> Attachments: HADOOP-16453.001.patch, HADOOP-16453.002.patch
>
>
> When there is no String Constructor for the exception, we Log a Trace
> Message. Given that log and throw is not a very good approach I think the
> right thing would be to just not log it at all as HADOOP-16431.
> {code:java}
> private static T wrapWithMessage(
> T exception, String msg) throws T {
> Class clazz = exception.getClass();
> try {
> Constructor ctor =
> clazz.getConstructor(String.class);
> Throwable t = ctor.newInstance(msg);
> return (T)(t.initCause(exception));
> } catch (Throwable e) {
> LOG.trace("Unable to wrap exception of type {}: it has no (String) "
> + "constructor", clazz, e);
> throw exception;
> }
> }
> {code}
> *exception stack：*
> {code:java}
> 19/07/12 11:23:45 INFO mapreduce.JobSubmitter: Executing with tokens: [Kind:
> HDFS_DELEGATION_TOKEN, Service: ha-hdfs:azorprc-xiaomi, Ident: (token for
> sql_prc: HDFS_DELEGATION_TOKEN owner=sql_prc/hadoop@XIAOMI.HADOOP,
> renewer=yarn_prc, realUser=, issueDate=1562901814007, maxDate=1594437814007,
> sequenceNumber=3349939, masterKeyId=1400)]
> 19/07/12 11:23:46 TRACE net.NetUtils: Unable to wrap exception of type class
> java.nio.channels.ClosedByInterruptException: it has no (String) constructor
> java.lang.NoSuchMethodException:
> java.nio.channels.ClosedByInterruptException.(java.lang.String)
> at java.lang.Class.getConstructor0(Class.java:3082)
> at java.lang.Class.getConstructor(Class.java:1825)
> at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:830)
> at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:806)
> at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1559)
> at org.apache.hadoop.ipc.Client.call(Client.java:1501)
> at org.apache.hadoop.ipc.Client.call(Client.java:1411)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:228)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116)
> at com.sun.proxy.$Proxy9.getFileInfo(Unknown Source)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:949)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.RequestHedgingProxyProvider$RequestHedgingInvocationHandler$1.call(RequestHedgingProxyProvider.java:143)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 INFO
[jira] [Commented] (HADOOP-16453) Update how exceptions are handled in NetUtils.java
[
https://issues.apache.org/jira/browse/HADOOP-16453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16898085#comment-16898085
]
Lisheng Sun commented on HADOOP-16453:
--
Thanx [~elgoiri] for your suggestion. I have updated this patch as your
comments and uploaded the v2 patch.
Could you mind help review it? Thank [~elgoiri] [~ayushtkn] again.
> Update how exceptions are handled in NetUtils.java
> --
>
> Key: HADOOP-16453
> URL: https://issues.apache.org/jira/browse/HADOOP-16453
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Lisheng Sun
>Assignee: Lisheng Sun
>Priority: Minor
> Attachments: HADOOP-16453.001.patch, HADOOP-16453.002.patch
>
>
> When there is no String Constructor for the exception, we Log a Trace
> Message. Given that log and throw is not a very good approach I think the
> right thing would be to just not log it at all as HADOOP-16431.
> {code:java}
> private static T wrapWithMessage(
> T exception, String msg) throws T {
> Class clazz = exception.getClass();
> try {
> Constructor ctor =
> clazz.getConstructor(String.class);
> Throwable t = ctor.newInstance(msg);
> return (T)(t.initCause(exception));
> } catch (Throwable e) {
> LOG.trace("Unable to wrap exception of type {}: it has no (String) "
> + "constructor", clazz, e);
> throw exception;
> }
> }
> {code}
> *exception stack：*
> {code:java}
> 19/07/12 11:23:45 INFO mapreduce.JobSubmitter: Executing with tokens: [Kind:
> HDFS_DELEGATION_TOKEN, Service: ha-hdfs:azorprc-xiaomi, Ident: (token for
> sql_prc: HDFS_DELEGATION_TOKEN owner=sql_prc/hadoop@XIAOMI.HADOOP,
> renewer=yarn_prc, realUser=, issueDate=1562901814007, maxDate=1594437814007,
> sequenceNumber=3349939, masterKeyId=1400)]
> 19/07/12 11:23:46 TRACE net.NetUtils: Unable to wrap exception of type class
> java.nio.channels.ClosedByInterruptException: it has no (String) constructor
> java.lang.NoSuchMethodException:
> java.nio.channels.ClosedByInterruptException.(java.lang.String)
> at java.lang.Class.getConstructor0(Class.java:3082)
> at java.lang.Class.getConstructor(Class.java:1825)
> at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:830)
> at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:806)
> at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1559)
> at org.apache.hadoop.ipc.Client.call(Client.java:1501)
> at org.apache.hadoop.ipc.Client.call(Client.java:1411)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:228)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116)
> at com.sun.proxy.$Proxy9.getFileInfo(Unknown Source)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:949)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.hdfs.server.namenode.ha.RequestHedgingProxyProvider$RequestHedgingInvocationHandler$1.call(RequestHedgingProxyProvider.java:143)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 INFO Configuration.deprecation: No unit for
> dfs.client.datanode-restart.timeout(30) assuming SECONDS
> 19/07/12 11:23:46 WARN ipc.Client: Exception encountered while connecting to
> the server : java.io.InterruptedIOException: Interrupted while waiting for IO
> on channel java.nio.channels.SocketChannel[connected
> local=/10.118.30.48:34324 remote=/10.69.11.137:11200]. 6 millis timeout
> left.
> 19/07/12 11:23:48 INFO conf.Configuration:
