[ https://issues.apache.org/jira/browse/YARN-9183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jason Lowe updated YARN-9183: ----------------------------- Priority: Blocker (was: Major) I think this is much worse than just a failed unit test. A simple MapReduce sleep job no longer succeeds with each map task failing with what looks like the same error: {noformat} 2019-01-09 22:59:36,423 WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child : org.apache.hadoop.metrics2.MetricsException: Metrics source RpcDetailedActivityForPort-1 already exists! at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.newSourceName(DefaultMetricsSystem.java:152) at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.sourceName(DefaultMetricsSystem.java:125) at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.register(MetricsSystemImpl.java:229) at org.apache.hadoop.ipc.metrics.RpcDetailedMetrics.create(RpcDetailedMetrics.java:55) at org.apache.hadoop.ipc.Client.<init>(Client.java:1341) at org.apache.hadoop.ipc.ClientCache.getClient(ClientCache.java:57) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.<init>(ProtobufRpcEngine.java:149) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.<init>(ProtobufRpcEngine.java:136) at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.<init>(ProtobufRpcEngine.java:120) at org.apache.hadoop.ipc.ProtobufRpcEngine.getProxy(ProtobufRpcEngine.java:102) at org.apache.hadoop.ipc.RPC.getProtocolProxy(RPC.java:624) at org.apache.hadoop.hdfs.NameNodeProxiesClient.createProxyWithAlignmentContext(NameNodeProxiesClient.java:370) at org.apache.hadoop.hdfs.NameNodeProxiesClient.createNonHAProxyWithClientProtocol(NameNodeProxiesClient.java:348) at org.apache.hadoop.hdfs.server.namenode.ha.ClientHAProxyFactory.createProxy(ClientHAProxyFactory.java:46) at org.apache.hadoop.hdfs.server.namenode.ha.AbstractNNFailoverProxyProvider.createProxyIfNeeded(AbstractNNFailoverProxyProvider.java:152) at org.apache.hadoop.hdfs.server.namenode.ha.IPFailoverProxyProvider.getProxy(IPFailoverProxyProvider.java:57) at org.apache.hadoop.hdfs.server.namenode.ha.IPFailoverProxyProvider.getProxy(IPFailoverProxyProvider.java:44) at org.apache.hadoop.io.retry.RetryInvocationHandler$ProxyDescriptor.<init>(RetryInvocationHandler.java:197) at org.apache.hadoop.io.retry.RetryInvocationHandler.<init>(RetryInvocationHandler.java:328) at org.apache.hadoop.io.retry.RetryInvocationHandler.<init>(RetryInvocationHandler.java:322) at org.apache.hadoop.io.retry.RetryProxy.create(RetryProxy.java:59) at org.apache.hadoop.hdfs.NameNodeProxiesClient.createHAProxy(NameNodeProxiesClient.java:326) at org.apache.hadoop.hdfs.NameNodeProxiesClient.createProxyWithClientProtocol(NameNodeProxiesClient.java:144) at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:356) at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:290) at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:176) at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3312) at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:124) at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:3361) at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3329) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:479) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:227) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:177) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:172) 2019-01-09 22:59:36,431 INFO [main] org.apache.hadoop.mapred.Task: Running cleanup for the task 2019-01-09 22:59:36,432 INFO [main] org.apache.hadoop.mapred.YarnChild: Exception cleaning up: java.lang.NullPointerException at org.apache.hadoop.mapred.Task.taskCleanup(Task.java:1458) at org.apache.hadoop.mapred.YarnChild$3.run(YarnChild.java:200) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:197) 2019-01-09 22:59:36,544 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping MapTask metrics system... 2019-01-09 22:59:36,544 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system stopped. 2019-01-09 22:59:36,545 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system shutdown complete. {noformat} Not being able to run jobs is pretty bad, so marking this as a Blocker. > TestAMRMTokens fails > -------------------- > > Key: YARN-9183 > URL: https://issues.apache.org/jira/browse/YARN-9183 > Project: Hadoop YARN > Issue Type: Bug > Reporter: Akira Ajisaka > Assignee: Abhishek Modi > Priority: Blocker > > TestAMRMTokens.testMasterKeyRollOver and TestAMRMTokens.testTokenExpiry is > failing. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org