Re: class org.apache.hadoop.yarn.proto.YarnProtos$ApplicationIdProto overrides final method getUnknownFields
lgLkoIzo9yOjPdx2NN6DllKbV65UjTay43uUDyql9g3tl RhiJIcoAExkSTykWqAIPR88LfilLy1JlQ+0RD8OXiWOVVQfhOHpQ0R/jcAkM2lZa BjM8j36yJvoBVsfOHQIDAQAB -END PUBLIC KEY- On 03/03/14 19:05, Ted Yu wrote: Can you tell us the hadoop release you're using ? Seems there is inconsistency in protobuf library. On Mon, Mar 3, 2014 at 8:01 AM, Margusja <mailto:mar...@roo.ee>> wrote: Hi I even don't know what information to provide but my container log is: 2014-03-03 17:36:05,311 FATAL [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Error starting MRAppMaster java.lang.VerifyError: class org.apache.hadoop.yarn.proto.YarnProtos$ApplicationIdProto overrides final method getUnknownFields.()Lcom/google/protobuf/UnknownFieldSet; at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:800) at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) at java.net.URLClassLoader.defineClass(URLClassLoader.java:449) at java.net.URLClassLoader.access$100(URLClassLoader.java:71) at java.net.URLClassLoader$1.run(URLClassLoader.java:361) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:425) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) at java.lang.ClassLoader.loadClass(ClassLoader.java:358) at java.lang.Class.getDeclaredConstructors0(Native Method) at java.lang.Class.privateGetDeclaredConstructors(Class.java:2493) at java.lang.Class.getConstructor0(Class.java:2803) at java.lang.Class.getConstructor(Class.java:1718) at org.apache.hadoop.yarn.factories.impl.pb.RecordFactoryPBImpl.newRecordInstance(RecordFactoryPBImpl.java:62) at org.apache.hadoop.yarn.util.Records.newRecord(Records.java:36) at org.apache.hadoop.yarn.api.records.ApplicationId.newInstance(ApplicationId.java:49) at org.apache.hadoop.yarn.util.ConverterUtils.toApplicationAttemptId(ConverterUtils.java:137) at org.apache.hadoop.yarn.util.ConverterUtils.toContainerId(ConverterUtils.java:177) at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1343) Where to start digging? -- Tervitades, Margus (Margusja) Roo +372 51 48 780 http://margus.roo.ee http://ee.linkedin.com/in/margusroo skype: margusja ldapsearch -x -h ldap.sk.ee <http://ldap.sk.ee> -b c=EE "(serialNumber=37303140314)" -BEGIN PUBLIC KEY- MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCvbeg7LwEC2SCpAEewwpC3ajxE 5ZsRMCB77L8bae9G7TslgLkoIzo9yOjPdx2NN6DllKbV65UjTay43uUDyql9g3tl RhiJIcoAExkSTykWqAIPR88LfilLy1JlQ+0RD8OXiWOVVQfhOHpQ0R/jcAkM2lZa BjM8j36yJvoBVsfOHQIDAQAB -END PUBLIC KEY-
Re: class org.apache.hadoop.yarn.proto.YarnProtos$ApplicationIdProto overrides final method getUnknownFields
:13:19,496 INFO org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: Deleting absolute path : /tmp/hadoop-hdfs/nm-local-dir/usercache/hduser/appcache/application_1393848686226_0019/container_1393848686226_0019_02_01 2014-03-03 19:13:19,498 WARN org.apache.hadoop.yarn.server.nodemanager.NMAuditLogger: USER=hduser OPERATION=Container Finished - Failed TARGET=ContainerImplRESULT=FAILURE DESCRIPTION=Container failed with state: EXITED_WITH_FAILURE APPID=application_1393848686226_0019 CONTAINERID=container_1393848686226_0019_02_01 2014-03-03 19:13:19,498 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Container: Container container_1393848686226_0019_02_01 transitioned from EXITED_WITH_FAILURE to DONE 2014-03-03 19:13:19,498 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.application.Application: Removing container_1393848686226_0019_02_01 from application application_1393848686226_0019 2014-03-03 19:13:19,499 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices: Got event CONTAINER_STOP for appId application_1393848686226_0019 2014-03-03 19:13:20,160 INFO org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Sending out status for container: container_id { app_attempt_id { application_id { id: 19 cluster_timestamp: 1393848686226 } attemptId: 2 } id: 1 } state: C_COMPLETE diagnostics: "Exception from container-launch: \norg.apache.hadoop.util.Shell$ExitCodeException: \n\tat org.apache.hadoop.util.Shell.runCommand(Shell.java:464)\n\tat org.apache.hadoop.util.Shell.run(Shell.java:379)\n\tat org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:589)\n\tat org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195)\n\tat org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:283)\n\tat org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:79)\n\tat java.util.concurrent.FutureTask.run(FutureTask.java:262)\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)\n\tat java.lang.Thread.run(Thread.java:744)\n\n\n" exit_status: 1 2014-03-03 19:13:20,161 INFO org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Removed completed container container_1393848686226_0019_02_01 2014-03-03 19:13:20,542 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Starting resource-monitoring for container_1393848686226_0019_02_01 2014-03-03 19:13:20,543 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Stopping resource-monitoring for container_1393848686226_0019_02_01 2014-03-03 19:13:21,164 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.application.Application: Application application_1393848686226_0019 transitioned from RUNNING to APPLICATION_RESOURCES_CLEANINGUP 2014-03-03 19:13:21,164 INFO org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: Deleting absolute path : /tmp/hadoop-hdfs/nm-local-dir/usercache/hduser/appcache/application_1393848686226_0019 2014-03-03 19:13:21,165 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices: Got event APPLICATION_STOP for appId application_1393848686226_0019 2014-03-03 19:13:21,165 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.application.Application: Application application_1393848686226_0019 transitioned from APPLICATION_RESOURCES_CLEANINGUP to FINISHED 2014-03-03 19:13:21,165 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.NonAggregatingLogHandler: Scheduling Log Deletion for application: application_1393848686226_0019, with delay of 10800 seconds ... Tervitades, Margus (Margusja) Roo +372 51 48 780 http://margus.roo.ee http://ee.linkedin.com/in/margusroo skype: margusja ldapsearch -x -h ldap.sk.ee -b c=EE "(serialNumber=37303140314)" -BEGIN PUBLIC KEY- MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCvbeg7LwEC2SCpAEewwpC3ajxE 5ZsRMCB77L8bae9G7TslgLkoIzo9yOjPdx2NN6DllKbV65UjTay43uUDyql9g3tl RhiJIcoAExkSTykWqAIPR88LfilLy1JlQ+0RD8OXiWOVVQfhOHpQ0R/jcAkM2lZa BjM8j36yJvoBVsfOHQIDAQAB -END PUBLIC KEY- On 03/03/14 19:05, Ted Yu wrote: Can you tell us the hadoop release you're using ? Seems there is inconsistency in protobuf library. On Mon, Mar 3, 2014 at 8:01 AM, Margusja <mailto:mar...@roo.ee>> wrote: Hi I even don't know what information to provide but my container log is: 2014-03-03 17:36:05,311 FATAL [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Error starting MRAppMaster java.lang.VerifyError: class org.apache.hadoop.yarn.proto.Ya
Re: class org.apache.hadoop.yarn.proto.YarnProtos$ApplicationIdProto overrides final method getUnknownFields
Thank you for replay, I got it work. [hduser@vm38 ~]$ /usr/lib/hadoop-yarn/bin/yarn version Hadoop 2.2.0.2.0.6.0-101 Subversion g...@github.com:hortonworks/hadoop.git -r b07b2906c36defd389c8b5bd22bebc1bead8115b Compiled by jenkins on 2014-01-09T05:18Z Compiled with protoc 2.5.0 From source with checksum 704f1e463ebc4fb89353011407e965 This command was run using /usr/lib/hadoop/hadoop-common-2.2.0.2.0.6.0-101.jar [hduser@vm38 ~]$ The main problem I think was I had yarn binary in two places and I used wrong one that didn't use my yarn-site.xml. Every time I look into .staging/job.../job.xml there were values from yarn-default.xml even I set them in yarn-site.xml. Typical mess up :) Tervitades, Margus (Margusja) Roo +372 51 48 780 http://margus.roo.ee http://ee.linkedin.com/in/margusroo skype: margusja ldapsearch -x -h ldap.sk.ee -b c=EE "(serialNumber=37303140314)" -BEGIN PUBLIC KEY- MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCvbeg7LwEC2SCpAEewwpC3ajxE 5ZsRMCB77L8bae9G7TslgLkoIzo9yOjPdx2NN6DllKbV65UjTay43uUDyql9g3tl RhiJIcoAExkSTykWqAIPR88LfilLy1JlQ+0RD8OXiWOVVQfhOHpQ0R/jcAkM2lZa BjM8j36yJvoBVsfOHQIDAQAB -END PUBLIC KEY- On 04/03/14 05:14, Rohith Sharma K S wrote: Hi The reason for " org.apache.hadoop.yarn.proto.YarnProtos$ApplicationIdProto overrides final method getUnknownFields.()Lcom/google/protobuf/UnknownFieldSet" is hadoop is compiled with protoc-2.5.0 version, but in the classpath lower version of protobuf is present. 1. Check MRAppMaster classpath, which version of protobuf is in classpath. Expected to have 2.5.0 version. Thanks & Regards Rohith Sharma K S -Original Message- From: Margusja [mailto:mar...@roo.ee] Sent: 03 March 2014 22:45 To: user@hadoop.apache.org Subject: Re: class org.apache.hadoop.yarn.proto.YarnProtos$ApplicationIdProto overrides final method getUnknownFields Hi 2.2.0 and 2.3.0 gave me the same container log. A little bit more details. I'll try to use external java client who submits job. some lines from maven pom.xml file: org.apache.hadoop hadoop-client 2.3.0 org.apache.hadoop hadoop-core 1.2.1 lines from external client: ... 2014-03-03 17:36:01 INFO FileInputFormat:287 - Total input paths to process : 1 2014-03-03 17:36:02 INFO JobSubmitter:396 - number of splits:1 2014-03-03 17:36:03 INFO JobSubmitter:479 - Submitting tokens for job: job_1393848686226_0018 2014-03-03 17:36:04 INFO YarnClientImpl:166 - Submitted application application_1393848686226_0018 2014-03-03 17:36:04 INFO Job:1289 - The url to track the job: http://vm38.dbweb.ee:8088/proxy/application_1393848686226_0018/ 2014-03-03 17:36:04 INFO Job:1334 - Running job: job_1393848686226_0018 2014-03-03 17:36:10 INFO Job:1355 - Job job_1393848686226_0018 running in uber mode : false 2014-03-03 17:36:10 INFO Job:1362 - map 0% reduce 0% 2014-03-03 17:36:10 INFO Job:1375 - Job job_1393848686226_0018 failed with state FAILED due to: Application application_1393848686226_0018 failed 2 times due to AM Container for appattempt_1393848686226_0018_02 exited with exitCode: 1 due to: Exception from container-launch: org.apache.hadoop.util.Shell$ExitCodeException: at org.apache.hadoop.util.Shell.runCommand(Shell.java:464) at org.apache.hadoop.util.Shell.run(Shell.java:379) at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:589) at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:283) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:79) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) ... Lines from namenode: ... 14/03/03 19:12:42 INFO namenode.FSEditLog: Number of transactions: 900 Total time for transactions(ms): 69 Number of transactions batched in Syncs: 0 Number of syncs: 542 SyncTimes(ms): 9783 14/03/03 19:12:42 INFO BlockStateChange: BLOCK* addToInvalidates: blk_1073742050_1226 90.190.106.33:50010 14/03/03 19:12:42 INFO hdfs.StateChange: BLOCK* allocateBlock: /user/hduser/input/data666.noheader.data. BP-802201089-90.190.106.33-1393506052071 blk_1073742056_1232{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[90.190.106.33:50010|RBW]]} 14/03/03 19:12:44 INFO hdfs.StateChange: BLOCK* InvalidateBlocks: ask 90.190.106.33:50010 to delete [blk_1073742050_1226] 14/03/03 19:12:53 INFO BlockStateChange: BLOCK* addStoredBlock: blockMap updated: 90.190.106.33:5001
RE: class org.apache.hadoop.yarn.proto.YarnProtos$ApplicationIdProto overrides final method getUnknownFields
Hi The reason for " org.apache.hadoop.yarn.proto.YarnProtos$ApplicationIdProto overrides final method getUnknownFields.()Lcom/google/protobuf/UnknownFieldSet" is hadoop is compiled with protoc-2.5.0 version, but in the classpath lower version of protobuf is present. 1. Check MRAppMaster classpath, which version of protobuf is in classpath. Expected to have 2.5.0 version. Thanks & Regards Rohith Sharma K S -Original Message- From: Margusja [mailto:mar...@roo.ee] Sent: 03 March 2014 22:45 To: user@hadoop.apache.org Subject: Re: class org.apache.hadoop.yarn.proto.YarnProtos$ApplicationIdProto overrides final method getUnknownFields Hi 2.2.0 and 2.3.0 gave me the same container log. A little bit more details. I'll try to use external java client who submits job. some lines from maven pom.xml file: org.apache.hadoop hadoop-client 2.3.0 org.apache.hadoop hadoop-core 1.2.1 lines from external client: ... 2014-03-03 17:36:01 INFO FileInputFormat:287 - Total input paths to process : 1 2014-03-03 17:36:02 INFO JobSubmitter:396 - number of splits:1 2014-03-03 17:36:03 INFO JobSubmitter:479 - Submitting tokens for job: job_1393848686226_0018 2014-03-03 17:36:04 INFO YarnClientImpl:166 - Submitted application application_1393848686226_0018 2014-03-03 17:36:04 INFO Job:1289 - The url to track the job: http://vm38.dbweb.ee:8088/proxy/application_1393848686226_0018/ 2014-03-03 17:36:04 INFO Job:1334 - Running job: job_1393848686226_0018 2014-03-03 17:36:10 INFO Job:1355 - Job job_1393848686226_0018 running in uber mode : false 2014-03-03 17:36:10 INFO Job:1362 - map 0% reduce 0% 2014-03-03 17:36:10 INFO Job:1375 - Job job_1393848686226_0018 failed with state FAILED due to: Application application_1393848686226_0018 failed 2 times due to AM Container for appattempt_1393848686226_0018_02 exited with exitCode: 1 due to: Exception from container-launch: org.apache.hadoop.util.Shell$ExitCodeException: at org.apache.hadoop.util.Shell.runCommand(Shell.java:464) at org.apache.hadoop.util.Shell.run(Shell.java:379) at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:589) at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:283) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:79) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) ... Lines from namenode: ... 14/03/03 19:12:42 INFO namenode.FSEditLog: Number of transactions: 900 Total time for transactions(ms): 69 Number of transactions batched in Syncs: 0 Number of syncs: 542 SyncTimes(ms): 9783 14/03/03 19:12:42 INFO BlockStateChange: BLOCK* addToInvalidates: blk_1073742050_1226 90.190.106.33:50010 14/03/03 19:12:42 INFO hdfs.StateChange: BLOCK* allocateBlock: /user/hduser/input/data666.noheader.data. BP-802201089-90.190.106.33-1393506052071 blk_1073742056_1232{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[90.190.106.33:50010|RBW]]} 14/03/03 19:12:44 INFO hdfs.StateChange: BLOCK* InvalidateBlocks: ask 90.190.106.33:50010 to delete [blk_1073742050_1226] 14/03/03 19:12:53 INFO BlockStateChange: BLOCK* addStoredBlock: blockMap updated: 90.190.106.33:50010 is added to blk_1073742056_1232{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[90.190.106.33:50010|RBW]]} size 0 14/03/03 19:12:53 INFO hdfs.StateChange: DIR* completeFile: /user/hduser/input/data666.noheader.data is closed by DFSClient_NONMAPREDUCE_-915999412_15 14/03/03 19:12:54 INFO BlockStateChange: BLOCK* addToInvalidates: blk_1073742051_1227 90.190.106.33:50010 14/03/03 19:12:54 INFO hdfs.StateChange: BLOCK* allocateBlock: /user/hduser/input/data666.noheader.data.info. BP-802201089-90.190.106.33-1393506052071 blk_1073742057_1233{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[90.190.106.33:50010|RBW]]} 14/03/03 19:12:54 INFO BlockStateChange: BLOCK* addStoredBlock: blockMap updated: 90.190.106.33:50010 is added to blk_1073742057_1233{blockUCState=UNDER_CONSTRUCTION, primaryNodeIndex=-1, replicas=[ReplicaUnderConstruction[90.190.106.33:50010|RBW]]} size 0 14/03/03 19:12:54 INFO hdfs.StateChange: DIR* completeFile: /user/hduser/input/data666.noheader.data.info is closed by DFSClient_NONMAPREDUCE_-915999412_15 14/03/03 19:12:55 INFO hdfs.StateChange: BLOCK* allocateBlock: /user/hduser/.staging/job_1393848686226_0019/job
Re: class org.apache.hadoop.yarn.proto.YarnProtos$ApplicationIdProto overrides final method getUnknownFields
gHandler: > Scheduling Log Deletion for application: application_1393848686226_0019, > with delay of 10800 seconds > ... > > > > Tervitades, Margus (Margusja) Roo > +372 51 48 780 > http://margus.roo.ee > http://ee.linkedin.com/in/margusroo > skype: margusja > ldapsearch -x -h ldap.sk.ee -b c=EE "(serialNumber=37303140314)" > -BEGIN PUBLIC KEY----- > MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCvbeg7LwEC2SCpAEewwpC3ajxE > 5ZsRMCB77L8bae9G7TslgLkoIzo9yOjPdx2NN6DllKbV65UjTay43uUDyql9g3tl > RhiJIcoAExkSTykWqAIPR88LfilLy1JlQ+0RD8OXiWOVVQfhOHpQ0R/jcAkM2lZa > BjM8j36yJvoBVsfOHQIDAQAB > -END PUBLIC KEY- > > On 03/03/14 19:05, Ted Yu wrote: > >> Can you tell us the hadoop release you're using ? >> >> Seems there is inconsistency in protobuf library. >> >> >> On Mon, Mar 3, 2014 at 8:01 AM, Margusja > mar...@roo.ee>> wrote: >> >> Hi >> >> I even don't know what information to provide but my container log is: >> >> 2014-03-03 17:36:05,311 FATAL [main] >> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Error starting >> MRAppMaster >> java.lang.VerifyError: class >> org.apache.hadoop.yarn.proto.YarnProtos$ApplicationIdProto >> overrides final method >> getUnknownFields.()Lcom/google/protobuf/UnknownFieldSet; >> at java.lang.ClassLoader.defineClass1(Native Method) >> at java.lang.ClassLoader.defineClass(ClassLoader.java:800) >> at >> java.security.SecureClassLoader.defineClass( >> SecureClassLoader.java:142) >> at >> java.net.URLClassLoader.defineClass(URLClassLoader.java:449) >> at java.net.URLClassLoader.access$100(URLClassLoader.java:71) >> at java.net.URLClassLoader$1.run(URLClassLoader.java:361) >> at java.net.URLClassLoader$1.run(URLClassLoader.java:355) >> at java.security.AccessController.doPrivileged(Native Method) >> at java.net.URLClassLoader.findClass(URLClassLoader.java:354) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:425) >> at >> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:358) >> at java.lang.Class.getDeclaredConstructors0(Native Method) >> at >> java.lang.Class.privateGetDeclaredConstructors(Class.java:2493) >> at java.lang.Class.getConstructor0(Class.java:2803) >> at java.lang.Class.getConstructor(Class.java:1718) >> at >> org.apache.hadoop.yarn.factories.impl.pb.RecordFactoryPBImpl. >> newRecordInstance(RecordFactoryPBImpl.java:62) >> at >> org.apache.hadoop.yarn.util.Records.newRecord(Records.java:36) >> at >> org.apache.hadoop.yarn.api.records.ApplicationId. >> newInstance(ApplicationId.java:49) >> at >> org.apache.hadoop.yarn.util.ConverterUtils.toApplicationAttemptId( >> ConverterUtils.java:137) >> at >> org.apache.hadoop.yarn.util.ConverterUtils.toContainerId( >> ConverterUtils.java:177) >> at >> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main( >> MRAppMaster.java:1343) >> >> >> Where to start digging? >> >> -- Tervitades, Margus (Margusja) Roo >> +372 51 48 780 >> >> http://margus.roo.ee >> http://ee.linkedin.com/in/margusroo >> skype: margusja >> ldapsearch -x -h ldap.sk.ee <http://ldap.sk.ee> -b c=EE >> >> "(serialNumber=37303140314)" >> -BEGIN PUBLIC KEY- >> MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCvbeg7LwEC2SCpAEewwpC3ajxE >> 5ZsRMCB77L8bae9G7TslgLkoIzo9yOjPdx2NN6DllKbV65UjTay43uUDyql9g3tl >> RhiJIcoAExkSTykWqAIPR88LfilLy1JlQ+0RD8OXiWOVVQfhOHpQ0R/jcAkM2lZa >> BjM8j36yJvoBVsfOHQIDAQAB >> -END PUBLIC KEY- >> >> >> >
Re: class org.apache.hadoop.yarn.proto.YarnProtos$ApplicationIdProto overrides final method getUnknownFields
ILURE DESCRIPTION=Container failed with state: EXITED_WITH_FAILURE APPID=application_1393848686226_0019 CONTAINERID=container_1393848686226_0019_02_01 2014-03-03 19:13:19,498 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Container: Container container_1393848686226_0019_02_01 transitioned from EXITED_WITH_FAILURE to DONE 2014-03-03 19:13:19,498 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.application.Application: Removing container_1393848686226_0019_02_01 from application application_1393848686226_0019 2014-03-03 19:13:19,499 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices: Got event CONTAINER_STOP for appId application_1393848686226_0019 2014-03-03 19:13:20,160 INFO org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Sending out status for container: container_id { app_attempt_id { application_id { id: 19 cluster_timestamp: 1393848686226 } attemptId: 2 } id: 1 } state: C_COMPLETE diagnostics: "Exception from container-launch: \norg.apache.hadoop.util.Shell$ExitCodeException: \n\tat org.apache.hadoop.util.Shell.runCommand(Shell.java:464)\n\tat org.apache.hadoop.util.Shell.run(Shell.java:379)\n\tat org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:589)\n\tat org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195)\n\tat org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:283)\n\tat org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:79)\n\tat java.util.concurrent.FutureTask.run(FutureTask.java:262)\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)\n\tat java.lang.Thread.run(Thread.java:744)\n\n\n" exit_status: 1 2014-03-03 19:13:20,161 INFO org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Removed completed container container_1393848686226_0019_02_01 2014-03-03 19:13:20,542 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Starting resource-monitoring for container_1393848686226_0019_02_01 2014-03-03 19:13:20,543 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Stopping resource-monitoring for container_1393848686226_0019_02_01 2014-03-03 19:13:21,164 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.application.Application: Application application_1393848686226_0019 transitioned from RUNNING to APPLICATION_RESOURCES_CLEANINGUP 2014-03-03 19:13:21,164 INFO org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: Deleting absolute path : /tmp/hadoop-hdfs/nm-local-dir/usercache/hduser/appcache/application_1393848686226_0019 2014-03-03 19:13:21,165 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices: Got event APPLICATION_STOP for appId application_1393848686226_0019 2014-03-03 19:13:21,165 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.application.Application: Application application_1393848686226_0019 transitioned from APPLICATION_RESOURCES_CLEANINGUP to FINISHED 2014-03-03 19:13:21,165 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.NonAggregatingLogHandler: Scheduling Log Deletion for application: application_1393848686226_0019, with delay of 10800 seconds ... Tervitades, Margus (Margusja) Roo +372 51 48 780 http://margus.roo.ee http://ee.linkedin.com/in/margusroo skype: margusja ldapsearch -x -h ldap.sk.ee -b c=EE "(serialNumber=37303140314)" -BEGIN PUBLIC KEY- MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCvbeg7LwEC2SCpAEewwpC3ajxE 5ZsRMCB77L8bae9G7TslgLkoIzo9yOjPdx2NN6DllKbV65UjTay43uUDyql9g3tl RhiJIcoAExkSTykWqAIPR88LfilLy1JlQ+0RD8OXiWOVVQfhOHpQ0R/jcAkM2lZa BjM8j36yJvoBVsfOHQIDAQAB -END PUBLIC KEY- On 03/03/14 19:05, Ted Yu wrote: Can you tell us the hadoop release you're using ? Seems there is inconsistency in protobuf library. On Mon, Mar 3, 2014 at 8:01 AM, Margusja <mailto:mar...@roo.ee>> wrote: Hi I even don't know what information to provide but my container log is: 2014-03-03 17:36:05,311 FATAL [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Error starting MRAppMaster java.lang.VerifyError: class org.apache.hadoop.yarn.proto.YarnProtos$ApplicationIdProto overrides final method getUnknownFields.()Lcom/google/protobuf/UnknownFieldSet; at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:800) at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
Re: class org.apache.hadoop.yarn.proto.YarnProtos$ApplicationIdProto overrides final method getUnknownFields
Can you tell us the hadoop release you're using ? Seems there is inconsistency in protobuf library. On Mon, Mar 3, 2014 at 8:01 AM, Margusja wrote: > Hi > > I even don't know what information to provide but my container log is: > > 2014-03-03 17:36:05,311 FATAL [main] > org.apache.hadoop.mapreduce.v2.app.MRAppMaster: > Error starting MRAppMaster > java.lang.VerifyError: class > org.apache.hadoop.yarn.proto.YarnProtos$ApplicationIdProto > overrides final method getUnknownFields.()Lcom/google/protobuf/ > UnknownFieldSet; > at java.lang.ClassLoader.defineClass1(Native Method) > at java.lang.ClassLoader.defineClass(ClassLoader.java:800) > at java.security.SecureClassLoader.defineClass( > SecureClassLoader.java:142) > at java.net.URLClassLoader.defineClass(URLClassLoader.java:449) > at java.net.URLClassLoader.access$100(URLClassLoader.java:71) > at java.net.URLClassLoader$1.run(URLClassLoader.java:361) > at java.net.URLClassLoader$1.run(URLClassLoader.java:355) > at java.security.AccessController.doPrivileged(Native Method) > at java.net.URLClassLoader.findClass(URLClassLoader.java:354) > at java.lang.ClassLoader.loadClass(ClassLoader.java:425) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) > at java.lang.ClassLoader.loadClass(ClassLoader.java:358) > at java.lang.Class.getDeclaredConstructors0(Native Method) > at java.lang.Class.privateGetDeclaredConstructors(Class.java:2493) > at java.lang.Class.getConstructor0(Class.java:2803) > at java.lang.Class.getConstructor(Class.java:1718) > at org.apache.hadoop.yarn.factories.impl.pb.RecordFactoryPBImpl. > newRecordInstance(RecordFactoryPBImpl.java:62) > at org.apache.hadoop.yarn.util.Records.newRecord(Records.java:36) > at org.apache.hadoop.yarn.api.records.ApplicationId. > newInstance(ApplicationId.java:49) > at org.apache.hadoop.yarn.util.ConverterUtils. > toApplicationAttemptId(ConverterUtils.java:137) > at org.apache.hadoop.yarn.util.ConverterUtils.toContainerId( > ConverterUtils.java:177) > at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main( > MRAppMaster.java:1343) > > > Where to start digging? > > -- > Tervitades, Margus (Margusja) Roo > +372 51 48 780 > http://margus.roo.ee > http://ee.linkedin.com/in/margusroo > skype: margusja > ldapsearch -x -h ldap.sk.ee -b c=EE "(serialNumber=37303140314)" > -BEGIN PUBLIC KEY- > MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCvbeg7LwEC2SCpAEewwpC3ajxE > 5ZsRMCB77L8bae9G7TslgLkoIzo9yOjPdx2NN6DllKbV65UjTay43uUDyql9g3tl > RhiJIcoAExkSTykWqAIPR88LfilLy1JlQ+0RD8OXiWOVVQfhOHpQ0R/jcAkM2lZa > BjM8j36yJvoBVsfOHQIDAQAB > -END PUBLIC KEY- > >
class org.apache.hadoop.yarn.proto.YarnProtos$ApplicationIdProto overrides final method getUnknownFields
Hi I even don't know what information to provide but my container log is: 2014-03-03 17:36:05,311 FATAL [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Error starting MRAppMaster java.lang.VerifyError: class org.apache.hadoop.yarn.proto.YarnProtos$ApplicationIdProto overrides final method getUnknownFields.()Lcom/google/protobuf/UnknownFieldSet; at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:800) at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) at java.net.URLClassLoader.defineClass(URLClassLoader.java:449) at java.net.URLClassLoader.access$100(URLClassLoader.java:71) at java.net.URLClassLoader$1.run(URLClassLoader.java:361) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:425) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) at java.lang.ClassLoader.loadClass(ClassLoader.java:358) at java.lang.Class.getDeclaredConstructors0(Native Method) at java.lang.Class.privateGetDeclaredConstructors(Class.java:2493) at java.lang.Class.getConstructor0(Class.java:2803) at java.lang.Class.getConstructor(Class.java:1718) at org.apache.hadoop.yarn.factories.impl.pb.RecordFactoryPBImpl.newRecordInstance(RecordFactoryPBImpl.java:62) at org.apache.hadoop.yarn.util.Records.newRecord(Records.java:36) at org.apache.hadoop.yarn.api.records.ApplicationId.newInstance(ApplicationId.java:49) at org.apache.hadoop.yarn.util.ConverterUtils.toApplicationAttemptId(ConverterUtils.java:137) at org.apache.hadoop.yarn.util.ConverterUtils.toContainerId(ConverterUtils.java:177) at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1343) Where to start digging? -- Tervitades, Margus (Margusja) Roo +372 51 48 780 http://margus.roo.ee http://ee.linkedin.com/in/margusroo skype: margusja ldapsearch -x -h ldap.sk.ee -b c=EE "(serialNumber=37303140314)" -BEGIN PUBLIC KEY- MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCvbeg7LwEC2SCpAEewwpC3ajxE 5ZsRMCB77L8bae9G7TslgLkoIzo9yOjPdx2NN6DllKbV65UjTay43uUDyql9g3tl RhiJIcoAExkSTykWqAIPR88LfilLy1JlQ+0RD8OXiWOVVQfhOHpQ0R/jcAkM2lZa BjM8j36yJvoBVsfOHQIDAQAB -END PUBLIC KEY-