Challenges after MapR 5.1 Upgrade.
I had at one point Myriad working fine in MapR 5.0. I updated to 5.1, and repackaged my hadoop tgz for remote distribution and now I have two problems occurring. 1. At first when I had the mapr direct shuffle enabled per the yarn-site.xml on the myriad documentaion, node managers would not start, and would fail with the error below. 2. Once I removed the mapr shuffle from the yarn-site, I got node managers started however, when I tried to launch a size 0, I got the other error below. Not sure what's happening here. Any thoughts would be appreciated. Like I said, this was working with 5.0, and now doesn't work in 5.1. Thanks! John SHuffle Error 16/04/04 13:46:34 INFO service.AbstractService: Service NodeManager failed in state INITED; cause: java.lang.RuntimeException: No class defined for mapr_direct_shuffle java.lang.RuntimeException: No class defined for mapr_direct_shuffle at org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices.serviceInit(AuxServices.java:139) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107) at org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.serviceInit(ContainerManagerImpl.java:250) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107) at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceInit(NodeManager.java:256) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:476) at org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:524) 16/04/04 13:46:34 INFO impl.MetricsSystemImpl: Stopping NodeManager metrics system... Zero Sized Node Manager Error: 16/04/04 14:22:49 INFO service.AbstractService: Service org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl failed in state STARTED; cause: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Recieved SHUTDOWN signal from Resourcemanager ,Registration of NodeManager failed, Message from ResourceManager: NodeManager from hadoopmapr4.brewingintel.com doesn't satisfy minimum allocations, Sending SHUTDOWN signal to the NodeManager. org.apache.hadoop.yarn.exceptions.YarnRuntimeException: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Recieved SHUTDOWN signal from Resourcemanager ,Registration of NodeManager failed, Message from ResourceManager: NodeManager from hadoopmapr4.brewingintel.com doesn't satisfy minimum allocations, Sending SHUTDOWN signal to the NodeManager. at org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.serviceStart(NodeStatusUpdaterImpl.java:230) at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) at org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120) at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceStart(NodeManager.java:267) at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) at org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:477) at org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:524) Caused by: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Recieved SHUTDOWN signal from Resourcemanager ,Registration of NodeManager failed, Message from ResourceManager: NodeManager from hadoopmapr4.brewingintel.com doesn't satisfy minimum allocations, Sending SHUTDOWN signal to the NodeManager. at org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.registerWithRM(NodeStatusUpdaterImpl.java:298) at org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.serviceStart(NodeStatusUpdaterImpl.java:224) ... 6 more
Re: Challenges after MapR 5.1 Upgrade.
Hey John, I think the MapR guys will have some answers for you on the first stack trace, sounds like a missing jar on you class path. On the second I'm interested in knowing what MyriadScheduler your attempting to run, FAIR, CAPACITY, etc (Will be in your yarn-site.xml). Darin On Mon, Apr 4, 2016 at 3:29 PM, John Omernik wrote: > I had at one point Myriad working fine in MapR 5.0. I updated to 5.1, and > repackaged my hadoop tgz for remote distribution and now I have two > problems occurring. > > 1. At first when I had the mapr direct shuffle enabled per the > yarn-site.xml on the myriad documentaion, node managers would not start, > and would fail with the error below. > > 2. Once I removed the mapr shuffle from the yarn-site, I got node managers > started however, when I tried to launch a size 0, I got the other error > below. Not sure what's happening here. > > Any thoughts would be appreciated. Like I said, this was working with 5.0, > and now doesn't work in 5.1. > > Thanks! > > John > > SHuffle Error > > 16/04/04 13:46:34 INFO service.AbstractService: Service NodeManager failed > in state INITED; cause: java.lang.RuntimeException: No class defined for > mapr_direct_shuffle > java.lang.RuntimeException: No class defined for mapr_direct_shuffle > at > > org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices.serviceInit(AuxServices.java:139) > at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) > at > > org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107) > at > > org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.serviceInit(ContainerManagerImpl.java:250) > at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) > at > > org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107) > at > > org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceInit(NodeManager.java:256) > at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) > at > > org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:476) > at > > org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:524) > 16/04/04 13:46:34 INFO impl.MetricsSystemImpl: Stopping NodeManager metrics > system... > > > Zero Sized Node Manager Error: > > 16/04/04 14:22:49 INFO service.AbstractService: Service > org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl failed in > state STARTED; cause: > org.apache.hadoop.yarn.exceptions.YarnRuntimeException: > org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Recieved SHUTDOWN > signal from Resourcemanager ,Registration of NodeManager failed, Message > from ResourceManager: NodeManager from hadoopmapr4.brewingintel.com > doesn't satisfy minimum allocations, Sending SHUTDOWN signal to the > NodeManager. > org.apache.hadoop.yarn.exceptions.YarnRuntimeException: > org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Recieved SHUTDOWN > signal from Resourcemanager ,Registration of NodeManager failed, Message > from ResourceManager: NodeManager from hadoopmapr4.brewingintel.com > doesn't satisfy minimum allocations, Sending SHUTDOWN signal to the > NodeManager. > at > > org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.serviceStart(NodeStatusUpdaterImpl.java:230) > at > org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) > at > > org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120) > at > > org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceStart(NodeManager.java:267) > at > org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) > at > > org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:477) > at > > org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:524) > Caused by: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Recieved > SHUTDOWN signal from Resourcemanager ,Registration of NodeManager failed, > Message from ResourceManager: NodeManager from > hadoopmapr4.brewingintel.com > doesn't satisfy minimum allocations, Sending SHUTDOWN signal to the > NodeManager. > at > > org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.registerWithRM(NodeStatusUpdaterImpl.java:298) > at > > org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.serviceStart(NodeStatusUpdaterImpl.java:224) > ... 6 more >
Re: Challenges after MapR 5.1 Upgrade.
yarn.resourcemanager.scheduler.class org.apache.myriad.scheduler.yarn.MyriadFairScheduler On Mon, Apr 4, 2016 at 2:52 PM, Darin Johnson wrote: > Hey John, I think the MapR guys will have some answers for you on the first > stack trace, sounds like a missing jar on you class path. On the second > I'm interested in knowing what MyriadScheduler your attempting to run, > FAIR, CAPACITY, etc (Will be in your yarn-site.xml). > > Darin > > On Mon, Apr 4, 2016 at 3:29 PM, John Omernik wrote: > > > I had at one point Myriad working fine in MapR 5.0. I updated to 5.1, > and > > repackaged my hadoop tgz for remote distribution and now I have two > > problems occurring. > > > > 1. At first when I had the mapr direct shuffle enabled per the > > yarn-site.xml on the myriad documentaion, node managers would not start, > > and would fail with the error below. > > > > 2. Once I removed the mapr shuffle from the yarn-site, I got node > managers > > started however, when I tried to launch a size 0, I got the other error > > below. Not sure what's happening here. > > > > Any thoughts would be appreciated. Like I said, this was working with > 5.0, > > and now doesn't work in 5.1. > > > > Thanks! > > > > John > > > > SHuffle Error > > > > 16/04/04 13:46:34 INFO service.AbstractService: Service NodeManager > failed > > in state INITED; cause: java.lang.RuntimeException: No class defined for > > mapr_direct_shuffle > > java.lang.RuntimeException: No class defined for mapr_direct_shuffle > > at > > > > > org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices.serviceInit(AuxServices.java:139) > > at > org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) > > at > > > > > org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107) > > at > > > > > org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.serviceInit(ContainerManagerImpl.java:250) > > at > org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) > > at > > > > > org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107) > > at > > > > > org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceInit(NodeManager.java:256) > > at > org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) > > at > > > > > org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:476) > > at > > > > > org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:524) > > 16/04/04 13:46:34 INFO impl.MetricsSystemImpl: Stopping NodeManager > metrics > > system... > > > > > > Zero Sized Node Manager Error: > > > > 16/04/04 14:22:49 INFO service.AbstractService: Service > > org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl failed in > > state STARTED; cause: > > org.apache.hadoop.yarn.exceptions.YarnRuntimeException: > > org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Recieved SHUTDOWN > > signal from Resourcemanager ,Registration of NodeManager failed, Message > > from ResourceManager: NodeManager from hadoopmapr4.brewingintel.com > > doesn't satisfy minimum allocations, Sending SHUTDOWN signal to the > > NodeManager. > > org.apache.hadoop.yarn.exceptions.YarnRuntimeException: > > org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Recieved SHUTDOWN > > signal from Resourcemanager ,Registration of NodeManager failed, Message > > from ResourceManager: NodeManager from hadoopmapr4.brewingintel.com > > doesn't satisfy minimum allocations, Sending SHUTDOWN signal to the > > NodeManager. > > at > > > > > org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.serviceStart(NodeStatusUpdaterImpl.java:230) > > at > > org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) > > at > > > > > org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120) > > at > > > > > org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceStart(NodeManager.java:267) > > at > > org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) > > at > > > > > org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:477) > > at > > > > > org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:524) > > Caused by: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: > Recieved > > SHUTDOWN signal from Resourcemanager ,Registration of NodeManager failed, > > Message from ResourceManager: NodeManager from > > hadoopmapr4.brewingintel.com > > doesn't satisfy minimum allocations, Sending SHUTDOWN signal to the > > NodeManager. > > at > > > > > org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.registerWithRM(NodeStatusUpdaterImpl.java:298) > > at > > > > > org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.serviceStart(NodeStatusUpdaterImpl.java:224) > > ... 6 more > > >
Re: Challenges after MapR 5.1 Upgrade.
So prior to the upgrade, I had this code block commented out, however, it doesn't work without the comments either: yarn.scheduler.minimum-allocation-vcores 0 yarn.scheduler.minimum-allocation-vcores 0 On Mon, Apr 4, 2016 at 3:04 PM, John Omernik wrote: > > > yarn.resourcemanager.scheduler.class > > org.apache.myriad.scheduler.yarn.MyriadFairScheduler > > > > > On Mon, Apr 4, 2016 at 2:52 PM, Darin Johnson > wrote: > >> Hey John, I think the MapR guys will have some answers for you on the >> first >> stack trace, sounds like a missing jar on you class path. On the second >> I'm interested in knowing what MyriadScheduler your attempting to run, >> FAIR, CAPACITY, etc (Will be in your yarn-site.xml). >> >> Darin >> >> On Mon, Apr 4, 2016 at 3:29 PM, John Omernik wrote: >> >> > I had at one point Myriad working fine in MapR 5.0. I updated to 5.1, >> and >> > repackaged my hadoop tgz for remote distribution and now I have two >> > problems occurring. >> > >> > 1. At first when I had the mapr direct shuffle enabled per the >> > yarn-site.xml on the myriad documentaion, node managers would not start, >> > and would fail with the error below. >> > >> > 2. Once I removed the mapr shuffle from the yarn-site, I got node >> managers >> > started however, when I tried to launch a size 0, I got the other error >> > below. Not sure what's happening here. >> > >> > Any thoughts would be appreciated. Like I said, this was working with >> 5.0, >> > and now doesn't work in 5.1. >> > >> > Thanks! >> > >> > John >> > >> > SHuffle Error >> > >> > 16/04/04 13:46:34 INFO service.AbstractService: Service NodeManager >> failed >> > in state INITED; cause: java.lang.RuntimeException: No class defined for >> > mapr_direct_shuffle >> > java.lang.RuntimeException: No class defined for mapr_direct_shuffle >> > at >> > >> > >> org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices.serviceInit(AuxServices.java:139) >> > at >> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) >> > at >> > >> > >> org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107) >> > at >> > >> > >> org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.serviceInit(ContainerManagerImpl.java:250) >> > at >> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) >> > at >> > >> > >> org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107) >> > at >> > >> > >> org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceInit(NodeManager.java:256) >> > at >> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) >> > at >> > >> > >> org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:476) >> > at >> > >> > >> org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:524) >> > 16/04/04 13:46:34 INFO impl.MetricsSystemImpl: Stopping NodeManager >> metrics >> > system... >> > >> > >> > Zero Sized Node Manager Error: >> > >> > 16/04/04 14:22:49 INFO service.AbstractService: Service >> > org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl failed >> in >> > state STARTED; cause: >> > org.apache.hadoop.yarn.exceptions.YarnRuntimeException: >> > org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Recieved >> SHUTDOWN >> > signal from Resourcemanager ,Registration of NodeManager failed, Message >> > from ResourceManager: NodeManager from hadoopmapr4.brewingintel.com >> > doesn't satisfy minimum allocations, Sending SHUTDOWN signal to the >> > NodeManager. >> > org.apache.hadoop.yarn.exceptions.YarnRuntimeException: >> > org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Recieved >> SHUTDOWN >> > signal from Resourcemanager ,Registration of NodeManager failed, Message >> > from ResourceManager: NodeManager from hadoopmapr4.brewingintel.com >> > doesn't satisfy minimum allocations, Sending SHUTDOWN signal to the >> > NodeManager. >> > at >> > >> > >> org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.serviceStart(NodeStatusUpdaterImpl.java:230) >> > at >> > >> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) >> > at >> > >> > >> org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120) >> > at >> > >> > >> org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceStart(NodeManager.java:267) >> > at >> > >> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) >> > at >> > >> > >> org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:477) >> > at >> > >> > >> org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:524) >> > Caused by: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: >> Recieved >> > SHUTDOWN signal from Resourcemanager ,Registration of NodeManager >> failed, >> > Message from ResourceManager: Nod
Re: Challenges after MapR 5.1 Upgrade.
Hello John, Did you upgrade to 5.1 or installed new one? Feels like MapR default properties were not loaded - I need to poke around and then I will ask you for additional info Thanks,Yuliya From: John Omernik To: dev@myriad.incubator.apache.org Sent: Monday, April 4, 2016 12:29 PM Subject: Challenges after MapR 5.1 Upgrade. I had at one point Myriad working fine in MapR 5.0. I updated to 5.1, and repackaged my hadoop tgz for remote distribution and now I have two problems occurring. 1. At first when I had the mapr direct shuffle enabled per the yarn-site.xml on the myriad documentaion, node managers would not start, and would fail with the error below. 2. Once I removed the mapr shuffle from the yarn-site, I got node managers started however, when I tried to launch a size 0, I got the other error below. Not sure what's happening here. Any thoughts would be appreciated. Like I said, this was working with 5.0, and now doesn't work in 5.1. Thanks! John SHuffle Error 16/04/04 13:46:34 INFO service.AbstractService: Service NodeManager failed in state INITED; cause: java.lang.RuntimeException: No class defined for mapr_direct_shuffle java.lang.RuntimeException: No class defined for mapr_direct_shuffle at org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices.serviceInit(AuxServices.java:139) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107) at org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.serviceInit(ContainerManagerImpl.java:250) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107) at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceInit(NodeManager.java:256) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:476) at org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:524) 16/04/04 13:46:34 INFO impl.MetricsSystemImpl: Stopping NodeManager metrics system... Zero Sized Node Manager Error: 16/04/04 14:22:49 INFO service.AbstractService: Service org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl failed in state STARTED; cause: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Recieved SHUTDOWN signal from Resourcemanager ,Registration of NodeManager failed, Message from ResourceManager: NodeManager from hadoopmapr4.brewingintel.com doesn't satisfy minimum allocations, Sending SHUTDOWN signal to the NodeManager. org.apache.hadoop.yarn.exceptions.YarnRuntimeException: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Recieved SHUTDOWN signal from Resourcemanager ,Registration of NodeManager failed, Message from ResourceManager: NodeManager from hadoopmapr4.brewingintel.com doesn't satisfy minimum allocations, Sending SHUTDOWN signal to the NodeManager. at org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.serviceStart(NodeStatusUpdaterImpl.java:230) at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) at org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120) at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceStart(NodeManager.java:267) at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) at org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:477) at org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:524) Caused by: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Recieved SHUTDOWN signal from Resourcemanager ,Registration of NodeManager failed, Message from ResourceManager: NodeManager from hadoopmapr4.brewingintel.com doesn't satisfy minimum allocations, Sending SHUTDOWN signal to the NodeManager. at org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.registerWithRM(NodeStatusUpdaterImpl.java:298) at org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.serviceStart(NodeStatusUpdaterImpl.java:224) ... 6 more
Re: Challenges after MapR 5.1 Upgrade.
YarnDefaultProperties.java that defines class for mapr_direct_shuffle should be there even in 5.0, so nothing new there even if maprfs jar is outdated - could you also check that? Also could you paste content of your yarn-site.xml here? Thanks,Yuliya From: yuliya Feldman To: "dev@myriad.incubator.apache.org" Sent: Monday, April 4, 2016 1:43 PM Subject: Re: Challenges after MapR 5.1 Upgrade. Hello John, Did you upgrade to 5.1 or installed new one? Feels like MapR default properties were not loaded - I need to poke around and then I will ask you for additional info Thanks,Yuliya From: John Omernik To: dev@myriad.incubator.apache.org Sent: Monday, April 4, 2016 12:29 PM Subject: Challenges after MapR 5.1 Upgrade. I had at one point Myriad working fine in MapR 5.0. I updated to 5.1, and repackaged my hadoop tgz for remote distribution and now I have two problems occurring. 1. At first when I had the mapr direct shuffle enabled per the yarn-site.xml on the myriad documentaion, node managers would not start, and would fail with the error below. 2. Once I removed the mapr shuffle from the yarn-site, I got node managers started however, when I tried to launch a size 0, I got the other error below. Not sure what's happening here. Any thoughts would be appreciated. Like I said, this was working with 5.0, and now doesn't work in 5.1. Thanks! John SHuffle Error 16/04/04 13:46:34 INFO service.AbstractService: Service NodeManager failed in state INITED; cause: java.lang.RuntimeException: No class defined for mapr_direct_shuffle java.lang.RuntimeException: No class defined for mapr_direct_shuffle at org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices.serviceInit(AuxServices.java:139) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107) at org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.serviceInit(ContainerManagerImpl.java:250) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107) at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceInit(NodeManager.java:256) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:476) at org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:524) 16/04/04 13:46:34 INFO impl.MetricsSystemImpl: Stopping NodeManager metrics system... Zero Sized Node Manager Error: 16/04/04 14:22:49 INFO service.AbstractService: Service org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl failed in state STARTED; cause: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Recieved SHUTDOWN signal from Resourcemanager ,Registration of NodeManager failed, Message from ResourceManager: NodeManager from hadoopmapr4.brewingintel.com doesn't satisfy minimum allocations, Sending SHUTDOWN signal to the NodeManager. org.apache.hadoop.yarn.exceptions.YarnRuntimeException: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Recieved SHUTDOWN signal from Resourcemanager ,Registration of NodeManager failed, Message from ResourceManager: NodeManager from hadoopmapr4.brewingintel.com doesn't satisfy minimum allocations, Sending SHUTDOWN signal to the NodeManager. at org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.serviceStart(NodeStatusUpdaterImpl.java:230) at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) at org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120) at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceStart(NodeManager.java:267) at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) at org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:477) at org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:524) Caused by: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Recieved SHUTDOWN signal from Resourcemanager ,Registration of NodeManager failed, Message from ResourceManager: NodeManager from hadoopmapr4.brewingintel.com doesn't satisfy minimum allocations, Sending SHUTDOWN signal to the NodeManager. at org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.registerWithRM(NodeStatusUpdaterImpl.java:298) at org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.serviceStart(NodeStatusUpdaterImpl.java:224) ... 6 more
Re: Challenges after MapR 5.1 Upgrade.
This was a Upgrade from 5.0. I will post here, note: I have removed the
mapr_shuffle to get node managers to work, however, I am seeing other odd
things, so any help would be appreciated.
yarn.nodemanager.aux-services
mapreduce_shuffle,myriad_executor
yarn.resourcemanager.hostname
myriadprod.marathonprod.mesos
yarn.nodemanager.aux-services.mapreduce_shuffle.class
org.apache.hadoop.mapred.ShuffleHandler
yarn.nodemanager.aux-services.myriad_executor.class
org.apache.myriad.executor.MyriadExecutorAuxService
yarn.nm.liveness-monitor.expiry-interval-ms
2000
yarn.am.liveness-monitor.expiry-interval-ms
1
yarn.resourcemanager.nm.liveness-monitor.interval-ms
1000
yarn.nodemanager.resource.cpu-vcores
${nodemanager.resource.cpu-vcores}
yarn.nodemanager.resource.memory-mb
${nodemanager.resource.memory-mb}
yarn.scheduler.minimum-allocation-mb
512
yarn.scheduler.minimum-allocation-vcores
1
yarn.nodemanager.address
${myriad.yarn.nodemanager.address}
yarn.nodemanager.webapp.address
${myriad.yarn.nodemanager.webapp.address}
yarn.nodemanager.webapp.https.address
${myriad.yarn.nodemanager.webapp.address}
yarn.nodemanager.localizer.address
${myriad.yarn.nodemanager.localizer.address}
yarn.resourcemanager.scheduler.class
org.apache.myriad.scheduler.yarn.MyriadFairScheduler
yarn.scheduler.minimum-allocation-vcores
0
yarn.scheduler.minimum-allocation-vcores
0
Who will execute(launch) the containers.
yarn.nodemanager.container-executor.class
${yarn.nodemanager.container-executor.class}
The class which should help the LCE handle
resources.
yarn.nodemanager.linux-container-executor.resources-handler.class
${yarn.nodemanager.linux-container-executor.resources-handler.class}
yarn.nodemanager.linux-container-executor.cgroups.hierarchy
${yarn.nodemanager.linux-container-executor.cgroups.hierarchy}
yarn.nodemanager.linux-container-executor.cgroups.mount
${yarn.nodemanager.linux-container-executor.cgroups.mount}
yarn.nodemanager.linux-container-executor.cgroups.mount-path
${yarn.nodemanager.linux-container-executor.cgroups.mount-path}
yarn.nodemanager.linux-container-executor.group
${yarn.nodemanager.linux-container-executor.group}
yarn.nodemanager.linux-container-executor.path
${yarn.home}/bin/container-executor
yarn.http.policy
HTTP_ONLY
On Mon, Apr 4, 2016 at 3:53 PM, yuliya Feldman
wrote:
> YarnDefaultProperties.java that defines class for mapr_direct_shuffle
> should be there even in 5.0, so nothing new there even if maprfs jar is
> outdated - could you also check that?
> Also could you paste content of your yarn-site.xml here?
> Thanks,Yuliya
>
> From: yuliya Feldman
> To: "dev@myriad.incubator.apache.org"
> Sent: Monday, April 4, 2016 1:43 PM
> Subject: Re: Challenges after MapR 5.1 Upgrade.
>
> Hello John,
> Did you upgrade to 5.1 or installed new one?
> Feels like MapR default properties were not loaded - I need to poke around
> and then I will ask you for additional info
> Thanks,Yuliya
>
> From: John Omernik
> To: dev@myriad.incubator.apache.org
> Sent: Monday, April 4, 2016 12:29 PM
> Subject: Challenges after MapR 5.1 Upgrade.
>
> I had at one point Myriad working fine in MapR 5.0. I updated to 5.1, and
> repackaged my hadoop tgz for remote distribution and now I have two
> problems occurring.
>
> 1. At first when I had the mapr direct shuffle enabled per the
> yarn-site.xml on the myriad documentaion, node managers would not start,
> and would fail with the error below.
>
> 2. Once I removed the mapr shuffle from the yarn-site, I got node managers
> started however, when I tried to launch a size 0, I got the other error
> below. Not sure what's happening here.
>
> Any thoughts would be appreciated. Like I said, this was working with 5.0,
> and now doesn't work in 5.1.
>
> Thanks!
>
> John
>
> SHuffle Error
>
> 16/04/04 13:46:34 INFO service.AbstractService: Service NodeManager failed
> in state INITED; cause: java.lang.RuntimeException: No class defined for
> mapr_direct_shuffle
> java.lang.RuntimeException: No class defined for mapr_direct_shuffle
> at
>
> org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices.serviceInit(AuxServices.java:139)
>
Re: Challenges after MapR 5.1 Upgrade.
Hmmm...
You did not have to specify
yarn.nodemanager.aux-services.mapreduce_shuffle.class
org.apache.hadoop.mapred.ShuffleHandler
either
Can you check if maprfs*.jar is on the classpath (should be - otherwise you
would probably get more weird errors) and it has following class:
YarnDefaultProperties
Thanks,Yuliya
From: John Omernik
To: dev@myriad.incubator.apache.org; yuliya Feldman
Sent: Monday, April 4, 2016 2:18 PM
Subject: Re: Challenges after MapR 5.1 Upgrade.
This was a Upgrade from 5.0. I will post here, note: I have removed the
mapr_shuffle to get node managers to work, however, I am seeing other odd
things, so any help would be appreciated.
yarn.nodemanager.aux-services
mapreduce_shuffle,myriad_executor
yarn.resourcemanager.hostname
myriadprod.marathonprod.mesos
yarn.nodemanager.aux-services.mapreduce_shuffle.class
org.apache.hadoop.mapred.ShuffleHandler
yarn.nodemanager.aux-services.myriad_executor.class
org.apache.myriad.executor.MyriadExecutorAuxService
yarn.nm.liveness-monitor.expiry-interval-ms
2000
yarn.am.liveness-monitor.expiry-interval-ms
1
yarn.resourcemanager.nm.liveness-monitor.interval-ms
1000
yarn.nodemanager.resource.cpu-vcores
${nodemanager.resource.cpu-vcores}
yarn.nodemanager.resource.memory-mb
${nodemanager.resource.memory-mb}
yarn.scheduler.minimum-allocation-mb
512
yarn.scheduler.minimum-allocation-vcores
1
yarn.nodemanager.address
${myriad.yarn.nodemanager.address}
yarn.nodemanager.webapp.address
${myriad.yarn.nodemanager.webapp.address}
yarn.nodemanager.webapp.https.address
${myriad.yarn.nodemanager.webapp.address}
yarn.nodemanager.localizer.address
${myriad.yarn.nodemanager.localizer.address}
yarn.resourcemanager.scheduler.class
org.apache.myriad.scheduler.yarn.MyriadFairScheduler
yarn.scheduler.minimum-allocation-vcores
0
yarn.scheduler.minimum-allocation-vcores
0
Who will execute(launch) the containers.
yarn.nodemanager.container-executor.class
${yarn.nodemanager.container-executor.class}
The class which should help the LCE handle
resources.
yarn.nodemanager.linux-container-executor.resources-handler.class
${yarn.nodemanager.linux-container-executor.resources-handler.class}
yarn.nodemanager.linux-container-executor.cgroups.hierarchy
${yarn.nodemanager.linux-container-executor.cgroups.hierarchy}
yarn.nodemanager.linux-container-executor.cgroups.mount
${yarn.nodemanager.linux-container-executor.cgroups.mount}
yarn.nodemanager.linux-container-executor.cgroups.mount-path
${yarn.nodemanager.linux-container-executor.cgroups.mount-path}
yarn.nodemanager.linux-container-executor.group
${yarn.nodemanager.linux-container-executor.group}
yarn.nodemanager.linux-container-executor.path
${yarn.home}/bin/container-executor
yarn.http.policy
HTTP_ONLY
On Mon, Apr 4, 2016 at 3:53 PM, yuliya Feldman
wrote:
> YarnDefaultProperties.java that defines class for mapr_direct_shuffle
> should be there even in 5.0, so nothing new there even if maprfs jar is
> outdated - could you also check that?
> Also could you paste content of your yarn-site.xml here?
> Thanks,Yuliya
>
> From: yuliya Feldman
> To: "dev@myriad.incubator.apache.org"
> Sent: Monday, April 4, 2016 1:43 PM
> Subject: Re: Challenges after MapR 5.1 Upgrade.
>
> Hello John,
> Did you upgrade to 5.1 or installed new one?
> Feels like MapR default properties were not loaded - I need to poke around
> and then I will ask you for additional info
> Thanks,Yuliya
>
> From: John Omernik
> To: dev@myriad.incubator.apache.org
> Sent: Monday, April 4, 2016 12:29 PM
> Subject: Challenges after MapR 5.1 Upgrade.
>
> I had at one point Myriad working fine in MapR 5.0. I updated to 5.1, and
> repackaged my hadoop tgz for remote distribution and now I have two
> problems occurring.
>
> 1. At first when I had the mapr direct shuffle enabled per the
> yarn-site.xml on the myriad documentaion, node managers would not start,
> and would fail with the error below.
>
> 2. Once I removed the mapr shuffle from the yarn-site, I got node managers
> started however, when I tried to launch a size 0, I got the other error
> below. Not sure what's happening here.
>
> Any thoughts would be appreciated. Like I said, this was working with 5.0,
Re: Challenges after MapR 5.1 Upgrade.
Hey John,
I noticed these lines in your yarn-site.xml:
yarn.scheduler.minimum-allocation-mb
512
yarn.scheduler.minimum-allocation-vcores
1
If your attempting to launch a zero resource nodemanager for fgs that will
result in the first stack trace. Both should be explicitly 0 for that
feature to work (defaults are 1024 and 1 resp, which will fail). You do
have them set below to 0, however I'm in certain which would take
precedence.
On Apr 4, 2016 5:19 PM, "John Omernik" wrote:
> This was a Upgrade from 5.0. I will post here, note: I have removed the
> mapr_shuffle to get node managers to work, however, I am seeing other odd
> things, so any help would be appreciated.
>
>
>
>
>
>
> yarn.nodemanager.aux-services
> mapreduce_shuffle,myriad_executor
>
>
>
> yarn.resourcemanager.hostname
> myriadprod.marathonprod.mesos
>
>
> yarn.nodemanager.aux-services.mapreduce_shuffle.class
> org.apache.hadoop.mapred.ShuffleHandler
>
>
> yarn.nodemanager.aux-services.myriad_executor.class
> org.apache.myriad.executor.MyriadExecutorAuxService
>
>
> yarn.nm.liveness-monitor.expiry-interval-ms
> 2000
>
>
> yarn.am.liveness-monitor.expiry-interval-ms
> 1
>
>
> yarn.resourcemanager.nm.liveness-monitor.interval-ms
> 1000
>
>
>
> yarn.nodemanager.resource.cpu-vcores
> ${nodemanager.resource.cpu-vcores}
>
>
> yarn.nodemanager.resource.memory-mb
> ${nodemanager.resource.memory-mb}
>
>
>
>
> yarn.scheduler.minimum-allocation-mb
> 512
>
>
>
> yarn.scheduler.minimum-allocation-vcores
> 1
>
>
>
>
>
>
> yarn.nodemanager.address
> ${myriad.yarn.nodemanager.address}
>
>
> yarn.nodemanager.webapp.address
> ${myriad.yarn.nodemanager.webapp.address}
>
>
> yarn.nodemanager.webapp.https.address
> ${myriad.yarn.nodemanager.webapp.address}
>
>
> yarn.nodemanager.localizer.address
> ${myriad.yarn.nodemanager.localizer.address}
>
>
>
>
> yarn.resourcemanager.scheduler.class
> org.apache.myriad.scheduler.yarn.MyriadFairScheduler
>
>
>
>
> yarn.scheduler.minimum-allocation-vcores
> 0
>
>
> yarn.scheduler.minimum-allocation-vcores
> 0
>
>
>
> Who will execute(launch) the containers.
> yarn.nodemanager.container-executor.class
> ${yarn.nodemanager.container-executor.class}
>
>
> The class which should help the LCE handle
> resources.
>
>
> yarn.nodemanager.linux-container-executor.resources-handler.class
>
>
> ${yarn.nodemanager.linux-container-executor.resources-handler.class}
>
>
>
> yarn.nodemanager.linux-container-executor.cgroups.hierarchy
>
>
> ${yarn.nodemanager.linux-container-executor.cgroups.hierarchy}
>
>
>
> yarn.nodemanager.linux-container-executor.cgroups.mount
>
> ${yarn.nodemanager.linux-container-executor.cgroups.mount}
>
>
>
> yarn.nodemanager.linux-container-executor.cgroups.mount-path
>
>
> ${yarn.nodemanager.linux-container-executor.cgroups.mount-path}
>
>
> yarn.nodemanager.linux-container-executor.group
> ${yarn.nodemanager.linux-container-executor.group}
>
>
> yarn.nodemanager.linux-container-executor.path
> ${yarn.home}/bin/container-executor
>
>
> yarn.http.policy
> HTTP_ONLY
>
>
>
> On Mon, Apr 4, 2016 at 3:53 PM, yuliya Feldman >
> wrote:
>
> > YarnDefaultProperties.java that defines class for mapr_direct_shuffle
> > should be there even in 5.0, so nothing new there even if maprfs jar is
> > outdated - could you also check that?
> > Also could you paste content of your yarn-site.xml here?
> > Thanks,Yuliya
> >
> > From: yuliya Feldman
> > To: "dev@myriad.incubator.apache.org"
> > Sent: Monday, April 4, 2016 1:43 PM
> > Subject: Re: Challenges after MapR 5.1 Upgrade.
> >
> > Hello John,
> > Did you upgrade to 5.1 or installed new one?
> > Feels like MapR default properties were not loaded - I need to poke
> around
> > and then I will ask you for additional info
> > Thanks,Yuliya
Re: Challenges after MapR 5.1 Upgrade.
Thanks Darin, and thank you to Yuliya as well. Darin, your fix worked,
Yuliya helped me troubleshoot what appears to be a failed MapR 5.1
upgrade. Basically many of the libs that MapR links into the hadoop
classpath didn't get linked. I reinstalled hadoop components and all seems
well now.
Thanks!
John
On Mon, Apr 4, 2016 at 7:57 PM, Darin Johnson
wrote:
> Hey John,
>
> I noticed these lines in your yarn-site.xml:
>
>
> yarn.scheduler.minimum-allocation-mb
> 512
>
>
>
>
> yarn.scheduler.minimum-allocation-vcores
>
> 1
>
>
> If your attempting to launch a zero resource nodemanager for fgs that will
> result in the first stack trace. Both should be explicitly 0 for that
> feature to work (defaults are 1024 and 1 resp, which will fail). You do
> have them set below to 0, however I'm in certain which would take
> precedence.
> On Apr 4, 2016 5:19 PM, "John Omernik" wrote:
>
> > This was a Upgrade from 5.0. I will post here, note: I have removed the
> > mapr_shuffle to get node managers to work, however, I am seeing other odd
> > things, so any help would be appreciated.
> >
> >
> >
> >
> >
> >
> > yarn.nodemanager.aux-services
> > mapreduce_shuffle,myriad_executor
> >
> >
> >
> > yarn.resourcemanager.hostname
> > myriadprod.marathonprod.mesos
> >
> >
> >
> yarn.nodemanager.aux-services.mapreduce_shuffle.class
> > org.apache.hadoop.mapred.ShuffleHandler
> >
> >
> > yarn.nodemanager.aux-services.myriad_executor.class
> >
> org.apache.myriad.executor.MyriadExecutorAuxService
> >
> >
> > yarn.nm.liveness-monitor.expiry-interval-ms
> > 2000
> >
> >
> > yarn.am.liveness-monitor.expiry-interval-ms
> > 1
> >
> >
> > yarn.resourcemanager.nm.liveness-monitor.interval-ms
> > 1000
> >
> >
> >
> > yarn.nodemanager.resource.cpu-vcores
> > ${nodemanager.resource.cpu-vcores}
> >
> >
> > yarn.nodemanager.resource.memory-mb
> > ${nodemanager.resource.memory-mb}
> >
> >
> >
> >
> > yarn.scheduler.minimum-allocation-mb
> > 512
> >
> >
> >
> > yarn.scheduler.minimum-allocation-vcores
> > 1
> >
> >
> >
> >
> >
> >
> > yarn.nodemanager.address
> > ${myriad.yarn.nodemanager.address}
> >
> >
> > yarn.nodemanager.webapp.address
> > ${myriad.yarn.nodemanager.webapp.address}
> >
> >
> > yarn.nodemanager.webapp.https.address
> > ${myriad.yarn.nodemanager.webapp.address}
> >
> >
> > yarn.nodemanager.localizer.address
> > ${myriad.yarn.nodemanager.localizer.address}
> >
> >
> >
> >
> > yarn.resourcemanager.scheduler.class
> >
> org.apache.myriad.scheduler.yarn.MyriadFairScheduler
> >
> >
> >
> >
> > yarn.scheduler.minimum-allocation-vcores
> > 0
> >
> >
> > yarn.scheduler.minimum-allocation-vcores
> > 0
> >
> >
> >
> > Who will execute(launch) the
> containers.
> > yarn.nodemanager.container-executor.class
> > ${yarn.nodemanager.container-executor.class}
> >
> >
> > The class which should help the LCE handle
> > resources.
> >
> >
> >
> yarn.nodemanager.linux-container-executor.resources-handler.class
> >
> >
> >
> ${yarn.nodemanager.linux-container-executor.resources-handler.class}
> >
> >
> >
> > yarn.nodemanager.linux-container-executor.cgroups.hierarchy
> >
> >
> >
> ${yarn.nodemanager.linux-container-executor.cgroups.hierarchy}
> >
> >
> >
> > yarn.nodemanager.linux-container-executor.cgroups.mount
> >
> > ${yarn.nodemanager.linux-container-executor.cgroups.mount}
> >
> >
> >
> > yarn.nodemanager.linux-container-executor.cgroups.mount-path
> >
> >
> >
> ${yarn.nodemanager.linux-container-executor.cgroups.mount-path}
> >
> >
> > yarn.nodemanage
Re: Challenges after MapR 5.1 Upgrade.
t;
> yarn.nodemanager.linux-container-executor.resources-handler.class
> > >
> > >
> > >
> >
> ${yarn.nodemanager.linux-container-executor.resources-handler.class}
> > >
> > >
> > >
> > >
> yarn.nodemanager.linux-container-executor.cgroups.hierarchy
> > >
> > >
> > >
> >
> ${yarn.nodemanager.linux-container-executor.cgroups.hierarchy}
> > >
> > >
> > >
> > > yarn.nodemanager.linux-container-executor.cgroups.mount
> > >
> > >
> ${yarn.nodemanager.linux-container-executor.cgroups.mount}
> > >
> > >
> > >
> > >
> yarn.nodemanager.linux-container-executor.cgroups.mount-path
> > >
> > >
> > >
> >
> ${yarn.nodemanager.linux-container-executor.cgroups.mount-path}
> > >
> > >
> > > yarn.nodemanager.linux-container-executor.group
> > >
> ${yarn.nodemanager.linux-container-executor.group}
> > >
> > >
> > > yarn.nodemanager.linux-container-executor.path
> > > ${yarn.home}/bin/container-executor
> > >
> > >
> > > yarn.http.policy
> > > HTTP_ONLY
> > >
> > >
> > >
> > > On Mon, Apr 4, 2016 at 3:53 PM, yuliya Feldman
> > > > >
> > > wrote:
> > >
> > > > YarnDefaultProperties.java that defines class for mapr_direct_shuffle
> > > > should be there even in 5.0, so nothing new there even if maprfs jar
> is
> > > > outdated - could you also check that?
> > > > Also could you paste content of your yarn-site.xml here?
> > > > Thanks,Yuliya
> > > >
> > > > From: yuliya Feldman
> > > > To: "dev@myriad.incubator.apache.org" <
> > dev@myriad.incubator.apache.org>
> > > > Sent: Monday, April 4, 2016 1:43 PM
> > > > Subject: Re: Challenges after MapR 5.1 Upgrade.
> > > >
> > > > Hello John,
> > > > Did you upgrade to 5.1 or installed new one?
> > > > Feels like MapR default properties were not loaded - I need to poke
> > > around
> > > > and then I will ask you for additional info
> > > > Thanks,Yuliya
> > > >
> > > > From: John Omernik
> > > > To: dev@myriad.incubator.apache.org
> > > > Sent: Monday, April 4, 2016 12:29 PM
> > > > Subject: Challenges after MapR 5.1 Upgrade.
> > > >
> > > > I had at one point Myriad working fine in MapR 5.0. I updated to
> 5.1,
> > > and
> > > > repackaged my hadoop tgz for remote distribution and now I have two
> > > > problems occurring.
> > > >
> > > > 1. At first when I had the mapr direct shuffle enabled per the
> > > > yarn-site.xml on the myriad documentaion, node managers would not
> > start,
> > > > and would fail with the error below.
> > > >
> > > > 2. Once I removed the mapr shuffle from the yarn-site, I got node
> > > managers
> > > > started however, when I tried to launch a size 0, I got the other
> error
> > > > below. Not sure what's happening here.
> > > >
> > > > Any thoughts would be appreciated. Like I said, this was working with
> > > 5.0,
> > > > and now doesn't work in 5.1.
> > > >
> > > > Thanks!
> > > >
> > > > John
> > > >
> > > > SHuffle Error
> > > >
> > > > 16/04/04 13:46:34 INFO service.AbstractService: Service NodeManager
> > > failed
> > > > in state INITED; cause: java.lang.RuntimeException: No class defined
> > for
> > > > mapr_direct_shuffle
> > > > java.lang.RuntimeException: No class defined for mapr_direct_shuffle
> > > > at
> > > >
> > > >
> > >
> >
> org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices.serviceInit(AuxServices.java:139)
> > > > at
> > >
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> > > > at
> > > >
> > > >
> > >
> >
> org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:107)
> > > > at
> > > >
> > > >
> > >
> >
> org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.serviceIn
