Hey Worthy, Awesome! Let use know if you need anything else. :)
Cheers, Chris On 10/21/13 7:03 AM, "Worthy LaFollette" <[email protected]> wrote: >For closure, I got it to work fine. The trick was to implement the >suggestion of the JIRA mentioned upstream for OS/X > >Thanks for your help! > >W- > > >On Tue, Oct 15, 2013 at 11:06 PM, Chris Riccomini ><[email protected]>wrote: > >> Hey Worthy, >> >> Awesome! >> >> That zookeeper.connect issue you see should be fixed now -- it's the bug >> that I was talking about yesterday. If you re-checkout or git pull the >> latest code from hello-samza on Github, that should work. >> >> Cheers, >> Chris >> >> On 10/15/13 9:03 PM, "Worthy LaFollette" <[email protected]> wrote: >> >> >Those were all there were in the logs. >> > >> >Found part of the issue: >> > >> >https://issues.apache.org/jira/browse/HADOOP-7489 >> > >> >I set the recommended env variable and got much farther. The job ran >>for >> >a >> >bit, but then failed with the same error. Will try again, but set the >> >value in the hadoop script directly. >> > >> >***** >> > >> >2013-10-15 20:58:17.882 java[14099:111b] Unable to load realm info >> >from SCDynamicStore >> >Exception in thread "main" java.lang.IllegalArgumentException: >> >requirement failed: Missing required property 'zookeeper.connect' >> > at scala.Predef$.require(Predef.scala:145) >> > at >> >>>kafka.utils.VerifiableProperties.getString(VerifiableProperties.scala:17 >>>5) >> > at kafka.utils.ZKConfig.<init>(ZkUtils.scala:775) >> > at kafka.consumer.ConsumerConfig.<init>(ConsumerConfig.scala:73) >> > at kafka.consumer.ConsumerConfig.<init>(ConsumerConfig.scala:77) >> > at >> >>>org.apache.samza.config.KafkaConfig.getKafkaSystemConsumerConfig(KafkaCo >>>nf >> >ig.scala:91) >> > at >> >>>org.apache.samza.system.kafka.KafkaSystemFactory.getAdmin(KafkaSystemFac >>>to >> >ry.scala:106) >> > at >> >>>org.apache.samza.system.kafka.KafkaSystemFactory.getAdmin(KafkaSystemFac >>>to >> >ry.scala:41) >> > at org.apache.samza.util.Util$$anonfun$1.apply(Util.scala:93) >> > at org.apache.samza.util.Util$$anonfun$1.apply(Util.scala:88) >> > at >> >>>scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.sc >>>al >> >a:206) >> > at >> >>>scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.sc >>>al >> >a:206) >> > at scala.collection.immutable.Set$Set2.foreach(Set.scala:101) >> > at >> scala.collection.TraversableLike$class.map(TraversableLike.scala:206) >> > at scala.collection.immutable.Set$Set2.map(Set.scala:87) >> > at >> org.apache.samza.util.Util$.getMaxInputStreamPartitions(Util.scala:88) >> > at >> >>>org.apache.samza.job.yarn.SamzaAppMasterTaskManager.<init>(SamzaAppMaste >>>rT >> >askManager.scala:92) >> > at >> >org.apache.samza.job.yarn.SamzaAppMaster$.main(SamzaAppMaster.scala:75) >> > at >> org.apache.samza.job.yarn.SamzaAppMaster.main(SamzaAppMaster.scala) >> > >> > >> > >> > >> >On Tue, Oct 15, 2013 at 8:37 PM, Chris Riccomini >> ><[email protected]>wrote: >> > >> >> Hey Worthy, >> >> >> >> That log looks truncated to me. Is that the complete file? If not, >> >>maybe >> >> you can throw it up on a Github gist or pastebin link? >> >> >> >> My log looks like this: >> >> >> >> 13/10/14 12:10:51 INFO nodemanager.NodeManager: STARTUP_MSG: >> >> /************************************************************ >> >> STARTUP_MSG: Starting NodeManager >> >> STARTUP_MSG: host = criccomi-mn/172.19.218.230 >> >> STARTUP_MSG: args = [] >> >> STARTUP_MSG: version = 2.0.5-alpha >> >> STARTUP_MSG: classpath = ...a very long class path... >> >> STARTUP_MSG: build = http://svn.apache.org/repos/asf/hadoop/common >>-r >> >> 1488459; compiled by 'jenkins' on 2013-06-01T04:05Z >> >> STARTUP_MSG: java = 1.6.0_51 >> >> ************************************************************/ >> >> 2013-10-14 12:10:52.433 java[64477:213] Unable to load realm mapping >> >>info >> >> from SCDynamicStore >> >> 13/10/14 12:10:52 WARN util.NativeCodeLoader: Unable to load >> >>native-hadoop >> >> library for your platform... using builtin-java classes where >>applicable >> >> 13/10/14 12:10:53 WARN nodemanager.ContainerExecutor: setsid is not >> >> available on this machine. So not using it. >> >> 13/10/14 12:10:53 INFO nodemanager.ContainerExecutor: setsid exited >>with >> >> exit code 0 >> >> 13/10/14 12:10:53 INFO event.AsyncDispatcher: Registering class >> >> >> >>>>org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Co >>>>nt >> >>ai >> >> nerEventType for class >> >> >> >>>>org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerMan >>>>ag >> >>er >> >> Impl$ContainerEventDispatcher >> >> 13/10/14 12:10:53 INFO event.AsyncDispatcher: Registering class >> >> >> >>>>org.apache.hadoop.yarn.server.nodemanager.containermanager.application. >>>>Ap >> >>pl >> >> icationEventType for class >> >> >> >>>>org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerMan >>>>ag >> >>er >> >> Impl$ApplicationEventDispatcher >> >> 13/10/14 12:10:53 INFO event.AsyncDispatcher: Registering class >> >> >> >>>>org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ev >>>>en >> >>t. >> >> LocalizationEventType for class >> >> >> >>>>org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.Re >>>>so >> >>ur >> >> ceLocalizationService >> >> 13/10/14 12:10:53 INFO event.AsyncDispatcher: Registering class >> >> >> >>>>org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServicesE >>>>ve >> >>nt >> >> Type for class >> >> >>org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices >> >> 13/10/14 12:10:53 INFO event.AsyncDispatcher: Registering class >> >> >> >>>>org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.Cont >>>>ai >> >>ne >> >> rsMonitorEventType for class >> >> >> >>>>org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.Cont >>>>ai >> >>ne >> >> rsMonitorImpl >> >> 13/10/14 12:10:53 INFO event.AsyncDispatcher: Registering class >> >> >> >>>>org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.Con >>>>ta >> >>in >> >> ersLauncherEventType for class >> >> >> >>>>org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.Con >>>>ta >> >>in >> >> ersLauncher >> >> 13/10/14 12:10:53 INFO event.AsyncDispatcher: Registering class >> >> org.apache.hadoop.yarn.server.nodemanager.ContainerManagerEventType >>for >> >> class >> >> >> >>>>org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerMan >>>>ag >> >>er >> >> Impl >> >> 13/10/14 12:10:53 INFO event.AsyncDispatcher: Registering class >> >> org.apache.hadoop.yarn.server.nodemanager.NodeManagerEventType for >>class >> >> org.apache.hadoop.yarn.server.nodemanager.NodeManager >> >> 13/10/14 12:10:53 INFO impl.MetricsConfig: loaded properties from >> >> hadoop-metrics2.properties >> >> 13/10/14 12:10:54 INFO impl.MetricsSystemImpl: Scheduled snapshot >>period >> >> at 10 second(s). >> >> 13/10/14 12:10:54 INFO impl.MetricsSystemImpl: NodeManager metrics >> >>system >> >> started >> >> 13/10/14 12:10:54 INFO service.AbstractService: >> >> Service:org.apache.hadoop.yarn.server.nodemanager.DeletionService is >> >> inited. >> >> 13/10/14 12:10:54 INFO service.AbstractService: >> >> >> >>>>Service:org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerServi >>>>ce >> >> is inited. >> >> 13/10/14 12:10:54 INFO service.AbstractService: >> >> >> >>>>Service:org.apache.hadoop.yarn.server.nodemanager.NodeHealthCheckerServ >>>>ic >> >>e >> >> is inited. >> >> 13/10/14 12:10:54 INFO service.AbstractService: >> >> >> >>>>Service:org.apache.hadoop.yarn.server.nodemanager.NodeResourceMonitorIm >>>>pl >> >> is inited. >> >> 13/10/14 12:10:54 INFO event.AsyncDispatcher: Registering class >> >> >> >>>>org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.e >>>>ve >> >>nt >> >> .LogHandlerEventType for class >> >> >> >>>>org.apache.hadoop.yarn.server.nodemanager.containermanager.loghandler.N >>>>on >> >>Ag >> >> gregatingLogHandler >> >> 13/10/14 12:10:54 INFO event.AsyncDispatcher: Registering class >> >> >> >>>>org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ev >>>>en >> >>t. >> >> LocalizerEventType for class >> >> >> >>>>org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.Re >>>>so >> >>ur >> >> ceLocalizationService$LocalizerTracker >> >> 13/10/14 12:10:54 INFO service.AbstractService: >> >> >> >>>>Service:org.apache.hadoop.yarn.server.nodemanager.containermanager.loca >>>>li >> >>ze >> >> r.ResourceLocalizationService$LocalizerTracker is inited. >> >> 13/10/14 12:10:54 INFO service.AbstractService: >> >> >> >>>>Service:org.apache.hadoop.yarn.server.nodemanager.containermanager.loca >>>>li >> >>ze >> >> r.ResourceLocalizationService is inited. >> >> 13/10/14 12:10:54 INFO service.AbstractService: >> >> Service:containers-launcher is inited. >> >> 13/10/14 12:10:54 INFO service.AbstractService: >> >> >> >>>>Service:org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxS >>>>er >> >>vi >> >> ces is inited. >> >> 13/10/14 12:10:54 INFO monitor.ContainersMonitorImpl: Using >> >> ResourceCalculatorPlugin : null >> >> 13/10/14 12:10:54 INFO monitor.ContainersMonitorImpl: Using >> >> ResourceCalculatorProcessTree : null >> >> 13/10/14 12:10:54 INFO monitor.ContainersMonitorImpl: Physical memory >> >> check enabled: true >> >> 13/10/14 12:10:54 INFO monitor.ContainersMonitorImpl: Virtual memory >> >>check >> >> enabled: true >> >> 13/10/14 12:10:54 INFO service.AbstractService: >> >>Service:containers-monitor >> >> is inited. >> >> 13/10/14 12:10:54 INFO service.AbstractService: Service:Dispatcher is >> >> inited. >> >> 13/10/14 12:10:54 INFO service.AbstractService: >> >> >> >>>>Service:org.apache.hadoop.yarn.server.nodemanager.containermanager.logh >>>>an >> >>dl >> >> er.NonAggregatingLogHandler is inited. >> >> 13/10/14 12:10:54 INFO service.AbstractService: >> >> >> >>>>Service:org.apache.hadoop.yarn.server.nodemanager.containermanager.Cont >>>>ai >> >>ne >> >> rManagerImpl is inited. >> >> 13/10/14 12:10:54 INFO service.AbstractService: >> >> Service:org.apache.hadoop.yarn.server.nodemanager.webapp.WebServer is >> >> inited. >> >> 13/10/14 12:10:54 INFO service.AbstractService: Service:Dispatcher is >> >> inited. >> >> 13/10/14 12:10:54 INFO nodemanager.NodeStatusUpdaterImpl: Initialized >> >> nodemanager for null: physical-memory=8192 virtual-memory=81920 >> >> physical-cores=8 virtual-cores=16 >> >> 13/10/14 12:10:54 INFO service.AbstractService: >> >> >>Service:org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl >> >>is >> >> inited. >> >> 13/10/14 12:10:54 INFO service.AbstractService: >> >> Service:org.apache.hadoop.yarn.server.nodemanager.NodeManager is >>inited. >> >> 13/10/14 12:10:54 INFO service.AbstractService: >> >> Service:org.apache.hadoop.yarn.server.nodemanager.DeletionService is >> >> started. >> >> 13/10/14 12:10:54 INFO service.AbstractService: >> >> >> >>>>Service:org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerServi >>>>ce >> >> is started. >> >> 13/10/14 12:10:54 INFO service.AbstractService: >> >> >> >>>>Service:org.apache.hadoop.yarn.server.nodemanager.NodeHealthCheckerServ >>>>ic >> >>e >> >> is started. >> >> 13/10/14 12:10:54 INFO service.AbstractService: >> >> >> >>>>Service:org.apache.hadoop.yarn.server.nodemanager.NodeResourceMonitorIm >>>>pl >> >> is started. >> >> 13/10/14 12:10:54 INFO ipc.Server: Starting Socket Reader #1 for port >> >>61403 >> >> 13/10/14 12:10:54 INFO pb.RpcServerFactoryPBImpl: Adding protocol >> >> org.apache.hadoop.yarn.api.ContainerManagerPB to the server >> >> 13/10/14 12:10:54 INFO ipc.Server: IPC Server Responder: starting >> >> 13/10/14 12:10:54 INFO ipc.Server: IPC Server listener on 61403: >> >>starting >> >> 13/10/14 12:10:54 INFO containermanager.ContainerManagerImpl: >> >> ContainerManager started at criccomi-mn/172.19.218.230:61403 >> >> 13/10/14 12:10:54 INFO ipc.Server: Starting Socket Reader #1 for port >> >>8040 >> >> 13/10/14 12:10:54 INFO pb.RpcServerFactoryPBImpl: Adding protocol >> >> org.apache.hadoop.yarn.server.nodemanager.api.LocalizationProtocolPB >>to >> >> the server >> >> 13/10/14 12:10:54 INFO ipc.Server: IPC Server Responder: starting >> >> 13/10/14 12:10:54 INFO ipc.Server: IPC Server listener on 8040: >>starting >> >> 13/10/14 12:10:54 INFO localizer.ResourceLocalizationService: >>Localizer >> >> started on port 8040 >> >> 13/10/14 12:10:54 INFO service.AbstractService: >> >> >> >>>>Service:org.apache.hadoop.yarn.server.nodemanager.containermanager.loca >>>>li >> >>ze >> >> r.ResourceLocalizationService$LocalizerTracker is started. >> >> 13/10/14 12:10:54 INFO service.AbstractService: >> >> >> >>>>Service:org.apache.hadoop.yarn.server.nodemanager.containermanager.loca >>>>li >> >>ze >> >> r.ResourceLocalizationService is started. >> >> 13/10/14 12:10:54 INFO service.AbstractService: >> >> Service:containers-launcher is started. >> >> 13/10/14 12:10:54 INFO service.AbstractService: >> >> >> >>>>Service:org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxS >>>>er >> >>vi >> >> ces is started. >> >> 13/10/14 12:10:54 INFO monitor.ContainersMonitorImpl: >> >> ResourceCalculatorPlugin is unavailable on this system. >> >> >> >>>>org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.Cont >>>>ai >> >>ne >> >> rsMonitorImpl is disabled. >> >> 13/10/14 12:10:54 INFO service.AbstractService: >> >>Service:containers-monitor >> >> is started. >> >> 13/10/14 12:10:54 INFO service.AbstractService: Service:Dispatcher is >> >> started. >> >> 13/10/14 12:10:54 INFO service.AbstractService: >> >> >> >>>>Service:org.apache.hadoop.yarn.server.nodemanager.containermanager.logh >>>>an >> >>dl >> >> er.NonAggregatingLogHandler is started. >> >> 13/10/14 12:10:54 INFO service.AbstractService: >> >> >> >>>>Service:org.apache.hadoop.yarn.server.nodemanager.containermanager.Cont >>>>ai >> >>ne >> >> rManagerImpl is started. >> >> 13/10/14 12:10:54 INFO webapp.WebServer: Instantiating NMWebApp at >> >> 0.0.0.0:8042 >> >> 13/10/14 12:10:54 INFO mortbay.log: Logging to >> >> org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via >> >> org.mortbay.log.Slf4jLog >> >> 13/10/14 12:10:55 INFO http.HttpServer: Added global filter 'safety' >> >> (class=org.apache.hadoop.http.HttpServer$QuotingInputFilter) >> >> 13/10/14 12:10:55 INFO http.HttpServer: Added filter >>static_user_filter >> >> >>(class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) >> >>to >> >> context node >> >> 13/10/14 12:10:55 INFO http.HttpServer: Added filter >>static_user_filter >> >> >>(class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) >> >>to >> >> context static >> >> 13/10/14 12:10:55 INFO http.HttpServer: Added filter >>static_user_filter >> >> >>(class=org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter) >> >>to >> >> context logs >> >> 13/10/14 12:10:55 INFO http.HttpServer: adding path spec: /node >> >> 13/10/14 12:10:55 INFO http.HttpServer: adding path spec: /node/* >> >> 13/10/14 12:10:55 INFO http.HttpServer: adding path spec: /ws >> >> 13/10/14 12:10:55 INFO http.HttpServer: adding path spec: /ws/* >> >> 13/10/14 12:10:55 INFO http.HttpServer: Added global filter 'guice' >> >> (class=com.google.inject.servlet.GuiceFilter) >> >> 13/10/14 12:10:55 INFO http.HttpServer: Jetty bound to port 8042 >> >> 13/10/14 12:10:55 INFO mortbay.log: jetty-6.1.26 >> >> 13/10/14 12:10:55 INFO mortbay.log: Extract >> >> >> >>>>jar:file:/Users/criccomi/Code/hello-samza/deploy/yarn/share/hadoop/yarn >>>>/h >> >>ad >> >> oop-yarn-common-2.0.5-alpha.jar!/webapps/node to >> >> >> >>>>/var/folders/jl/cxz2crt15cq_5jj9x0lnqwwh00033y/T/Jetty_0_0_0_0_8042_nod >>>>e_ >> >>__ >> >> _19tj0x/webapp >> >> 13/10/14 12:10:55 INFO mortbay.log: Started >> >> [email protected]:8042 >> >> 13/10/14 12:10:55 INFO webapp.WebApps: Web app /node started at 8042 >> >> 13/10/14 12:10:56 INFO webapp.WebApps: Registered webapp guice >>modules >> >> 13/10/14 12:10:56 INFO service.AbstractService: >> >> Service:org.apache.hadoop.yarn.server.nodemanager.webapp.WebServer is >> >> started. >> >> 13/10/14 12:10:56 INFO service.AbstractService: Service:Dispatcher is >> >> started. >> >> 13/10/14 12:10:56 INFO nodemanager.NodeStatusUpdaterImpl: Connecting >>to >> >> ResourceManager at /0.0.0.0:8031 >> >> 13/10/14 12:10:57 INFO ipc.Client: Retrying connect to server: >> >> 0.0.0.0/0.0.0.0:8031. Already tried 0 time(s); retry policy is >> >> RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 >>SECONDS) >> >> 13/10/14 12:10:57 INFO nodemanager.NodeStatusUpdaterImpl: Registered >> >>with >> >> ResourceManager as criccomi-mn:61403 with total resource of >> >><memory:8192, >> >> vCores:16> >> >> 13/10/14 12:10:57 INFO service.AbstractService: >> >> >>Service:org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl >> >>is >> >> started. >> >> 13/10/14 12:10:57 INFO service.AbstractService: >> >> Service:org.apache.hadoop.yarn.server.nodemanager.NodeManager is >> >>started. >> >> 13/10/14 12:11:09 INFO service.AbstractService: >> >> >>Service:org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl >> >>is >> >> stopped. >> >> 13/10/14 12:11:09 INFO service.AbstractService: Service:Dispatcher is >> >> stopped. >> >> 13/10/14 12:11:09 INFO mortbay.log: Stopped >> >> [email protected]:8042 >> >> 13/10/14 12:11:09 INFO service.AbstractService: >> >> Service:org.apache.hadoop.yarn.server.nodemanager.webapp.WebServer is >> >> stopped. >> >> 13/10/14 12:11:09 INFO ipc.Server: Stopping server on 61403 >> >> 13/10/14 12:11:09 INFO ipc.Server: Stopping IPC Server Responder >> >> 13/10/14 12:11:09 INFO ipc.Server: Stopping IPC Server listener on >>61403 >> >> 13/10/14 12:11:09 INFO service.AbstractService: >> >> >> >>>>Service:org.apache.hadoop.yarn.server.nodemanager.containermanager.logh >>>>an >> >>dl >> >> er.NonAggregatingLogHandler is stopped. >> >> 13/10/14 12:11:09 INFO service.AbstractService: Service:Dispatcher is >> >> stopped. >> >> 13/10/14 12:11:09 INFO monitor.ContainersMonitorImpl: >> >> ResourceCalculatorPlugin is unavailable on this system. >> >> >> >>>>org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.Cont >>>>ai >> >>ne >> >> rsMonitorImpl is disabled. >> >> 13/10/14 12:11:09 INFO service.AbstractService: >> >>Service:containers-monitor >> >> is stopped. >> >> 13/10/14 12:11:09 INFO service.AbstractService: >> >> >> >>>>Service:org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxS >>>>er >> >>vi >> >> ces is stopped. >> >> 13/10/14 12:11:09 INFO service.AbstractService: >> >> Service:containers-launcher is stopped. >> >> 13/10/14 12:11:09 INFO ipc.Server: Stopping server on 8040 >> >> 13/10/14 12:11:09 INFO ipc.Server: Stopping IPC Server listener on >>8040 >> >> 13/10/14 12:11:09 INFO ipc.Server: Stopping IPC Server Responder >> >> 13/10/14 12:11:09 INFO service.AbstractService: >> >> >> >>>>Service:org.apache.hadoop.yarn.server.nodemanager.containermanager.loca >>>>li >> >>ze >> >> r.ResourceLocalizationService$LocalizerTracker is stopped. >> >> 13/10/14 12:11:09 INFO service.AbstractService: >> >> >> >>>>Service:org.apache.hadoop.yarn.server.nodemanager.containermanager.loca >>>>li >> >>ze >> >> r.ResourceLocalizationService is stopped. >> >> 13/10/14 12:11:09 INFO localizer.ResourceLocalizationService: Public >> >>cache >> >> exiting >> >> 13/10/14 12:11:09 INFO service.AbstractService: >> >> >> >>>>Service:org.apache.hadoop.yarn.server.nodemanager.containermanager.Cont >>>>ai >> >>ne >> >> rManagerImpl is stopped. >> >> 13/10/14 12:11:09 INFO service.AbstractService: >> >> >> >>>>Service:org.apache.hadoop.yarn.server.nodemanager.NodeResourceMonitorIm >>>>pl >> >> is stopped. >> >> 13/10/14 12:11:09 INFO service.AbstractService: >> >> >> >>>>Service:org.apache.hadoop.yarn.server.nodemanager.LocalDirsHandlerServi >>>>ce >> >> is stopped. >> >> 13/10/14 12:11:09 INFO service.AbstractService: >> >> >> >>>>Service:org.apache.hadoop.yarn.server.nodemanager.NodeHealthCheckerServ >>>>ic >> >>e >> >> is stopped. >> >> 13/10/14 12:11:09 INFO service.AbstractService: >> >> Service:org.apache.hadoop.yarn.server.nodemanager.DeletionService is >> >> stopped. >> >> 13/10/14 12:11:09 INFO service.AbstractService: >> >> Service:org.apache.hadoop.yarn.server.nodemanager.NodeManager is >> >>stopped. >> >> 13/10/14 12:11:09 INFO impl.MetricsSystemImpl: Stopping NodeManager >> >> metrics system... >> >> 13/10/14 12:11:09 INFO impl.MetricsSystemImpl: NodeManager metrics >> >>system >> >> stopped. >> >> 13/10/14 12:11:09 INFO impl.MetricsSystemImpl: NodeManager metrics >> >>system >> >> shutdown complete. >> >> 13/10/14 12:11:09 INFO nodemanager.NodeManager: SHUTDOWN_MSG: >> >> /************************************************************ >> >> SHUTDOWN_MSG: Shutting down NodeManager at criccomi-mn/172.19.218.230 >> >> ************************************************************/ >> >> >> >> >> >> Cheers, >> >> Chris >> >> >> >> On 10/15/13 7:16 PM, "Worthy LaFollette" <[email protected]> wrote: >> >> >> >> >nm.log: >> >> >13/10/14 19:22:49 INFO nodemanager.NodeManager: STARTUP_MSG: >> >> >/************************************************************ >> >> >STARTUP_MSG: Starting NodeManager >> >> >STARTUP_MSG: host = Worthys-MacBook-Pro-2.local/10.252.27.133 >> >> >STARTUP_MSG: args = [] >> >> >STARTUP_MSG: version = 2.0.5-alpha >> >> > >> >> >rm.log: >> >> >13/10/14 19:22:49 INFO resourcemanager.ResourceManager: STARTUP_MSG: >> >> >/************************************************************ >> >> >STARTUP_MSG: Starting ResourceManager >> >> >STARTUP_MSG: host = Worthys-MacBook-Pro-2.local/10.252.27.133 >> >> >STARTUP_MSG: args = [] >> >> >STARTUP_MSG: version = 2.0.5-alpha >> >> > >> >> > >> >> > >> >> > >> >> >On Tue, Oct 15, 2013 at 9:34 AM, Chris Riccomini >> >> ><[email protected]>wrote: >> >> > >> >> >> Hey Worthy, >> >> >> >> >> >> What do the deploy/yarn/logs/rm.log and deploy/yarn/logs/nm.log >>files >> >> >>say? >> >> >> >> >> >> I'm a bit stumped. We might have to mail the YARN mailing list for >> >> >>help. I >> >> >> ran a clean checkout for hello-samza last night, and fixed one bug >> >> >>(which >> >> >> is now up on Github), but I don't believe it's related to your >>issue. >> >> >> Other than that one bug, I was able to startup and run >>successfully >> >> >> without seeing this problem. >> >> >> >> >> >> Cheers, >> >> >> Chris >> >> >> >> >> >> P.S. Bug fix was >> >> >> >> >> >> >> >> >> >> >> >>https://github.com/linkedin/hello-samza/commit/4d35f4e664a3afea75e7a399e6 >> >> >>5b >> >> >> 8308cc2ed8a6 >> >> >> >> >> >> On 10/14/13 10:36 PM, "Worthy LaFollette" <[email protected]> >>wrote: >> >> >> >> >> >> >I just ran the example as specified, no changes on what otherwise >> >> >>should >> >> >> >have been a clean machine (Mac OS - 10.8.5). >> >> >> > >> >> >> >1) No >> >> >> >2) Former - just bin/grid >> >> >> >3) No >> >> >> >4) No >> >> >> > >> >> >> >Also confirmed that no Network filters were in play to prevent >>the >> >> >> >connection. >> >> >> > >> >> >> > >> >> >> >On Mon, Oct 14, 2013 at 8:19 PM, Chris Riccomini >> >> >> ><[email protected]>wrote: >> >> >> > >> >> >> >> Hey Worthy, >> >> >> >> >> >> >> >> Hmm. That's exciting. A few questions: >> >> >> >> >> >> >> >> 1. Do you have kerberos setup on your machine? Running `klist` >> >>should >> >> >> >>give >> >> >> >> you a list of Kerberos credentials. >> >> >> >> 2. Are you simply running the bin/grid script, or are you >>trying >> >>to >> >> >>run >> >> >> >> Samza jobs on a "real" YARN cluster? >> >> >> >> 3. Did you change the yarn-site.xml configuration at all? >> >> >> >> 4. Do you have any Hadoop-related environment variables set? ($ >> >>env | >> >> >> >>grep >> >> >> >> HADOOP, $ env | grep YARN). >> >> >> >> >> >> >> >> Cheers, >> >> >> >> Chris >> >> >> >> ________________________________________ >> >> >> >> From: Worthy LaFollette [[email protected]] >> >> >> >> Sent: Monday, October 14, 2013 7:37 PM >> >> >> >> To: [email protected] >> >> >> >> Subject: Re: Hello Samza example >> >> >> >> >> >> >> >> Rebuilt and it worked... Now getting a different error: >> >> >> >> >> >> >> >> Application application_1381794085727_0001 failed 1 times due >>to >> >> >>Error >> >> >> >> launching appattempt_1381794085727_0001_000001. Got exception: >> >> >> >> java.lang.reflect.UndeclaredThrowableException >> >> >> >> at >> >> >> >> >> >> >> >> >> >> >> >> >> >> >>>>>>>>org.apache.hadoop.yarn.exceptions.impl.pb.YarnRemoteExceptionPBImpl >>>>>>>>.u >> >>>>>>nw >> >> >>>>ra >> >> >> >>pAndThrowException(YarnRemoteExceptionPBImpl.java:135) >> >> >> >> >> >> >> >> at >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >>org.apache.hadoop.yarn.api.impl.pb.client.ContainerManagerPBClientImpl.st >> >> >> >>artContainer(ContainerManagerPBClientImpl.java:109) >> >> >> >> >> >> >> >> at >> >> >> >> >> >> >> >> >> >> >> >> >> >> >>>>>>>>org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher >>>>>>>>.l >> >>>>>>au >> >> >>>>nc >> >> >> >>h(AMLauncher.java:111) >> >> >> >> >> >> >> >> at >> >> >> >> >> >> >> >> >> >> >> >> >> >> >>>>>>>>org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher >>>>>>>>.r >> >>>>>>un >> >> >>>>(A >> >> >> >>MLauncher.java:255) >> >> >> >> >> >> >> >> at >> >> >> >> >> >> >> >> >> >> >> >> >> >> >>>>>>>>java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecuto >>>>>>>>r. >> >>>>>>ja >> >> >>>>va >> >> >> >>:1145) >> >> >> >> >> >> >> >> at >> >> >> >> >> >> >> >> >> >> >> >> >> >> >>>>>>>>java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecut >>>>>>>>or >> >>>>>>.j >> >> >>>>av >> >> >> >>a:615) >> >> >> >> >> >> >> >> at java.lang.Thread.run(Thread.java:722) >> >> >> >> Caused by: com.google.protobuf.ServiceException: >> >> >> >> java.util.ServiceConfigurationError: >> >> >> >> org.apache.hadoop.security.SecurityInfo: Error reading >> >>configuration >> >> >> >>file >> >> >> >> at >> >> >> >> >> >> >> >> >> >> >> >> >> >> >>>>>>>>org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcE >>>>>>>>ng >> >>>>>>in >> >> >>>>e. >> >> >> >>java:212) >> >> >> >> >> >> >> >> at com.sun.proxy.$Proxy72.startContainer(Unknown Source) >> >> >> >> at >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >>org.apache.hadoop.yarn.api.impl.pb.client.ContainerManagerPBClientImpl.st >> >> >> >>artContainer(ContainerManagerPBClientImpl.java:106) >> >> >> >> >> >> >> >> ... 5 more >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> On Mon, Oct 14, 2013 at 5:40 PM, Chris Riccomini >> >> >> >><[email protected] >> >> >> >> >wrote: >> >> >> >> >> >> >> >> > Hey Worthy, >> >> >> >> > >> >> >> >> > This was a bug that I introduced last week. We received a >>pull >> >> >> >>request on >> >> >> >> > the GH repo, which got merged in about 8 hours ago. >> >> >> >> > >> >> >> >> > https://github.com/linkedin/hello-samza/pull/5 >> >> >> >> > >> >> >> >> > Let me know if it still isn't working (or send a new pull >> >>request >> >> >>:). >> >> >> >> > >> >> >> >> > Cheers, >> >> >> >> > Chris >> >> >> >> > >> >> >> >> > On 10/14/13 5:24 PM, "Worthy LaFollette" <[email protected]> >> >> wrote: >> >> >> >> > >> >> >> >> > >On a related note - I am getting the following for the >> >>wikipedia >> >> >> >>buildt. >> >> >> >> > >I >> >> >> >> > >am suspecting I have something misconfigured somewhere.. >> >> >> >> > > >> >> >> >> > >****** >> >> >> >> > > >> >> >> >> > >ERROR] Failed to execute goal >> >> >> >> > >org.apache.maven.plugins:maven-compiler-plugin:3.1:compile >> >> >> >> > >(default-compile) on project samza-wikipedia: Compilation >> >>failure >> >> >> >> > >[ERROR] >> >> >> >> > >> >> >> >> >> >> >> >> >> >> >>>>>>>>>/Users/worthylafollette/repos/incubator-samza/hello-samza/samza-wi >>>>>>>>>ki >> >>>>>>>pe >> >> >>>>>di >> >> >> >>>a/ >> >> >> >> > >> >> >> >> >> >> >> >> >> >> >>>>>>>>>src/main/java/samza/examples/wikipedia/system/WikipediaSystemAdmin >>>>>>>>>.j >> >>>>>>>av >> >> >>>>>a: >> >> >> >>>[3 >> >> >> >> > >2,8] >> >> >> >> > >samza.examples.wikipedia.system.WikipediaSystemAdmin is not >> >> >>abstract >> >> >> >>and >> >> >> >> > >does not override abstract method >> >> >> >> > >getLastOffsets(java.util.Set<java.lang.String>) in >> >> >> >> > >org.apache.samza.system.SystemAdmin >> >> >> >> > >[ERROR] -> [Help 1] >> >> >> >> > >[ERROR] >> >> >> >> > >[ERROR] To see the full stack trace of the errors, re-run >>Maven >> >> >>with >> >> >> >>the >> >> >> >> > >-e >> >> >> >> > >switch. >> >> >> >> > >[ERROR] Re-run Maven using the -X switch to enable full >>debug >> >> >> >>logging. >> >> >> >> > >[ERROR] >> >> >> >> > >[ERROR] For more information about the errors and possible >> >> >>solutions, >> >> >> >> > >please read the following articles: >> >> >> >> > >[ERROR] [Help 1] >> >> >> >> > > >> >> >> >> http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException >> >> >> >> > >[ERROR] >> >> >> >> > >[ERROR] After correcting the problems, you can resume the >>build >> >> >>with >> >> >> >>the >> >> >> >> > >command >> >> >> >> > >[ERROR] mvn <goals> -rf :samza-wikipedia >> >> >> >> > > >> >> >> >> > > >> >> >> >> > >On Mon, Oct 14, 2013 at 12:15 PM, Jakob Homan >> >><[email protected]> >> >> >> >> wrote: >> >> >> >> > > >> >> >> >> > >> Can you post exactly what's missing? If it's short, here >> >>would >> >> >>be >> >> >> >> fine. >> >> >> >> > >> If it's longer, github gists would be good. >> >> >> >> > >> >> >> >> >> > >> Thanks, >> >> >> >> > >> jakob >> >> >> >> > >> >> >> >> >> > >> >> >> >> >> > >> On Mon, Oct 14, 2013 at 11:40 AM, Isabelly Rocha < >> >> >> >> [email protected] >> >> >> >> > >> >wrote: >> >> >> >> > >> >> >> >> >> > >> > Hey, >> >> >> >> > >> > >> >> >> >> > >> > I want to use Samza and I started downloading this >>"Hello >> >> >>Sanza" >> >> >> >> > >>example, >> >> >> >> > >> > but when I run the grid script I get a lot of file >>missing >> >> >>error >> >> >> >> > >> messages.. >> >> >> >> > >> > Is it not updated or it's probably me doing something >> >>wrong? >> >> >> >> > >> > >> >> >> >> > >> > Thanks for your attention, >> >> >> >> > >> > Isabelly Rocha >> >> >> >> > >> > >> >> >> >> > >> >> >> >> >> > >> >> >> >> > >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >>
