[ https://issues.apache.org/jira/browse/GEARPUMP-147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Manu Zhang updated GEARPUMP-147: -------------------------------- Assignee: Huafeng Wang > java.lang.ClassNotFoundException when running on worker with > worker.executor-share-same-jvm-as-worker = true > ------------------------------------------------------------------------------------------------------------ > > Key: GEARPUMP-147 > URL: https://issues.apache.org/jira/browse/GEARPUMP-147 > Project: Apache Gearpump > Issue Type: Task > Reporter: Karol Brejna > Assignee: Huafeng Wang > Attachments: gear.pc.conf, gear.pc.slimworker.conf, > logs.slimworker.zip, prepare.sh > > > Doing GEARPUMP-146 I hit a problem: I was able to run a master on pc and a > worker on raspi. After deploying wordcountjava-2.11-0.8.0-assembly.jar the > example didn't start. > I reproduced this on "pure pc" cluster. > I've prepared dedicated config files: > * gear.pc.conf - normal (default) configuration for > master and workers ran on a PC > * gear.pc.slimworker.conf - intends to have lower requirements for memory, > cpu - to be used on a device (raspi, edison) > *Obtaining binary and creating config files* > {code} > wget > https://github.com/gearpump/gearpump/releases/download/0.8.0/gearpump-2.11-0.8.0.zip > unzip gearpump-2.11-0.8.0.zip > cd gearpump-2.11-0.8.0 > cp conf/gear.conf conf/gear.original.conf > cp conf/gear.conf conf/gear.pc.conf > cp conf/gear.conf conf/gear.pc.slimworker.conf > {code} > *Preparing "PC" conf* > The only real change here is putting the IP of the machine that will run the > master: > {code} > cluster { > masters = ["192.168.1.115:3000"] > } > > hostname = "192.168.1.115" > {code} > > *Preparing device conf* > Compared to previous config, this config should set up less worker slots, > smaller -Xmx jvm switch for worker, and executor-share-same-jvm-as-worker set > to true: > {code} > worker.slots = 100 > worker.executor-share-same-jvm-as-worker = true > executor { > vmargs = "-server -Xms512M -Xmx512M -Xss1M > -XX:+HeapDumpOnOutOfMemoryError -XX:+UseConcMarkSweepGC > -XX:CMSInitiatingOccupancyFraction=80 -XX:+UseParNewGC -XX:NewRatio=3 > -Djava.rmi.server.hostname=localhost" > extraClasspath = "" > } > {code} > *Running the cluster:* > {code} > echo "Starting the master" > cp conf/gear.pc.conf conf/gear.conf > bin/master -ip 192.168.1.115 -port 3000 & > echo "Starting services" > bin/services & > echo "Starting a worker (configured as on the device)" > cp conf/gear.pc.slimworker.conf conf/gear.conf > bin/worker & > {code} > *Deploing the DAG* > I've deployed wordcountjava-2.11-0.8.0-assembly.jar. > The app deploys, but doesn't run. > (After changing worker.executor-share-same-jvm-as-worker to false and > restarting the cluster, example dag works.) > See the attached logs: > {code} > 2016-05-18 08:47:31,453 ERROR OneForOneStrategy: > io.gearpump.streaming.examples.wordcountjava.Split > java.lang.ClassNotFoundException: > io.gearpump.streaming.examples.wordcountjava.Split > at java.net.URLClassLoader.findClass(URLClassLoader.java:381) > at java.lang.ClassLoader.loadClass(ClassLoader.java:424) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) > at java.lang.ClassLoader.loadClass(ClassLoader.java:357) > at io.gearpump.streaming.task.TaskUtil$.loadClass(TaskUtil.scala:30) > at > io.gearpump.streaming.executor.TaskLauncher.launch(TaskLauncher.scala:55) > at > io.gearpump.streaming.executor.Executor$$anonfun$dynamicDagPhase1$1.applyOrElse(Executor.scala:147) > at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:170) > at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) > at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) > at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171) > at akka.actor.Actor$class.aroundReceive(Actor.scala:480) > at > io.gearpump.streaming.executor.Executor.aroundReceive(Executor.scala:55) > at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526) > at akka.actor.ActorCell.invoke(ActorCell.scala:495) > at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257) > at akka.dispatch.Mailbox.run(Mailbox.scala:224) > at akka.dispatch.Mailbox.exec(Mailbox.scala:234) > at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) > at > scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) > at > scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) > at > scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) > {code} > -- This message was sent by Atlassian JIRA (v6.3.4#6332)