The protobuf-java-2.5.0.jar should be in the Slider lib directory. It will get pushed out to the AM lib directory from there.
There are some other issues with the script output. The install package command has not succeeded, and says "ERROR main.ServiceLauncher - Unable to access supplied pkg file at /tmp/mybuild/incubator-slider/ app-packages/memcached/jmemcached-1.0.0.zip" Also the line INFO agent.AgentClientProvider - Validating app definition /tmp/mybuild/incubator-slider/app-packages/memcached/jmemcached-1.0.1.zip indicates that it is reading the app definition from the local filesystem, when it should be reading it from HDFS. Maybe since this is a single VM setup it is able to continue despite these errors, but if you were running on a real cluster you'd have to make sure install package was working and fix the app definition path in the appConfig. On Tue, Apr 18, 2017 at 6:35 PM, Foolish Ewe <foolish...@hotmail.com> wrote: > Hello All: > > > I'm trying to run the memcached example locally but it fails pretty quick > and claims "protobuf-java-2.5.0.jar does not exist". > > > Consider the following script (test-slider.sh): > > #!/bin/bash > > # this does not work when run during docker-build, we need to do this once > we are logged in > > echo "memcached built, registering memcached with slider and then running > it" > > /usr/lib/slider/bin/slider install-package --package > /tmp/mybuild/incubator-slider/app-packages/memcached/jmemcached-1.0.0.zip > --name jmemcached --debug > > echo "Listing applications - before starting jmemcached" > > /usr/lib/slider/bin/slider list --manager localhost:8032 > > # startup memcached properly, yarn task name will be mymemcached > > /usr/lib/slider/bin/slider create mymemcached --template > /tmp/mybuild/incubator-slider/app-packages/memcached/appConfig.json > --resources > /tmp/mybuild/incubator-slider/app-packages/memcached/resources-default.json > --manager localhost:8032 --debug > > echo "Listing applications - after starting jmemcached" > > /usr/lib/slider/bin/slider list --manager localhost:8032 > > echo "Finished $0" > > I am having difficulty in locating the logs after the failure, any idea > where they would be? yarn logs cannot find them, and I'm not having luck > with the name node. I did modify the yarn-site.xml read at startup to > include the lines: > > <!-- Begin modifications for debugging slider --> > > <!-- 60 minutes after a failure to see what is left in the directory--> > > <property> > > <name>yarn.nodemanager.delete.debug-delay-sec</name> > > <value>3600</value> > > </property> > > > <!--time before the process gets a -9 (Should it be 30 seconds?)--> > > <property> > > <name>yarn.nodemanager.sleep-delay-before-sigkill.ms</name> > > <value>3600000</value> > > </property> > > <!-- End modifications for debugging slider --> > > > Regarding the error, I'm seeing, the slider install-package appears to > work correctly from what I can tell, but the slider create encounters run > time errors when launched, it seems to be looking for > protobuf-java-2.5.0.jar. I tried making fat jars for the memcached > application and the various slider packages, to no avail. Any ideas how to > resolve these errors reported in the resource manager: > Application application_1492557996590_0002 failed 2 times due to AM > Container for appattempt_1492557996590_0002_000002 exited with exitCode: > -1000 > For more detailed output, check application tracking page: > http://quickstart.cloudera:8088/proxy/application_1492557996590_0002/Then, > click on links to logs of each attempt. > Diagnostics: File file:/root/.slider/cluster/mymemcached/tmp/application_ > 1492557996590_0002/am/lib/protobuf-java-2.5.0.jar does not exist > java.io.FileNotFoundException: File file:/root/.slider/cluster/ > mymemcached/tmp/application_1492557996590_0002/am/lib/protobuf-java-2.5.0.jar > does not exist > at org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus( > RawLocalFileSystem.java:542) > at org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal( > RawLocalFileSystem.java:755) > at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus( > RawLocalFileSystem.java:532) > at org.apache.hadoop.fs.FilterFileSystem.getFileStatus( > FilterFileSystem.java:425) > at org.apache.hadoop.yarn.util.FSDownload.copy(FSDownload.java:251) > at org.apache.hadoop.yarn.util.FSDownload.access$000(FSDownload.java:61) > at org.apache.hadoop.yarn.util.FSDownload$2.run(FSDownload.java:359) > at org.apache.hadoop.yarn.util.FSDownload$2.run(FSDownload.java:357) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at org.apache.hadoop.security.UserGroupInformation.doAs( > UserGroupInformation.java:1693) > at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:356) > at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:60) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at java.util.concurrent.ThreadPoolExecutor.runWorker( > ThreadPoolExecutor.java:1142) > at java.util.concurrent.ThreadPoolExecutor$Worker.run( > ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > Failing this attempt. Failing the application. > > > The test-slider.sh script's output is as follows: > > # bash -x /tmp/mybuild/test_slider.sh > > + echo 'memcached built, registering memcached with slider and then > running it' > > memcached built, registering memcached with slider and then running it > > + /usr/lib/slider/bin/slider install-package --package > /tmp/mybuild/incubator-slider/app-packages/memcached/jmemcached-1.0.0.zip > --name jmemcached --debug > > SLF4J: Class path contains multiple SLF4J bindings. > > SLF4J: Found binding in [jar:file:/usr/lib/slider/lib/ > slider-core-0.60.0-incubating.jar!/org/slf4j/impl/ > StaticLoggerBinder.class] > > SLF4J: Found binding in [jar:file:/usr/lib/slider/lib/ > slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class] > > SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an > explanation. > > SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] > > 2017-04-19 01:10:44,080 [main] INFO client.RMProxy - Connecting to > ResourceManager at /0.0.0.0:8032 > > 2017-04-19 01:10:44,243 [main] ERROR main.ServiceLauncher - Unable to > access supplied pkg file at /tmp/mybuild/incubator-slider/ > app-packages/memcached/jmemcached-1.0.0.zip > > 2017-04-19 01:10:44,247 [main] INFO util.ExitUtil - Exiting with status 40 > > + echo 'Listing applications - before starting jmemcached' > > Listing applications - before starting jmemcached > > + /usr/lib/slider/bin/slider list --manager localhost:8032 > > SLF4J: Class path contains multiple SLF4J bindings. > > SLF4J: Found binding in [jar:file:/usr/lib/slider/lib/ > slider-core-0.60.0-incubating.jar!/org/slf4j/impl/ > StaticLoggerBinder.class] > > SLF4J: Found binding in [jar:file:/usr/lib/slider/lib/ > slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class] > > SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an > explanation. > > SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] > > 2017-04-19 01:10:46,066 [main] INFO client.RMProxy - Connecting to > ResourceManager at localhost/127.0.0.1:8032 > > 2017-04-19 01:10:46,217 [main] INFO util.ExitUtil - Exiting with status 0 > > + /usr/lib/slider/bin/slider create mymemcached --template > /tmp/mybuild/incubator-slider/app-packages/memcached/appConfig.json > --resources > /tmp/mybuild/incubator-slider/app-packages/memcached/resources-default.json > --manager localhost:8032 --debug > > SLF4J: Class path contains multiple SLF4J bindings. > > SLF4J: Found binding in [jar:file:/usr/lib/slider/lib/ > slider-core-0.60.0-incubating.jar!/org/slf4j/impl/ > StaticLoggerBinder.class] > > SLF4J: Found binding in [jar:file:/usr/lib/slider/lib/ > slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class] > > SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an > explanation. > > SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] > > 2017-04-19 01:10:48,070 [main] INFO client.RMProxy - Connecting to > ResourceManager at localhost/127.0.0.1:8032 > > 2017-04-19 01:10:48,647 [main] INFO agent.AgentClientProvider - > Validating app definition /tmp/mybuild/incubator-slider/ > app-packages/memcached/jmemcached-1.0.1.zip > > 2017-04-19 01:10:48,648 [main] INFO agent.AgentUtils - Reading metainfo > at /tmp/mybuild/incubator-slider/app-packages/memcached/ > jmemcached-1.0.1.zip > > 2017-04-19 01:10:48,675 [main] INFO tools.SliderUtils - Reading > metainfo.xml of size 2202 > > 2017-04-19 01:10:48,893 [main] INFO client.SliderClient - No credentials > requested > > 2017-04-19 01:10:48,938 [main] INFO agent.AgentUtils - Reading metainfo > at /tmp/mybuild/incubator-slider/app-packages/memcached/ > jmemcached-1.0.1.zip > > 2017-04-19 01:10:48,939 [main] INFO tools.SliderUtils - Reading > metainfo.xml of size 2202 > > 2017-04-19 01:10:48,976 [main] INFO launch.AbstractLauncher - Log include > patterns: > > 2017-04-19 01:10:48,976 [main] INFO launch.AbstractLauncher - Log exclude > patterns: > > 2017-04-19 01:10:49,861 [main] INFO slideram.SliderAMClientProvider - > Loading all dependencies for AM. > > 2017-04-19 01:10:49,862 [main] INFO tools.SliderUtils - Loading all > dependencies from /usr/lib/slider/lib > > 2017-04-19 01:10:51,324 [main] INFO agent.AgentClientProvider - > Automatically uploading the agent tarball at file:/root/.slider/cluster/ > mymemcached/tmp/application_1492557996590_0003/agent > > 2017-04-19 01:10:51,361 [main] INFO agent.AgentClientProvider - > Validating app definition /tmp/mybuild/incubator-slider/ > app-packages/memcached/jmemcached-1.0.1.zip > > 2017-04-19 01:10:51,361 [main] INFO agent.AgentUtils - Reading metainfo > at /tmp/mybuild/incubator-slider/app-packages/memcached/ > jmemcached-1.0.1.zip > > 2017-04-19 01:10:51,364 [main] INFO tools.SliderUtils - Reading > metainfo.xml of size 2202 > > 2017-04-19 01:10:51,429 [main] INFO Configuration.deprecation - > slider.registry.path is deprecated. Instead, use hadoop.registry.zk.root > > 2017-04-19 01:10:51,436 [main] INFO launch.AppMasterLauncher - Submitting > application to Resource Manager > > 2017-04-19 01:10:51,480 [main] INFO impl.YarnClientImpl - Submitted > application application_1492557996590_0003 > > 2017-04-19 01:10:51,484 [main] INFO util.ExitUtil - Exiting with status 0 > > + echo 'Listing applications - after starting jmemcached' > > Listing applications - after starting jmemcached > > + /usr/lib/slider/bin/slider list --manager localhost:8032 > > SLF4J: Class path contains multiple SLF4J bindings. > > SLF4J: Found binding in [jar:file:/usr/lib/slider/lib/ > slider-core-0.60.0-incubating.jar!/org/slf4j/impl/ > StaticLoggerBinder.class] > > SLF4J: Found binding in [jar:file:/usr/lib/slider/lib/ > slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class] > > SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an > explanation. > > SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] > > 2017-04-19 01:10:53,893 [main] INFO client.RMProxy - Connecting to > ResourceManager at localhost/127.0.0.1:8032 > > mymemcached ACCEPTED application_1492557996590_0003 > > 2017-04-19 01:10:54,186 [main] INFO util.ExitUtil - Exiting with status 0 > > + echo 'Finished /tmp/mybuild/test_slider.sh' > > Finished /tmp/mybuild/test_slider.sh > > > Thanks: > > > Bill >
