[jira] [Commented] (OOZIE-2482) Pyspark job fails with Oozie

Alexandre Linte (JIRA) Fri, 20 May 2016 09:00:08 -0700

    [ 
https://issues.apache.org/jira/browse/OOZIE-2482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15293604#comment-15293604
 ]


Alexandre Linte commented on OOZIE-2482:
----------------------------------------

Hi [~satishsaley], I won't be able to give you the logs for the application 
1461692698792_19525, the logs were purged.

Here are the logs for a pyspark job that fails with the same error (application 
1461692698792_29704 / 1461692698792_29705).

OOZIE LOGS
{noformat}
2016-05-20 17:42:00,627  INFO CallbackServlet:520 - USER[-] GROUP[-] TOKEN[-] 
APP[-] JOB[0012689-160510172237486-oozie-W] 
ACTION[0012689-160510172237486-oozie-W@spark-node] callback for action 
[0012689-160510172237486-oozie-W@spark-node]
2016-05-20 17:42:00,892  INFO SparkActionExecutor:520 - USER[shfs3453] GROUP[-] 
TOKEN[] APP[PysparkPi-test] JOB[0012689-160510172237486-oozie-W] 
ACTION[0012689-160510172237486-oozie-W@spark-node] action completed, external 
ID [job_1461692698792_29704]
2016-05-20 17:42:00,897  WARN SparkActionExecutor:523 - USER[shfs3453] GROUP[-] 
TOKEN[] APP[PysparkPi-test] JOB[0012689-160510172237486-oozie-W] 
ACTION[0012689-160510172237486-oozie-W@spark-node] Launcher ERROR, reason: Main 
class [org.apache.oozie.action.hadoop.SparkMain], main() threw exception, 
Application application_1461692698792_29705 finished with failed status
2016-05-20 17:42:00,897  WARN SparkActionExecutor:523 - USER[shfs3453] GROUP[-] 
TOKEN[] APP[PysparkPi-test] JOB[0012689-160510172237486-oozie-W] 
ACTION[0012689-160510172237486-oozie-W@spark-node] Launcher exception: 
Application application_1461692698792_29705 finished with failed status
org.apache.spark.SparkException: Application application_1461692698792_29705 
finished with failed status
        at org.apache.spark.deploy.yarn.Client.run(Client.scala:1034)
        at org.apache.spark.deploy.yarn.Client$.main(Client.scala:1081)
        at org.apache.spark.deploy.yarn.Client.main(Client.scala)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at 
org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:731)
        at 
org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181)
        at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206)
        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)
        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
        at org.apache.oozie.action.hadoop.SparkMain.runSpark(SparkMain.java:104)
        at org.apache.oozie.action.hadoop.SparkMain.run(SparkMain.java:95)
        at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:47)
        at org.apache.oozie.action.hadoop.SparkMain.main(SparkMain.java:38)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at 
org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:236)
        at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
        at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
        at 
org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runSubtask(LocalContainerLauncher.java:380)
        at 
org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runTask(LocalContainerLauncher.java:301)
        at 
org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.access$200(LocalContainerLauncher.java:187)
        at 
org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler$1.run(LocalContainerLauncher.java:230)
        at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)

2016-05-20 17:42:01,017  INFO ActionEndXCommand:520 - USER[shfs3453] GROUP[-] 
TOKEN[] APP[PysparkPi-test] JOB[0012689-160510172237486-oozie-W] 
ACTION[0012689-160510172237486-oozie-W@spark-node] ERROR is considered as 
FAILED for SLA
2016-05-20 17:42:01,080  INFO ActionStartXCommand:520 - USER[shfs3453] GROUP[-] 
TOKEN[] APP[PysparkPi-test] JOB[0012689-160510172237486-oozie-W] 
ACTION[0012689-160510172237486-oozie-W@fail] Start action 
[0012689-160510172237486-oozie-W@fail] with user-retry state : userRetryCount 
[0], userRetryMax [0], userRetryInterval [10]
2016-05-20 17:42:01,081  INFO ActionStartXCommand:520 - USER[shfs3453] GROUP[-] 
TOKEN[] APP[PysparkPi-test] JOB[0012689-160510172237486-oozie-W] 
ACTION[0012689-160510172237486-oozie-W@fail] 
[***0012689-160510172237486-oozie-W@fail***]Action status=DONE
{noformat}

RESOURCE MANAGER LOGS
{noformat}
2016-05-20 17:41:51,123 WARN 
org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl: The specific max 
attempts: 0 for application: 29705 is invalid, because it is out of the range 
[1, 2]. Use the global max attempts instead.
2016-05-20 17:41:52,880 WARN 
org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl: The specific max 
attempts: 0 for application: 29706 is invalid, because it is out of the range 
[1, 2]. Use the global max attempts instead.
2016-05-20 17:41:58,161 WARN 
org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger: USER=shfs3453 
OPERATION=Application Finished - Failed TARGET=RMAppManager     RESULT=FAILURE  
DESCRIPTION=App failed with state: FAILED       PERMISSIONS=Application 
application_1461692698792_29705 failed 2 times due to AM Container for 
appattempt_1461692698792_29705_000002 exited with  exitCode: 1
For more detailed output, check application tracking 
page:http://uabigrm02.rouen.francetelecom.fr:8088/cluster/app/application_1461692698792_29705Then,
 click on links to logs of each attempt.
Diagnostics: Exception from container-launch.
Container id: container_1461692698792_29705_02_000001
Exit code: 1
Stack trace: ExitCodeException exitCode=1:
        at org.apache.hadoop.util.Shell.runCommand(Shell.java:545)
        at org.apache.hadoop.util.Shell.run(Shell.java:456)
        at 
org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:722)
        at 
org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.launchContainer(LinuxContainerExecutor.java:297)
        at 
org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
        at 
org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)

Shell output: main : command provided 1
main : user is shfs3453
main : requested yarn user is shfs3453


Container exited with a non-zero exit code 1
Failing this attempt. Failing the application.  
APPID=application_1461692698792_29705
2016-05-20 17:41:59,375 INFO SecurityLogger.org.apache.hadoop.ipc.Server: Auth 
successful for appattempt_1461692698792_29706_000001 (auth:SIMPLE)
2016-05-20 17:41:59,381 INFO 
SecurityLogger.org.apache.hadoop.security.authorize.ServiceAuthorizationManager:
 Authorization successful for appattempt_1461692698792_29706_000001 
(auth:TOKEN) for protocol=interface 
org.apache.hadoop.yarn.api.ApplicationMasterProtocolPB
2016-05-20 17:42:00,745 INFO SecurityLogger.org.apache.hadoop.ipc.Server: Auth 
successful for oozie/uabigord01.rouen.francetelecom.fr@SANDBOX.HADOOP 
(auth:KERBEROS)
{noformat}

DATANODE LOGS (stderr)
{noformat}
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in 
[jar:file:/mnt/hd3/hadoop/yarn/local/filecache/187/slf4j-log4j12-1.6.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in 
[jar:file:/mnt/hd9/hadoop/yarn/local/filecache/249/spark-assembly-1.6.1-hadoop2.7.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in 
[jar:file:/opt/application/Hadoop/hadoop-2.7.2/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
log4j:ERROR setFile(null,true) call failed.
java.io.FileNotFoundException: 
/mnt/hd6/hadoop/yarn/log/application_1461692698792_29704/container_1461692698792_29704_01_000001
 (Is a directory)
        at java.io.FileOutputStream.open(Native Method)
        at java.io.FileOutputStream.<init>(FileOutputStream.java:221)
        at java.io.FileOutputStream.<init>(FileOutputStream.java:142)
        at org.apache.log4j.FileAppender.setFile(FileAppender.java:294)
        at org.apache.log4j.FileAppender.activateOptions(FileAppender.java:165)
        at 
org.apache.hadoop.yarn.ContainerLogAppender.activateOptions(ContainerLogAppender.java:55)
        at 
org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:307)
        at 
org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:172)
        at 
org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:104)
        at 
org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:809)
        at 
org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:735)
        at 
org.apache.log4j.PropertyConfigurator.configureRootCategory(PropertyConfigurator.java:615)
        at 
org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:502)
        at 
org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:547)
        at 
org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:483)
        at org.apache.log4j.LogManager.<clinit>(LogManager.java:127)
        at 
org.slf4j.impl.Log4jLoggerFactory.getLogger(Log4jLoggerFactory.java:64)
        at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:270)
        at 
org.apache.commons.logging.impl.SLF4JLogFactory.getInstance(SLF4JLogFactory.java:155)
        at 
org.apache.commons.logging.impl.SLF4JLogFactory.getInstance(SLF4JLogFactory.java:132)
        at org.apache.commons.logging.LogFactory.getLog(LogFactory.java:275)
        at 
org.apache.hadoop.service.AbstractService.<clinit>(AbstractService.java:43)
May 20, 2016 5:41:44 PM 
com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFO: Registering org.apache.hadoop.mapreduce.v2.app.webapp.JAXBContextResolver 
as a provider class
May 20, 2016 5:41:44 PM 
com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFO: Registering org.apache.hadoop.yarn.webapp.GenericExceptionHandler as a 
provider class
May 20, 2016 5:41:44 PM 
com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFO: Registering org.apache.hadoop.mapreduce.v2.app.webapp.AMWebServices as a 
root resource class
May 20, 2016 5:41:44 PM 
com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.9 09/02/2011 11:17 AM'
May 20, 2016 5:41:44 PM 
com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory 
getComponentProvider
INFO: Binding org.apache.hadoop.mapreduce.v2.app.webapp.JAXBContextResolver to 
GuiceManagedComponentProvider with the scope "Singleton"
May 20, 2016 5:41:44 PM 
com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory 
getComponentProvider
INFO: Binding org.apache.hadoop.yarn.webapp.GenericExceptionHandler to 
GuiceManagedComponentProvider with the scope "Singleton"
May 20, 2016 5:41:45 PM 
com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory 
getComponentProvider
INFO: Binding org.apache.hadoop.mapreduce.v2.app.webapp.AMWebServices to 
GuiceManagedComponentProvider with the scope "PerRequest"
Using properties file: null
Parsed arguments:
  master                  yarn-cluster
  deployMode              cluster
  executorMemory          null
  executorCores           null
  totalExecutorCores      null
  propertiesFile          null
  driverMemory            null
  driverCores             null
  driverExtraClassPath    null
  driverExtraLibraryPath  null
  driverExtraJavaOptions  null
  supervise               false
  queue                   null
  numExecutors            null
  files                   null
  pyFiles                 
hdfs://sandbox/User/shfs3453/WORK/Oozie/pyspark/lib/py4j-0.9-src.zip,hdfs://sandbox/User/shfs3453/WORK/Oozie/pyspark/lib/pyspark.zip
  archives                null
  mainClass               null
  primaryResource         
hdfs://sandbox/User/shfs3453/WORK/Oozie/pyspark/lib/pi.py
  name                    Pysparkpi example
  childArgs               [100]
  jars                    null
  packages                null
  packagesExclusions      null
  repositories            null
  verbose                 true

Spark properties used, including those specified through
 --conf and those from the properties file null:
  spark.executorEnv.SPARK_HOME -> /opt/application/Spark/current
  spark.executorEnv.PYTHONPATH -> /opt/application/Spark/current/python
  spark.yarn.appMasterEnv.SPARK_HOME -> /opt/application/Spark/current


Main class:
org.apache.spark.deploy.yarn.Client
Arguments:
--name
Pysparkpi example
--primary-py-file
hdfs://sandbox/User/shfs3453/WORK/Oozie/pyspark/lib/pi.py
--py-files
hdfs://sandbox/User/shfs3453/WORK/Oozie/pyspark/lib/py4j-0.9-src.zip,hdfs://sandbox/User/shfs3453/WORK/Oozie/pyspark/lib/pyspark.zip
--class
org.apache.spark.deploy.PythonRunner
--arg
100
System properties:
spark.executorEnv.SPARK_HOME -> /opt/application/Spark/current
spark.executorEnv.PYTHONPATH -> /opt/application/Spark/current/python
SPARK_SUBMIT -> true
spark.app.name -> Pysparkpi example
spark.submit.deployMode -> cluster
spark.yarn.appMasterEnv.SPARK_HOME -> /opt/application/Spark/current
spark.yarn.isPython -> true
spark.master -> yarn-cluster
Classpath elements:



Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.SparkMain], 
main() threw exception, Application application_1461692698792_29705 finished 
with failed status
org.apache.spark.SparkException: Application application_1461692698792_29705 
finished with failed status
        at org.apache.spark.deploy.yarn.Client.run(Client.scala:1034)
        at org.apache.spark.deploy.yarn.Client$.main(Client.scala:1081)
        at org.apache.spark.deploy.yarn.Client.main(Client.scala)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at 
org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:731)
        at 
org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181)
        at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206)
        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)
        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
        at org.apache.oozie.action.hadoop.SparkMain.runSpark(SparkMain.java:104)
        at org.apache.oozie.action.hadoop.SparkMain.run(SparkMain.java:95)
        at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:47)
        at org.apache.oozie.action.hadoop.SparkMain.main(SparkMain.java:38)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at 
org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:236)
        at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
        at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
        at 
org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runSubtask(LocalContainerLauncher.java:380)
        at 
org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runTask(LocalContainerLauncher.java:301)
        at 
org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.access$200(LocalContainerLauncher.java:187)
        at 
org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler$1.run(LocalContainerLauncher.java:230)
        at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
log4j:WARN No appenders could be found for logger 
(org.apache.hadoop.mapreduce.v2.app.MRAppMaster).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more 
info.
{noformat}

DATANODE LOGS (stdout)
{noformat}

Oozie Launcher starts

Heart beat
{"properties":[{"key":"oozie.launcher.job.id","value":"job_1461692698792_29704","isFinal":false,"resource":"programatically"},{"key":"oozie.job.id","value":"0012689-160510172237486-oozie-W","isFinal":false,"resource":"programatically"},{"key":"oozie.action.id","value":"0012689-160510172237486-oozie-W@spark-node","isFinal":false,"resource":"programatically"},{"key":"mapreduce.job.tags","value":"oozie-f217792cb72212277adf42b1fef23939","isFinal":false,"resource":"programatically"}]}Starting
 the execution of prepare actions
Completed the execution of prepare actions successfully

Files in current 
dir:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/.
======================
File: jackson-databind-2.3.1.jar
File: jets3t-0.7.1.jar
File: libhadoop.a
File: propagation-conf.xml
File: paranamer-2.6.jar
File: zookeeper-3.4.6.jar
File: tachyon-client-0.5.0.jar
File: spark-graphx_2.10-1.6.1.jar
File: akka-actor_2.10-2.2.3-shaded-protobuf.jar
File: spark-catalyst_2.10-1.6.1.jar
File: jetty-webapp-8.1.14.v20131031.jar
File: slf4j-api-1.6.6.jar
File: javax.mail.glassfish-1.4.1.v201005082020.jar
File: spark-mllib_2.10-1.6.1.jar
File: jsr305-1.3.9.jar
File: libhadoop.so.1.0.0
File: jetty-security-8.1.14.v20131031.jar
File: spark-core_2.10-1.6.1.jar
File: scala-compiler-2.10.0.jar
File: protobuf-java-2.4.1-shaded.jar
File: akka-remote_2.10-2.2.3-shaded-protobuf.jar
File: metrics-jvm-3.0.2.jar
File: metrics-graphite-3.0.0.jar
File: netty-all-4.0.23.Final.jar
File: container_tokens
File: objenesis-1.2.jar
File: curator-recipes-2.5.0.jar
File: chill-java-0.3.6.jar
File: json4s-ast_2.10-3.2.10.jar
File: commons-io-2.1.jar
File: jetty-io-8.1.14.v20131031.jar
File: jetty-continuation-8.1.14.v20131031.jar
File: action.xml
File: launch_container.sh
File: stream-2.7.0.jar
File: py4j-0.9-src.zip
File: .action.xml.crc
File: minlog-1.2.jar
File: libhdfs.a
File: netty-3.6.6.Final.jar
File: jetty-util-8.1.14.v20131031.jar
File: spark-yarn_2.10-1.6.1.jar
File: scalap-2.10.0.jar
File: job.xml
File: json-simple-1.1.jar
File: jetty-http-8.1.14.v20131031.jar
File: pyrolite-2.0.1.jar
File: log4j-1.2.16.jar
File: jline-0.9.94.jar
File: uncommons-maths-1.2.2a.jar
File: slf4j-log4j12-1.6.6.jar
File: py4j-0.8.2.1.jar
File: json4s-jackson_2.10-3.2.10.jar
Dir: jobSubmitDir
  File: job.splitmetainfo
  File: job.split
File: metrics-core-3.0.2.jar
File: jblas-1.2.3.jar
File: libhdfs.so.0.0.0
File: curator-client-2.7.1.jar
File: spark-sql_2.10-1.6.1.jar
File: concurrent-1.3.4.jar
File: jetty-jndi-8.1.14.v20131031.jar
File: oozie-sharelib-spark-4.2.0.jar
File: curator-framework-2.5.0.jar
File: pyspark.zip
File: javax.activation-1.1.0.v201105071233.jar
File: tachyon-0.6.4.jar
File: reflectasm-1.07-shaded.jar
File: spark-tools_2.10-1.6.1.jar
File: GC.log
File: libhadooppipes.a
File: spark-assembly-1.6.1-hadoop2.7.2.jar
File: javax.servlet-3.0.0.v201112011016.jar
File: jetty-servlet-8.1.14.v20131031.jar
File: scala-library-2.10.4.jar
File: jackson-annotations-2.3.0.jar
File: libhadoop.so
File: akka-slf4j_2.10-2.2.3-shaded-protobuf.jar
File: spark-network-yarn_2.10-1.6.1.jar
File: commons-lang3-3.3.2.jar
File: compress-lzf-1.0.0.jar
File: spark-1.6.1-yarn-shuffle.jar
File: jackson-core-2.3.1.jar
File: json4s-core_2.10-3.2.10.jar
File: jetty-plus-8.1.14.v20131031.jar
File: config-1.0.2.jar
File: oozie-sharelib-oozie-4.2.0.jar
File: scala-reflect-2.10.0.jar
File: jetty-xml-8.1.14.v20131031.jar
File: jetty-server-8.1.14.v20131031.jar
File: commons-lang-2.4.jar
File: snappy-java-1.0.5.3.jar
File: libhdfs.so
File: kryo-2.21.jar
File: spark-streaming_2.10-1.6.1.jar
File: commons-httpclient-3.1.jar
File: guava-14.0.1.jar
File: commons-codec-1.4.jar
File: libhadooputils.a
Dir: tmp
  Dir: Jetty_0_0_0_0_54973_mapreduce____.j5f3a5
File: chill_2.10-0.3.6.jar
File: lz4-1.2.0.jar
File: javax.transaction-1.1.1.v201105210645.jar
File: commons-net-2.2.jar
File: spark-hive_2.10-1.6.1.jar
File: metrics-json-3.0.2.jar
File: colt-1.2.0.jar
File: oozie-hadoop-utils-hadoop-2-4.2.0.jar

Oozie Java/Map-Reduce/Pig action launcher-job configuration
=================================================================
Workflow job id   : 0012689-160510172237486-oozie-W
Workflow action id: 0012689-160510172237486-oozie-W@spark-node

Classpath         :
------------------------
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001
  job.jar/job.jar
  job.jar/classes/
  job.jar/lib/*
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/jackson-databind-2.3.1.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/jets3t-0.7.1.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/paranamer-2.6.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/zookeeper-3.4.6.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/tachyon-client-0.5.0.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/spark-graphx_2.10-1.6.1.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/akka-actor_2.10-2.2.3-shaded-protobuf.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/spark-catalyst_2.10-1.6.1.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/jetty-webapp-8.1.14.v20131031.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/slf4j-api-1.6.6.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/javax.mail.glassfish-1.4.1.v201005082020.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/spark-mllib_2.10-1.6.1.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/jsr305-1.3.9.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/jetty-security-8.1.14.v20131031.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/spark-core_2.10-1.6.1.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/scala-compiler-2.10.0.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/protobuf-java-2.4.1-shaded.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/akka-remote_2.10-2.2.3-shaded-protobuf.jar
    
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/spark-catalyst_2.10-1.6.1.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/jetty-webapp-8.1.14.v20131031.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/slf4j-api-1.6.6.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/javax.mail.glassfish-1.4.1.v201005082020.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/spark-mllib_2.10-1.6.1.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/jsr305-1.3.9.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/jetty-security-8.1.14.v20131031.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/spark-core_2.10-1.6.1.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/scala-compiler-2.10.0.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/protobuf-java-2.4.1-shaded.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/akka-remote_2.10-2.2.3-shaded-protobuf.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/metrics-jvm-3.0.2.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/metrics-graphite-3.0.0.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/netty-all-4.0.23.Final.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/objenesis-1.2.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/curator-recipes-2.5.0.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/chill-java-0.3.6.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/json4s-ast_2.10-3.2.10.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/commons-io-2.1.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/jetty-io-8.1.14.v20131031.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/jetty-continuation-8.1.14.v20131031.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/stream-2.7.0.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/minlog-1.2.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/netty-3.6.6.Final.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/jetty-util-8.1.14.v20131031.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/spark-yarn_2.10-1.6.1.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/scalap-2.10.0.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/json-simple-1.1.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/jetty-http-8.1.14.v20131031.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/pyrolite-2.0.1.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/log4j-1.2.16.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/jline-0.9.94.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/uncommons-maths-1.2.2a.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/slf4j-log4j12-1.6.6.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/py4j-0.8.2.1.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/json4s-jackson_2.10-3.2.10.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/metrics-core-3.0.2.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/jblas-1.2.3.jar
    
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/curator-client-2.7.1.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/spark-sql_2.10-1.6.1.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/concurrent-1.3.4.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/jetty-jndi-8.1.14.v20131031.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/oozie-sharelib-spark-4.2.0.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/curator-framework-2.5.0.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/javax.activation-1.1.0.v201105071233.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/tachyon-0.6.4.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/reflectasm-1.07-shaded.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/spark-tools_2.10-1.6.1.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/spark-assembly-1.6.1-hadoop2.7.2.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/javax.servlet-3.0.0.v201112011016.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/jetty-servlet-8.1.14.v20131031.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/scala-library-2.10.4.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/jackson-annotations-2.3.0.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/akka-slf4j_2.10-2.2.3-shaded-protobuf.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/spark-network-yarn_2.10-1.6.1.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/commons-lang3-3.3.2.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/compress-lzf-1.0.0.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/spark-1.6.1-yarn-shuffle.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/jackson-core-2.3.1.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/json4s-core_2.10-3.2.10.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/jetty-plus-8.1.14.v20131031.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/config-1.0.2.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/oozie-sharelib-oozie-4.2.0.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/scala-reflect-2.10.0.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/jetty-xml-8.1.14.v20131031.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/jetty-server-8.1.14.v20131031.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/commons-lang-2.4.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/snappy-java-1.0.5.3.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/kryo-2.21.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/spark-streaming_2.10-1.6.1.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/commons-httpclient-3.1.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/guava-14.0.1.jar
    
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/commons-codec-1.4.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/chill_2.10-0.3.6.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/lz4-1.2.0.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/javax.transaction-1.1.1.v201105210645.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/commons-net-2.2.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/spark-hive_2.10-1.6.1.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/metrics-json-3.0.2.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/colt-1.2.0.jar
  
/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/oozie-hadoop-utils-hadoop-2-4.2.0.jar
  /opt/application/Hadoop/current/etc/hadoop/
  /opt/application/Hadoop/current/share/hadoop/common/hadoop-common-2.7.2.jar
  /opt/application/Hadoop/current/share/hadoop/common/hadoop-nfs-2.7.2.jar
  
/opt/application/Hadoop/current/share/hadoop/common/hadoop-common-2.7.2-tests.jar
  /opt/application/Hadoop/current/share/hadoop/common/lib/jersey-server-1.9.jar
  /opt/application/Hadoop/current/share/hadoop/common/lib/guava-11.0.2.jar
  /opt/application/Hadoop/current/share/hadoop/common/lib/jsp-api-2.1.jar
  /opt/application/Hadoop/current/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar
  /opt/application/Hadoop/current/share/hadoop/common/lib/junit-4.11.jar
  /opt/application/Hadoop/current/share/hadoop/common/lib/commons-codec-1.4.jar
  /opt/application/Hadoop/current/share/hadoop/common/lib/netty-3.6.2.Final.jar
  /opt/application/Hadoop/current/share/hadoop/common/lib/httpcore-4.2.5.jar
  /opt/application/Hadoop/current/share/hadoop/common/lib/asm-3.2.jar
  /opt/application/Hadoop/current/share/hadoop/common/lib/jetty-util-6.1.26.jar
  
/opt/application/Hadoop/current/share/hadoop/common/lib/curator-framework-2.7.1.jar
  
/opt/application/Hadoop/current/share/hadoop/common/lib/commons-digester-1.8.jar
  /opt/application/Hadoop/current/share/hadoop/common/lib/jetty-6.1.26.jar
  
/opt/application/Hadoop/current/share/hadoop/common/lib/jackson-mapper-asl-1.9.13.jar
  /opt/application/Hadoop/current/share/hadoop/common/lib/httpclient-4.2.5.jar
  
/opt/application/Hadoop/current/share/hadoop/common/lib/commons-collections-3.2.2.jar
  
/opt/application/Hadoop/current/share/hadoop/common/lib/hadoop-annotations-2.7.2.jar
  
/opt/application/Hadoop/current/share/hadoop/common/lib/protobuf-java-2.5.0.jar
  
/opt/application/Hadoop/current/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar
  /opt/application/Hadoop/current/share/hadoop/common/lib/hadoop-auth-2.7.2.jar
  /opt/application/Hadoop/current/share/hadoop/common/lib/paranamer-2.3.jar
  
/opt/application/Hadoop/current/share/hadoop/common/lib/api-asn1-api-1.0.0-M20.jar
  /opt/application/Hadoop/current/share/hadoop/common/lib/zookeeper-3.4.6.jar
  
/opt/application/Hadoop/current/share/hadoop/common/lib/htrace-core-3.1.0-incubating.jar
  /opt/application/Hadoop/current/share/hadoop/common/lib/xmlenc-0.52.jar
  /opt/application/Hadoop/current/share/hadoop/common/lib/commons-cli-1.2.jar
  /opt/application/Hadoop/current/share/hadoop/common/lib/xz-1.0.jar
  /opt/application/Hadoop/current/share/hadoop/common/lib/jsch-0.1.42.jar
  
/opt/application/Hadoop/current/share/hadoop/common/lib/apacheds-kerberos-codec-2.0.0-M15.jar
  /opt/application/Hadoop/current/share/hadoop/common/lib/api-util-1.0.0-M20.jar
  /opt/application/Hadoop/current/share/hadoop/common/lib/jersey-core-1.9.jar
  /opt/application/Hadoop/current/share/hadoop/common/lib/stax-api-1.0-2.jar
  /opt/application/Hadoop/current/share/hadoop/common/lib/jersey-json-1.9.jar
  /opt/application/Hadoop/current/share/hadoop/common/lib/avro-1.7.4.jar
  /opt/application/Hadoop/current/share/hadoop/common/lib/log4j-1.2.17.jar
  
/opt/application/Hadoop/current/share/hadoop/common/lib/commons-logging-1.1.3.jar
  /opt/application/Hadoop/current/share/hadoop/common/lib/jackson-xc-1.9.13.jar
  /opt/application/Hadoop/current/share/hadoop/common/lib/slf4j-api-1.7.10.jar
  
/opt/application/Hadoop/current/share/hadoop/common/lib/java-xmlbuilder-0.4.jar
  /opt/application/Hadoop/current/share/hadoop/common/lib/activation-1.1.jar
    /opt/application/Hadoop/current/share/hadoop/common/lib/jettison-1.1.jar
  /opt/application/Hadoop/current/share/hadoop/common/lib/servlet-api-2.5.jar
  /opt/application/Hadoop/current/share/hadoop/common/lib/jaxb-api-2.2.2.jar
  
/opt/application/Hadoop/current/share/hadoop/common/lib/apacheds-i18n-2.0.0-M15.jar
  
/opt/application/Hadoop/current/share/hadoop/common/lib/commons-beanutils-core-1.8.0.jar
  
/opt/application/Hadoop/current/share/hadoop/common/lib/commons-httpclient-3.1.jar
  
/opt/application/Hadoop/current/share/hadoop/common/lib/snappy-java-1.0.4.1.jar
  
/opt/application/Hadoop/current/share/hadoop/common/lib/commons-beanutils-1.7.0.jar
  /opt/application/Hadoop/current/share/hadoop/common/lib/commons-io-2.4.jar
  /opt/application/Hadoop/current/share/hadoop/common/lib/mockito-all-1.8.5.jar
  /opt/application/Hadoop/current/share/hadoop/common/lib/jsr305-3.0.0.jar
  
/opt/application/Hadoop/current/share/hadoop/common/lib/commons-math3-3.1.1.jar
  /opt/application/Hadoop/current/share/hadoop/common/lib/gson-2.2.4.jar
  /opt/application/Hadoop/current/share/hadoop/common/lib/hamcrest-core-1.3.jar
  
/opt/application/Hadoop/current/share/hadoop/common/lib/curator-recipes-2.7.1.jar
  /opt/application/Hadoop/current/share/hadoop/common/lib/jets3t-0.9.0.jar
  
/opt/application/Hadoop/current/share/hadoop/common/lib/jackson-jaxrs-1.9.13.jar
  /opt/application/Hadoop/current/share/hadoop/common/lib/commons-net-3.1.jar
  /opt/application/Hadoop/current/share/hadoop/common/lib/commons-lang-2.6.jar
  
/opt/application/Hadoop/current/share/hadoop/common/lib/commons-compress-1.4.1.jar
  
/opt/application/Hadoop/current/share/hadoop/common/lib/curator-client-2.7.1.jar
  
/opt/application/Hadoop/current/share/hadoop/common/lib/jackson-core-asl-1.9.13.jar
  
/opt/application/Hadoop/current/share/hadoop/common/lib/commons-configuration-1.6.jar
  /opt/application/Hadoop/current/share/hadoop/hdfs/hadoop-hdfs-2.7.2.jar
  /opt/application/Hadoop/current/share/hadoop/hdfs/hadoop-hdfs-2.7.2-tests.jar
  /opt/application/Hadoop/current/share/hadoop/hdfs/hadoop-hdfs-nfs-2.7.2.jar
  /opt/application/Hadoop/current/share/hadoop/hdfs/lib/jersey-server-1.9.jar
  /opt/application/Hadoop/current/share/hadoop/hdfs/lib/guava-11.0.2.jar
  /opt/application/Hadoop/current/share/hadoop/hdfs/lib/commons-codec-1.4.jar
  /opt/application/Hadoop/current/share/hadoop/hdfs/lib/netty-3.6.2.Final.jar
  /opt/application/Hadoop/current/share/hadoop/hdfs/lib/asm-3.2.jar
  /opt/application/Hadoop/current/share/hadoop/hdfs/lib/jetty-util-6.1.26.jar
  /opt/application/Hadoop/current/share/hadoop/hdfs/lib/jetty-6.1.26.jar
  
/opt/application/Hadoop/current/share/hadoop/hdfs/lib/jackson-mapper-asl-1.9.13.jar
  /opt/application/Hadoop/current/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar
  /opt/application/Hadoop/current/share/hadoop/hdfs/lib/leveldbjni-all-1.8.jar
  
/opt/application/Hadoop/current/share/hadoop/hdfs/lib/netty-all-4.0.23.Final.jar
  
/opt/application/Hadoop/current/share/hadoop/hdfs/lib/htrace-core-3.1.0-incubating.jar
  /opt/application/Hadoop/current/share/hadoop/hdfs/lib/xmlenc-0.52.jar
  /opt/application/Hadoop/current/share/hadoop/hdfs/lib/commons-cli-1.2.jar
  /opt/application/Hadoop/current/share/hadoop/hdfs/lib/jersey-core-1.9.jar
  
/opt/application/Hadoop/current/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar
  /opt/application/Hadoop/current/share/hadoop/hdfs/lib/log4j-1.2.17.jar
  
/opt/application/Hadoop/current/share/hadoop/hdfs/lib/commons-logging-1.1.3.jar
  /opt/application/Hadoop/current/share/hadoop/hdfs/lib/servlet-api-2.5.jar
  /opt/application/Hadoop/current/share/hadoop/hdfs/lib/commons-io-2.4.jar
  /opt/application/Hadoop/current/share/hadoop/hdfs/lib/jsr305-3.0.0.jar
  /opt/application/Hadoop/current/share/hadoop/hdfs/lib/xercesImpl-2.9.1.jar
  /opt/application/Hadoop/current/share/hadoop/hdfs/lib/xml-apis-1.3.04.jar
  /opt/application/Hadoop/current/share/hadoop/hdfs/lib/commons-lang-2.6.jar
  
/opt/application/Hadoop/current/share/hadoop/hdfs/lib/jackson-core-asl-1.9.13.jar
  
/opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-2.7.2.jar
  
/opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-2.7.2.jar
  /opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-client-2.7.2.jar
  /opt/application/Hadoop/current/share/hadoop/yarn/spatial-sdk-hadoop.jar
    
/opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-2.7.2.jar
  
/opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.7.2.jar
  
/opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-server-common-2.7.2.jar
  
/opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-2.7.2.jar
  
/opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-server-web-proxy-2.7.2.jar
  /opt/application/Hadoop/current/share/hadoop/yarn/spatial-sdk-json.jar
  /opt/application/Hadoop/current/share/hadoop/yarn/esri-geometry-api.jar
  /opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-common-2.7.2.jar
  
/opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-server-sharedcachemanager-2.7.2.jar
  
/opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-server-tests-2.7.2.jar
  
/opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-registry-2.7.2.jar
  /opt/application/Hadoop/current/share/hadoop/yarn/spatial-sdk-hive.jar
  /opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-api-2.7.2.jar
  /opt/application/Hadoop/current/share/hadoop/yarn/lib/jersey-server-1.9.jar
  /opt/application/Hadoop/current/share/hadoop/yarn/lib/guava-11.0.2.jar
  /opt/application/Hadoop/current/share/hadoop/yarn/lib/jaxb-impl-2.2.3-1.jar
  /opt/application/Hadoop/current/share/hadoop/yarn/lib/commons-codec-1.4.jar
  /opt/application/Hadoop/current/share/hadoop/yarn/lib/netty-3.6.2.Final.jar
  /opt/application/Hadoop/current/share/hadoop/yarn/lib/asm-3.2.jar
  /opt/application/Hadoop/current/share/hadoop/yarn/lib/jetty-util-6.1.26.jar
  /opt/application/Hadoop/current/share/hadoop/yarn/lib/jetty-6.1.26.jar
  
/opt/application/Hadoop/current/share/hadoop/yarn/lib/jackson-mapper-asl-1.9.13.jar
  
/opt/application/Hadoop/current/share/hadoop/yarn/lib/commons-collections-3.2.2.jar
  /opt/application/Hadoop/current/share/hadoop/yarn/lib/protobuf-java-2.5.0.jar
  /opt/application/Hadoop/current/share/hadoop/yarn/lib/guice-servlet-3.0.jar
  /opt/application/Hadoop/current/share/hadoop/yarn/lib/jersey-guice-1.9.jar
  /opt/application/Hadoop/current/share/hadoop/yarn/lib/guice-3.0.jar
  /opt/application/Hadoop/current/share/hadoop/yarn/lib/zookeeper-3.4.6.jar
  /opt/application/Hadoop/current/share/hadoop/yarn/lib/leveldbjni-all-1.8.jar
  /opt/application/Hadoop/current/share/hadoop/yarn/lib/commons-cli-1.2.jar
  /opt/application/Hadoop/current/share/hadoop/yarn/lib/xz-1.0.jar
  /opt/application/Hadoop/current/share/hadoop/yarn/lib/jersey-core-1.9.jar
  /opt/application/Hadoop/current/share/hadoop/yarn/lib/stax-api-1.0-2.jar
  /opt/application/Hadoop/current/share/hadoop/yarn/lib/jersey-json-1.9.jar
  /opt/application/Hadoop/current/share/hadoop/yarn/lib/log4j-1.2.17.jar
  
/opt/application/Hadoop/current/share/hadoop/yarn/lib/commons-logging-1.1.3.jar
  /opt/application/Hadoop/current/share/hadoop/yarn/lib/jackson-xc-1.9.13.jar
  /opt/application/Hadoop/current/share/hadoop/yarn/lib/activation-1.1.jar
  /opt/application/Hadoop/current/share/hadoop/yarn/lib/jettison-1.1.jar
  /opt/application/Hadoop/current/share/hadoop/yarn/lib/servlet-api-2.5.jar
  /opt/application/Hadoop/current/share/hadoop/yarn/lib/jaxb-api-2.2.2.jar
  /opt/application/Hadoop/current/share/hadoop/yarn/lib/jersey-client-1.9.jar
  /opt/application/Hadoop/current/share/hadoop/yarn/lib/commons-io-2.4.jar
  /opt/application/Hadoop/current/share/hadoop/yarn/lib/jsr305-3.0.0.jar
  
/opt/application/Hadoop/current/share/hadoop/yarn/lib/zookeeper-3.4.6-tests.jar
  /opt/application/Hadoop/current/share/hadoop/yarn/lib/jackson-jaxrs-1.9.13.jar
  /opt/application/Hadoop/current/share/hadoop/yarn/lib/commons-lang-2.6.jar
  
/opt/application/Hadoop/current/share/hadoop/yarn/lib/commons-compress-1.4.1.jar
  
/opt/application/Hadoop/current/share/hadoop/yarn/lib/jackson-core-asl-1.9.13.jar
  /opt/application/Hadoop/current/share/hadoop/yarn/lib/aopalliance-1.0.jar
  /opt/application/Hadoop/current/share/hadoop/yarn/lib/javax.inject-1.jar
  
/opt/application/Hadoop/current/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-2.7.2.jar
  
/opt/application/Hadoop/current/share/hadoop/mapreduce/hadoop-mapreduce-client-common-2.7.2.jar
  
/opt/application/Hadoop/current/share/hadoop/mapreduce/hadoop-mapreduce-client-app-2.7.2.jar
  
/opt/application/Hadoop/current/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.7.2.jar
  
/opt/application/Hadoop/current/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.7.2.jar
  
/opt/application/Hadoop/current/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.7.2-tests.jar
  
/opt/application/Hadoop/current/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-2.7.2.jar
  
/opt/application/Hadoop/current/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-2.7.2.jar
  
/opt/application/Hadoop/current/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.2.jar
  
/opt/application/Hadoop/current/share/hadoop/mapreduce/lib/jersey-server-1.9.jar
  /opt/application/Hadoop/current/share/hadoop/mapreduce/lib/junit-4.11.jar
  
/opt/application/Hadoop/current/share/hadoop/mapreduce/lib/netty-3.6.2.Final.jar
  /opt/application/Hadoop/current/share/hadoop/mapreduce/lib/asm-3.2.jar
  
/opt/application/Hadoop/current/share/hadoop/mapreduce/lib/jackson-mapper-asl-1.9.13.jar
  
/opt/application/Hadoop/current/share/hadoop/mapreduce/lib/hadoop-annotations-2.7.2.jar
  
/opt/application/Hadoop/current/share/hadoop/mapreduce/lib/protobuf-java-2.5.0.jar
  
/opt/application/Hadoop/current/share/hadoop/mapreduce/lib/guice-servlet-3.0.jar
  /opt/application/Hadoop/current/share/hadoop/mapreduce/lib/paranamer-2.3.jar
  
/opt/application/Hadoop/current/share/hadoop/mapreduce/lib/jersey-guice-1.9.jar
  /opt/application/Hadoop/current/share/hadoop/mapreduce/lib/guice-3.0.jar
  
/opt/application/Hadoop/current/share/hadoop/mapreduce/lib/leveldbjni-all-1.8.jar
  /opt/application/Hadoop/current/share/hadoop/mapreduce/lib/xz-1.0.jar
  /opt/application/Hadoop/current/share/hadoop/mapreduce/lib/jersey-core-1.9.jar
  /opt/application/Hadoop/current/share/hadoop/mapreduce/lib/avro-1.7.4.jar
  /opt/application/Hadoop/current/share/hadoop/mapreduce/lib/log4j-1.2.17.jar
  
/opt/application/Hadoop/current/share/hadoop/mapreduce/lib/snappy-java-1.0.4.1.jar
  /opt/application/Hadoop/current/share/hadoop/mapreduce/lib/commons-io-2.4.jar
  
/opt/application/Hadoop/current/share/hadoop/mapreduce/lib/hamcrest-core-1.3.jar
  
/opt/application/Hadoop/current/share/hadoop/mapreduce/lib/commons-compress-1.4.1.jar
  
/opt/application/Hadoop/current/share/hadoop/mapreduce/lib/jackson-core-asl-1.9.13.jar
  /opt/application/Hadoop/current/share/hadoop/mapreduce/lib/aopalliance-1.0.jar
  /opt/application/Hadoop/current/share/hadoop/mapreduce/lib/javax.inject-1.jar
------------------------

Main class        : org.apache.oozie.action.hadoop.SparkMain

Maximum output    : 2048

Arguments         :
                    100

Java System Properties:
------------------------
#
#Fri May 20 17:41:46 CEST 2016
java.runtime.name=OpenJDK Runtime Environment
sun.boot.library.path=/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.79.x86_64/jre/lib/amd64
java.vm.version=24.79-b02
oozie.action.externalChildIDs=/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/externalChildIDs
hadoop.root.logger=INFO,CLA
java.vm.vendor=Oracle Corporation
java.vendor.url=http\://java.oracle.com/
path.separator=\:
java.vm.name=OpenJDK 64-Bit Server VM
file.encoding.pkg=sun.io
oozie.job.launch.time=1463758882528
user.country=US
sun.java.launcher=SUN_STANDARD
sun.os.patch.level=unknown
java.vm.specification.name=Java Virtual Machine Specification
user.dir=/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001
oozie.action.newId=/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/newId
java.runtime.version=1.7.0_79-mockbuild_2015_07_24_09_26-b00
java.awt.graphicsenv=sun.awt.X11GraphicsEnvironment
java.endorsed.dirs=/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.79.x86_64/jre/lib/endorsed
os.arch=amd64
oozie.job.id=0012689-160510172237486-oozie-W
oozie.action.id=0012689-160510172237486-oozie-W@spark-node
yarn.app.container.log.dir=/mnt/hd6/hadoop/yarn/log/application_1461692698792_29704/container_1461692698792_29704_01_000001
java.io.tmpdir=./tmp
line.separator=\n
oozie.action.output.properties=/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/output.properties
java.vm.specification.vendor=Oracle Corporation
os.name=Linux
log4j.configuration=container-log4j.properties
sun.jnu.encoding=UTF-8
java.library.path=/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001\:/usr/java/packages/lib/amd64\:/usr/lib64\:/lib64\:/lib\:/usr/lib
oozie.action.conf.xml=/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/action.xml
java.specification.name=Java Platform API Specification
java.class.version=51.0
sun.management.compiler=HotSpot 64-Bit Tiered Compilers
os.version=2.6.32-573.7.1.el6.x86_64
oozie.action.error.properties=/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/error.properties
yarn.app.container.log.filesize=0
user.home=/home/shfs3453
user.timezone=Europe/Paris
java.awt.printerjob=sun.print.PSPrinterJob
file.encoding=UTF-8
java.specification.version=1.7
java.class.path=/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001\:job.jar/job.jar\:job.jar/classes/\:job.jar/lib/*\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/jackson-databind-2.3.1.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/jets3t-0.7.1.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/paranamer-2.6.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/zookeeper-3.4.6.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/tachyon-client-0.5.0.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/spark-graphx_2.10-1.6.1.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/akka-actor_2.10-2.2.3-shaded-protobuf.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/spark-catalyst_2.10-1.6.1.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/jetty-webapp-8.1.14.v20131031.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/slf4j-api-1.6.6.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/javax.mail.glassfish-1.4.1.v201005082020.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/spark-mllib_2.10-1.6.1.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/jsr305-1.3.9.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/jetty-security-8.1.14.v20131031.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/spark-core_2.10-1.6.1.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/scala-compiler-2.10.0.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/protobuf-java-2.4.1-shaded.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/akka-remote_2.10-2.2.3-shaded-protobuf.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/metrics-jvm-3.0.2.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/metrics-graphite-3.0.0.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/netty-all-4.0.23.Final.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/objenesis-1.2.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/curator-recipes-2.5.0.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/chill-java-0.3.6.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/json4s-ast_2.10-3.2.10.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/commons-io-2.1.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/jetty-io-8.1.14.v20131031.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/jetty-continuation-8.1.14.v20131031.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/stream-2.7.0.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/minlog-1.2.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/netty-3.6.6.Final.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/jetty-util-8.1.14.v20131031.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/spark-yarn_2.10-1.6.1.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/scalap-2.10.0.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/json-simple-1.1.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/jetty-http-8.1.14.v20131031.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/pyrolite-2.0.1.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/log4j-1.2.16.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/jline-0.9.94.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/uncommons-maths-1.2.2a.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/slf4j-log4j12-1.6.6.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/py4j-0.8.2.1.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/json4s-jackson_2.10-3.2.10.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/metrics-core-3.0.2.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/jblas-1.2.3.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/curator-client-2.7.1.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/spark-sql_2.10-1.6.1.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/concurrent-1.3.4.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/jetty-jndi-8.1.14.v20131031.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/oozie-sharelib-spark-4.2.0.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/curator-framework-2.5.0.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/javax.activation-1.1.0.v201105071233.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/tachyon-0.6.4.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/reflectasm-1.07-shaded.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/spark-tools_2.10-1.6.1.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/spark-assembly-1.6.1-hadoop2.7.2.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/javax.servlet-3.0.0.v201112011016.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/jetty-servlet-8.1.14.v20131031.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/scala-library-2.10.4.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/jackson-annotations-2.3.0.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/akka-slf4j_2.10-2.2.3-shaded-protobuf.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/spark-network-yarn_2.10-1.6.1.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/commons-lang3-3.3.2.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/compress-lzf-1.0.0.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/spark-1.6.1-yarn-shuffle.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/jackson-core-2.3.1.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/json4s-core_2.10-3.2.10.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/jetty-plus-8.1.14.v20131031.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/config-1.0.2.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/oozie-sharelib-oozie-4.2.0.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/scala-reflect-2.10.0.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/jetty-xml-8.1.14.v20131031.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/jetty-server-8.1.14.v20131031.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/commons-lang-2.4.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/snappy-java-1.0.5.3.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/kryo-2.21.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/spark-streaming_2.10-1.6.1.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/commons-httpclient-3.1.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/guava-14.0.1.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/commons-codec-1.4.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/chill_2.10-0.3.6.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/lz4-1.2.0.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/javax.transaction-1.1.1.v201105210645.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/commons-net-2.2.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/spark-hive_2.10-1.6.1.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/metrics-json-3.0.2.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/colt-1.2.0.jar\:/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/oozie-hadoop-utils-hadoop-2-4.2.0.jar\:/opt/application/Hadoop/current/etc/hadoop/\:/opt/application/Hadoop/current/share/hadoop/common/hadoop-common-2.7.2.jar\:/opt/application/Hadoop/current/share/hadoop/common/hadoop-nfs-2.7.2.jar\:/opt/application/Hadoop/current/share/hadoop/common/hadoop-common-2.7.2-tests.jar\:/opt/application/Hadoop/current/share/hadoop/common/lib/jersey-server-1.9.jar\:/opt/application/Hadoop/current/share/hadoop/common/lib/guava-11.0.2.jar\:/opt/application/Hadoop/current/share/hadoop/common/lib/jsp-api-2.1.jar\:/opt/application/Hadoop/current/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar\:/opt/application/Hadoop/current/share/hadoop/common/lib/junit-4.11.jar\:/opt/application/Hadoop/current/share/hadoop/common/lib/commons-codec-1.4.jar\:/opt/application/Hadoop/current/share/hadoop/common/lib/netty-3.6.2.Final.jar\:/opt/application/Hadoop/current/share/hadoop/common/lib/httpcore-4.2.5.jar\:/opt/application/Hadoop/current/share/hadoop/common/lib/asm-3.2.jar\:/opt/application/Hadoop/current/share/hadoop/common/lib/jetty-util-6.1.26.jar\:/opt/application/Hadoop/current/share/hadoop/common/lib/curator-framework-2.7.1.jar\:/opt/application/Hadoop/current/share/hadoop/common/lib/commons-digester-1.8.jar\:/opt/application/Hadoop/current/share/hadoop/common/lib/jetty-6.1.26.jar\:/opt/application/Hadoop/current/share/hadoop/common/lib/jackson-mapper-asl-1.9.13.jar\:/opt/application/Hadoop/current/share/hadoop/common/lib/httpclient-4.2.5.jar\:/opt/application/Hadoop/current/share/hadoop/common/lib/commons-collections-3.2.2.jar\:/opt/application/Hadoop/current/share/hadoop/common/lib/hadoop-annotations-2.7.2.jar\:/opt/application/Hadoop/current/share/hadoop/common/lib/protobuf-java-2.5.0.jar\:/opt/application/Hadoop/current/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar\:/opt/application/Hadoop/current/share/hadoop/common/lib/hadoop-auth-2.7.2.jar\:/opt/application/Hadoop/current/share/hadoop/common/lib/paranamer-2.3.jar\:/opt/application/Hadoop/current/share/hadoop/common/lib/api-asn1-api-1.0.0-M20.jar\:/opt/application/Hadoop/current/share/hadoop/common/lib/zookeeper-3.4.6.jar\:/opt/application/Hadoop/current/share/hadoop/common/lib/htrace-core-3.1.0-incubating.jar\:/opt/application/Hadoop/current/share/hadoop/common/lib/xmlenc-0.52.jar\:/opt/application/Hadoop/current/share/hadoop/common/lib/commons-cli-1.2.jar\:/opt/application/Hadoop/current/share/hadoop/common/lib/xz-1.0.jar\:/opt/application/Hadoop/current/share/hadoop/common/lib/jsch-0.1.42.jar\:/opt/application/Hadoop/current/share/hadoop/common/lib/apacheds-kerberos-codec-2.0.0-M15.jar\:/opt/application/Hadoop/current/share/hadoop/common/lib/api-util-1.0.0-M20.jar\:/opt/application/Hadoop/current/share/hadoop/common/lib/jersey-core-1.9.jar\:/opt/application/Hadoop/current/share/hadoop/common/lib/stax-api-1.0-2.jar\:/opt/application/Hadoop/current/share/hadoop/common/lib/jersey-json-1.9.jar\:/opt/application/Hadoop/current/share/hadoop/common/lib/avro-1.7.4.jar\:/opt/application/Hadoop/current/share/hadoop/common/lib/log4j-1.2.17.jar\:/opt/application/Hadoop/current/share/hadoop/common/lib/commons-logging-1.1.3.jar\:/opt/application/Hadoop/current/share/hadoop/common/lib/jackson-xc-1.9.13.jar\:/opt/application/Hadoop/current/share/hadoop/common/lib/slf4j-api-1.7.10.jar\:/opt/application/Hadoop/current/share/hadoop/common/lib/java-xmlbuilder-0.4.jar\:/opt/application/Hadoop/current/share/hadoop/common/lib/activation-1.1.jar\:/opt/application/Hadoop/current/share/hadoop/common/lib/jettison-1.1.jar\:/opt/application/Hadoop/current/share/hadoop/common/lib/servlet-api-2.5.jar\:/opt/application/Hadoop/current/share/hadoop/common/lib/jaxb-api-2.2.2.jar\:/opt/application/Hadoop/current/share/hadoop/common/lib/apacheds-i18n-2.0.0-M15.jar\:/opt/application/Hadoop/current/share/hadoop/common/lib/commons-beanutils-core-1.8.0.jar\:/opt/application/Hadoop/current/share/hadoop/common/lib/commons-httpclient-3.1.jar\:/opt/application/Hadoop/current/share/hadoop/common/lib/snappy-java-1.0.4.1.jar\:/opt/application/Hadoop/current/share/hadoop/common/lib/commons-beanutils-1.7.0.jar\:/opt/application/Hadoop/current/share/hadoop/common/lib/commons-io-2.4.jar\:/opt/application/Hadoop/current/share/hadoop/common/lib/mockito-all-1.8.5.jar\:/opt/application/Hadoop/current/share/hadoop/common/lib/jsr305-3.0.0.jar\:/opt/application/Hadoop/current/share/hadoop/common/lib/commons-math3-3.1.1.jar\:/opt/application/Hadoop/current/share/hadoop/common/lib/gson-2.2.4.jar\:/opt/application/Hadoop/current/share/hadoop/common/lib/hamcrest-core-1.3.jar\:/opt/application/Hadoop/current/share/hadoop/common/lib/curator-recipes-2.7.1.jar\:/opt/application/Hadoop/current/share/hadoop/common/lib/jets3t-0.9.0.jar\:/opt/application/Hadoop/current/share/hadoop/common/lib/jackson-jaxrs-1.9.13.jar\:/opt/application/Hadoop/current/share/hadoop/common/lib/commons-net-3.1.jar\:/opt/application/Hadoop/current/share/hadoop/common/lib/commons-lang-2.6.jar\:/opt/application/Hadoop/current/share/hadoop/common/lib/commons-compress-1.4.1.jar\:/opt/application/Hadoop/current/share/hadoop/common/lib/curator-client-2.7.1.jar\:/opt/application/Hadoop/current/share/hadoop/common/lib/jackson-core-asl-1.9.13.jar\:/opt/application/Hadoop/current/share/hadoop/common/lib/commons-configuration-1.6.jar\:/opt/application/Hadoop/current/share/hadoop/hdfs/hadoop-hdfs-2.7.2.jar\:/opt/application/Hadoop/current/share/hadoop/hdfs/hadoop-hdfs-2.7.2-tests.jar\:/opt/application/Hadoop/current/share/hadoop/hdfs/hadoop-hdfs-nfs-2.7.2.jar\:/opt/application/Hadoop/current/share/hadoop/hdfs/lib/jersey-server-1.9.jar\:/opt/application/Hadoop/current/share/hadoop/hdfs/lib/guava-11.0.2.jar\:/opt/application/Hadoop/current/share/hadoop/hdfs/lib/commons-codec-1.4.jar\:/opt/application/Hadoop/current/share/hadoop/hdfs/lib/netty-3.6.2.Final.jar\:/opt/application/Hadoop/current/share/hadoop/hdfs/lib/asm-3.2.jar\:/opt/application/Hadoop/current/share/hadoop/hdfs/lib/jetty-util-6.1.26.jar\:/opt/application/Hadoop/current/share/hadoop/hdfs/lib/jetty-6.1.26.jar\:/opt/application/Hadoop/current/share/hadoop/hdfs/lib/jackson-mapper-asl-1.9.13.jar\:/opt/application/Hadoop/current/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar\:/opt/application/Hadoop/current/share/hadoop/hdfs/lib/leveldbjni-all-1.8.jar\:/opt/application/Hadoop/current/share/hadoop/hdfs/lib/netty-all-4.0.23.Final.jar\:/opt/application/Hadoop/current/share/hadoop/hdfs/lib/htrace-core-3.1.0-incubating.jar\:/opt/application/Hadoop/current/share/hadoop/hdfs/lib/xmlenc-0.52.jar\:/opt/application/Hadoop/current/share/hadoop/hdfs/lib/commons-cli-1.2.jar\:/opt/application/Hadoop/current/share/hadoop/hdfs/lib/jersey-core-1.9.jar\:/opt/application/Hadoop/current/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar\:/opt/application/Hadoop/current/share/hadoop/hdfs/lib/log4j-1.2.17.jar\:/opt/application/Hadoop/current/share/hadoop/hdfs/lib/commons-logging-1.1.3.jar\:/opt/application/Hadoop/current/share/hadoop/hdfs/lib/servlet-api-2.5.jar\:/opt/application/Hadoop/current/share/hadoop/hdfs/lib/commons-io-2.4.jar\:/opt/application/Hadoop/current/share/hadoop/hdfs/lib/jsr305-3.0.0.jar\:/opt/application/Hadoop/current/share/hadoop/hdfs/lib/xercesImpl-2.9.1.jar\:/opt/application/Hadoop/current/share/hadoop/hdfs/lib/xml-apis-1.3.04.jar\:/opt/application/Hadoop/current/share/hadoop/hdfs/lib/commons-lang-2.6.jar\:/opt/application/Hadoop/current/share/hadoop/hdfs/lib/jackson-core-asl-1.9.13.jar\:/opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-2.7.2.jar\:/opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-2.7.2.jar\:/opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-client-2.7.2.jar\:/opt/application/Hadoop/current/share/hadoop/yarn/spatial-sdk-hadoop.jar\:/opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-2.7.2.jar\:/opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.7.2.jar\:/opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-server-common-2.7.2.jar\:/opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-2.7.2.jar\:/opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-server-web-proxy-2.7.2.jar\:/opt/application/Hadoop/current/share/hadoop/yarn/spatial-sdk-json.jar\:/opt/application/Hadoop/current/share/hadoop/yarn/esri-geometry-api.jar\:/opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-common-2.7.2.jar\:/opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-server-sharedcachemanager-2.7.2.jar\:/opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-server-tests-2.7.2.jar\:/opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-registry-2.7.2.jar\:/opt/application/Hadoop/current/share/hadoop/yarn/spatial-sdk-hive.jar\:/opt/application/Hadoop/current/share/hadoop/yarn/hadoop-yarn-api-2.7.2.jar\:/opt/application/Hadoop/current/share/hadoop/yarn/lib/jersey-server-1.9.jar\:/opt/application/Hadoop/current/share/hadoop/yarn/lib/guava-11.0.2.jar\:/opt/application/Hadoop/current/share/hadoop/yarn/lib/jaxb-impl-2.2.3-1.jar\:/opt/application/Hadoop/current/share/hadoop/yarn/lib/commons-codec-1.4.jar\:/opt/application/Hadoop/current/share/hadoop/yarn/lib/netty-3.6.2.Final.jar\:/opt/application/Hadoop/current/share/hadoop/yarn/lib/asm-3.2.jar\:/opt/application/Hadoop/current/share/hadoop/yarn/lib/jetty-util-6.1.26.jar\:/opt/application/Hadoop/current/share/hadoop/yarn/lib/jetty-6.1.26.jar\:/opt/application/Hadoop/current/share/hadoop/yarn/lib/jackson-mapper-asl-1.9.13.jar\:/opt/application/Hadoop/current/share/hadoop/yarn/lib/commons-collections-3.2.2.jar\:/opt/application/Hadoop/current/share/hadoop/yarn/lib/protobuf-java-2.5.0.jar\:/opt/application/Hadoop/current/share/hadoop/yarn/lib/guice-servlet-3.0.jar\:/opt/application/Hadoop/current/share/hadoop/yarn/lib/jersey-guice-1.9.jar\:/opt/application/Hadoop/current/share/hadoop/yarn/lib/guice-3.0.jar\:/opt/application/Hadoop/current/share/hadoop/yarn/lib/zookeeper-3.4.6.jar\:/opt/application/Hadoop/current/share/hadoop/yarn/lib/leveldbjni-all-1.8.jar\:/opt/application/Hadoop/current/share/hadoop/yarn/lib/commons-cli-1.2.jar\:/opt/application/Hadoop/current/share/hadoop/yarn/lib/xz-1.0.jar\:/opt/application/Hadoop/current/share/hadoop/yarn/lib/jersey-core-1.9.jar\:/opt/application/Hadoop/current/share/hadoop/yarn/lib/stax-api-1.0-2.jar\:/opt/application/Hadoop/current/share/hadoop/yarn/lib/jersey-json-1.9.jar\:/opt/application/Hadoop/current/share/hadoop/yarn/lib/log4j-1.2.17.jar\:/opt/application/Hadoop/current/share/hadoop/yarn/lib/commons-logging-1.1.3.jar\:/opt/application/Hadoop/current/share/hadoop/yarn/lib/jackson-xc-1.9.13.jar\:/opt/application/Hadoop/current/share/hadoop/yarn/lib/activation-1.1.jar\:/opt/application/Hadoop/current/share/hadoop/yarn/lib/jettison-1.1.jar\:/opt/application/Hadoop/current/share/hadoop/yarn/lib/servlet-api-2.5.jar\:/opt/application/Hadoop/current/share/hadoop/yarn/lib/jaxb-api-2.2.2.jar\:/opt/application/Hadoop/current/share/hadoop/yarn/lib/jersey-client-1.9.jar\:/opt/application/Hadoop/current/share/hadoop/yarn/lib/commons-io-2.4.jar\:/opt/application/Hadoop/current/share/hadoop/yarn/lib/jsr305-3.0.0.jar\:/opt/application/Hadoop/current/share/hadoop/yarn/lib/zookeeper-3.4.6-tests.jar\:/opt/application/Hadoop/current/share/hadoop/yarn/lib/jackson-jaxrs-1.9.13.jar\:/opt/application/Hadoop/current/share/hadoop/yarn/lib/commons-lang-2.6.jar\:/opt/application/Hadoop/current/share/hadoop/yarn/lib/commons-compress-1.4.1.jar\:/opt/application/Hadoop/current/share/hadoop/yarn/lib/jackson-core-asl-1.9.13.jar\:/opt/application/Hadoop/current/share/hadoop/yarn/lib/aopalliance-1.0.jar\:/opt/application/Hadoop/current/share/hadoop/yarn/lib/javax.inject-1.jar\:/opt/application/Hadoop/current/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-2.7.2.jar\:/opt/application/Hadoop/current/share/hadoop/mapreduce/hadoop-mapreduce-client-common-2.7.2.jar\:/opt/application/Hadoop/current/share/hadoop/mapreduce/hadoop-mapreduce-client-app-2.7.2.jar\:/opt/application/Hadoop/current/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.7.2.jar\:/opt/application/Hadoop/current/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.7.2.jar\:/opt/application/Hadoop/current/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.7.2-tests.jar\:/opt/application/Hadoop/current/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-2.7.2.jar\:/opt/application/Hadoop/current/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-2.7.2.jar\:/opt/application/Hadoop/current/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.2.jar\:/opt/application/Hadoop/current/share/hadoop/mapreduce/lib/jersey-server-1.9.jar\:/opt/application/Hadoop/current/share/hadoop/mapreduce/lib/junit-4.11.jar\:/opt/application/Hadoop/current/share/hadoop/mapreduce/lib/netty-3.6.2.Final.jar\:/opt/application/Hadoop/current/share/hadoop/mapreduce/lib/asm-3.2.jar\:/opt/application/Hadoop/current/share/hadoop/mapreduce/lib/jackson-mapper-asl-1.9.13.jar\:/opt/application/Hadoop/current/share/hadoop/mapreduce/lib/hadoop-annotations-2.7.2.jar\:/opt/application/Hadoop/current/share/hadoop/mapreduce/lib/protobuf-java-2.5.0.jar\:/opt/application/Hadoop/current/share/hadoop/mapreduce/lib/guice-servlet-3.0.jar\:/opt/application/Hadoop/current/share/hadoop/mapreduce/lib/paranamer-2.3.jar\:/opt/application/Hadoop/current/share/hadoop/mapreduce/lib/jersey-guice-1.9.jar\:/opt/application/Hadoop/current/share/hadoop/mapreduce/lib/guice-3.0.jar\:/opt/application/Hadoop/current/share/hadoop/mapreduce/lib/leveldbjni-all-1.8.jar\:/opt/application/Hadoop/current/share/hadoop/mapreduce/lib/xz-1.0.jar\:/opt/application/Hadoop/current/share/hadoop/mapreduce/lib/jersey-core-1.9.jar\:/opt/application/Hadoop/current/share/hadoop/mapreduce/lib/avro-1.7.4.jar\:/opt/application/Hadoop/current/share/hadoop/mapreduce/lib/log4j-1.2.17.jar\:/opt/application/Hadoop/current/share/hadoop/mapreduce/lib/snappy-java-1.0.4.1.jar\:/opt/application/Hadoop/current/share/hadoop/mapreduce/lib/commons-io-2.4.jar\:/opt/application/Hadoop/current/share/hadoop/mapreduce/lib/hamcrest-core-1.3.jar\:/opt/application/Hadoop/current/share/hadoop/mapreduce/lib/commons-compress-1.4.1.jar\:/opt/application/Hadoop/current/share/hadoop/mapreduce/lib/jackson-core-asl-1.9.13.jar\:/opt/application/Hadoop/current/share/hadoop/mapreduce/lib/aopalliance-1.0.jar\:/opt/application/Hadoop/current/share/hadoop/mapreduce/lib/javax.inject-1.jar
user.name=shfs3453
java.vm.specification.version=1.7
sun.java.command=org.apache.hadoop.mapreduce.v2.app.MRAppMaster
java.home=/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.79.x86_64/jre
sun.arch.data.model=64
user.language=en
java.specification.vendor=Oracle Corporation
awt.toolkit=sun.awt.X11.XToolkit
java.vm.info=mixed mode
java.version=1.7.0_79
java.ext.dirs=/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.79.x86_64/jre/lib/ext\:/usr/java/packages/lib/ext
sun.boot.class.path=/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.79.x86_64/jre/lib/resources.jar\:/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.79.x86_64/jre/lib/rt.jar\:/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.79.x86_64/jre/lib/sunrsasign.jar\:/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.79.x86_64/jre/lib/jsse.jar\:/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.79.x86_64/jre/lib/jce.jar\:/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.79.x86_64/jre/lib/charsets.jar\:/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.79.x86_64/jre/lib/rhino.jar\:/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.79.x86_64/jre/lib/jfr.jar\:/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.79.x86_64/jre/classes
java.vendor=Oracle Corporation
file.separator=/
oozie.launcher.job.id=job_1461692698792_29704
oozie.action.stats.properties=/mnt/hd0/hadoop/yarn/local/usercache/shfs3453/appcache/application_1461692698792_29704/container_1461692698792_29704_01_000001/stats.properties
java.vendor.url.bug=http\://bugreport.sun.com/bugreport/
sun.io.unicode.encoding=UnicodeLittle
sun.cpu.endian=little
sun.cpu.isalist=
------------------------

=================================================================

>>> Invoking Main class now >>>

Fetching child yarn jobs
tag id : oozie-f217792cb72212277adf42b1fef23939
Child yarn jobs are found -
Spark Action Main class        : org.apache.spark.deploy.SparkSubmit

Oozie Spark action configuration
=================================================================


                    --master
                    yarn-cluster
                    --deploy-mode
                    cluster
                    --name
                    Pysparkpi example
                    --conf
                    
spark.yarn.appMasterEnv.SPARK_HOME=/opt/application/Spark/current
                    --conf
                    spark.executorEnv.SPARK_HOME=/opt/application/Spark/current
                    --conf
                    
spark.executorEnv.PYTHONPATH=/opt/application/Spark/current/python
                    --py-files
                    
hdfs://sandbox/User/shfs3453/WORK/Oozie/pyspark/lib/py4j-0.9-src.zip,hdfs://sandbox/User/shfs3453/WORK/Oozie/pyspark/lib/pyspark.zip
                    --verbose
                    hdfs://sandbox/User/shfs3453/WORK/Oozie/pyspark/lib/pi.py
                    100

=================================================================

>>> Invoking Spark class now >>>

<<< Invocation of Main class completed <<<

Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.SparkMain], 
main() threw exception, Application application_1461692698792_29705 finished 
with failed status
org.apache.spark.SparkException: Application application_1461692698792_29705 
finished with failed status
        at org.apache.spark.deploy.yarn.Client.run(Client.scala:1034)
        at org.apache.spark.deploy.yarn.Client$.main(Client.scala:1081)
        at org.apache.spark.deploy.yarn.Client.main(Client.scala)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at 
org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:731)
        at 
org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181)
        at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206)
        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)
        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
        at org.apache.oozie.action.hadoop.SparkMain.runSpark(SparkMain.java:104)
        at org.apache.oozie.action.hadoop.SparkMain.run(SparkMain.java:95)
        at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:47)
        at org.apache.oozie.action.hadoop.SparkMain.main(SparkMain.java:38)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at 
org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:236)
        at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
        at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
        at 
org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runSubtask(LocalContainerLauncher.java:380)
        at 
org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runTask(LocalContainerLauncher.java:301)
        at 
org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.access$200(LocalContainerLauncher.java:187)
        at 
org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler$1.run(LocalContainerLauncher.java:230)
        at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)

Oozie Launcher failed, finishing Hadoop job gracefully

Oozie Launcher, uploading action data to HDFS sequence file: 
hdfs://sandbox/user/shfs3453/oozie/0012689-160510172237486-oozie-W/spark-node--spark/action-data.seq

Oozie Launcher ends
{noformat}

> Pyspark job fails with Oozie
> ----------------------------
>
>                 Key: OOZIE-2482
>                 URL: https://issues.apache.org/jira/browse/OOZIE-2482
>             Project: Oozie
>          Issue Type: Bug
>          Components: core, workflow
>    Affects Versions: 4.2.0
>         Environment: Hadoop 2.7.2, Spark 1.6.0 on Yarn, Oozie 4.2.0
> Cluster secured with Kerberos
>            Reporter: Alexandre Linte
>            Assignee: Satish Subhashrao Saley
>         Attachments: OOZIE-2482-1.patch, OOZIE-2482-2.patch, 
> OOZIE-2482-zip.patch, py4j-0.9-src.zip, pyspark.zip
>
>
> Hello,
> I'm trying to run pi.py example in a pyspark job with Oozie. Every try I made 
> failed for the same reason: key not found: SPARK_HOME.
> Note: A scala job works well in the environment with Oozie.
> The logs on the executors are:
> {noformat}
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in 
> [jar:file:/mnt/hd4/hadoop/yarn/local/filecache/145/slf4j-log4j12-1.6.6.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in 
> [jar:file:/mnt/hd2/hadoop/yarn/local/filecache/155/spark-assembly-1.6.0-hadoop2.7.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in 
> [jar:file:/opt/application/Hadoop/hadoop-2.7.2/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an 
> explanation.
> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
> log4j:ERROR setFile(null,true) call failed.
> java.io.FileNotFoundException: 
> /mnt/hd7/hadoop/yarn/log/application_1454673025841_13136/container_1454673025841_13136_01_000001
>  (Is a directory)
>         at java.io.FileOutputStream.open(Native Method)
>         at java.io.FileOutputStream.<init>(FileOutputStream.java:221)
>         at java.io.FileOutputStream.<init>(FileOutputStream.java:142)
>         at org.apache.log4j.FileAppender.setFile(FileAppender.java:294)
>         at 
> org.apache.log4j.FileAppender.activateOptions(FileAppender.java:165)
>         at 
> org.apache.hadoop.yarn.ContainerLogAppender.activateOptions(ContainerLogAppender.java:55)
>         at 
> org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:307)
>         at 
> org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:172)
>         at 
> org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:104)
>         at 
> org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:809)
>         at 
> org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:735)
>         at 
> org.apache.log4j.PropertyConfigurator.configureRootCategory(PropertyConfigurator.java:615)
>         at 
> org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:502)
>         at 
> org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:547)
>         at 
> org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:483)
>         at org.apache.log4j.LogManager.<clinit>(LogManager.java:127)
>         at 
> org.slf4j.impl.Log4jLoggerFactory.getLogger(Log4jLoggerFactory.java:64)
>         at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:285)
>         at 
> org.apache.commons.logging.impl.SLF4JLogFactory.getInstance(SLF4JLogFactory.java:155)
>         at 
> org.apache.commons.logging.impl.SLF4JLogFactory.getInstance(SLF4JLogFactory.java:132)
>         at org.apache.commons.logging.LogFactory.getLog(LogFactory.java:275)
>         at 
> org.apache.hadoop.service.AbstractService.<clinit>(AbstractService.java:43)
> Using properties file: null
> Parsed arguments:
>   master                  yarn-master
>   deployMode              cluster
>   executorMemory          null
>   executorCores           null
>   totalExecutorCores      null
>   propertiesFile          null
>   driverMemory            null
>   driverCores             null
>   driverExtraClassPath    null
>   driverExtraLibraryPath  null
>   driverExtraJavaOptions  null
>   supervise               false
>   queue                   null
>   numExecutors            null
>   files                   null
>   pyFiles                 null
>   archives                null
>   mainClass               null
>   primaryResource         
> hdfs://hadoopsandbox/User/toto/WORK/Oozie/pyspark/lib/pi.py
>   name                    Pysparkpi example
>   childArgs               [100]
>   jars                    null
>   packages                null
>   packagesExclusions      null
>   repositories            null
>   verbose                 true
> Spark properties used, including those specified through
>  --conf and those from the properties file null:
>   spark.executorEnv.SPARK_HOME -> /opt/application/Spark/current
>   spark.executorEnv.PYTHONPATH -> /opt/application/Spark/current/python
>   spark.yarn.appMasterEnv.SPARK_HOME -> /opt/application/Spark/current
> Main class:
> org.apache.spark.deploy.yarn.Client
> Arguments:
> --name
> Pysparkpi example
> --primary-py-file
> hdfs://hadoopsandbox/User/toto/WORK/Oozie/pyspark/lib/pi.py
> --class
> org.apache.spark.deploy.PythonRunner
> --arg
> 100
> System properties:
> spark.executorEnv.SPARK_HOME -> /opt/application/Spark/current
> spark.executorEnv.PYTHONPATH -> /opt/application/Spark/current/python
> SPARK_SUBMIT -> true
> spark.app.name -> Pysparkpi example
> spark.submit.deployMode -> cluster
> spark.yarn.appMasterEnv.SPARK_HOME -> /opt/application/Spark/current
> spark.yarn.isPython -> true
> spark.master -> yarn-cluster
> Classpath elements:
> Failing Oozie Launcher, Main class 
> [org.apache.oozie.action.hadoop.SparkMain], main() threw exception, key not 
> found: SPARK_HOME
> java.util.NoSuchElementException: key not found: SPARK_HOME
>         at scala.collection.MapLike$class.default(MapLike.scala:228)
>         at scala.collection.AbstractMap.default(Map.scala:58)
>         at scala.collection.MapLike$class.apply(MapLike.scala:141)
>         at scala.collection.AbstractMap.apply(Map.scala:58)
>         at 
> org.apache.spark.deploy.yarn.Client$$anonfun$findPySparkArchives$2.apply(Client.scala:1045)
>         at 
> org.apache.spark.deploy.yarn.Client$$anonfun$findPySparkArchives$2.apply(Client.scala:1044)
>         at scala.Option.getOrElse(Option.scala:120)
>         at 
> org.apache.spark.deploy.yarn.Client.findPySparkArchives(Client.scala:1044)
>         at 
> org.apache.spark.deploy.yarn.Client.createContainerLaunchContext(Client.scala:717)
>         at 
> org.apache.spark.deploy.yarn.Client.submitApplication(Client.scala:142)
>         at org.apache.spark.deploy.yarn.Client.run(Client.scala:1016)
>         at org.apache.spark.deploy.yarn.Client$.main(Client.scala:1076)
>         at org.apache.spark.deploy.yarn.Client.main(Client.scala)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>         at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:606)
>         at 
> org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:731)
>         at 
> org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181)
>         at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206)
>         at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)
>         at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
>         at 
> org.apache.oozie.action.hadoop.SparkMain.runSpark(SparkMain.java:104)
>         at org.apache.oozie.action.hadoop.SparkMain.run(SparkMain.java:95)
>         at 
> org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:47)
>         at org.apache.oozie.action.hadoop.SparkMain.main(SparkMain.java:38)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>         at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:606)
>         at 
> org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:236)
>         at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
>         at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
>         at 
> org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runSubtask(LocalContainerLauncher.java:380)
>         at 
> org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runTask(LocalContainerLauncher.java:301)
>         at 
> org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.access$200(LocalContainerLauncher.java:187)
>         at 
> org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler$1.run(LocalContainerLauncher.java:230)
>         at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>         at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>         at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:745)
> log4j:WARN No appenders could be found for logger 
> (org.apache.hadoop.mapreduce.v2.app.MRAppMaster).
> log4j:WARN Please initialize the log4j system properly.
> log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more 
> info.
> {noformat}
> The workflow used for Oozie is the following:
> {noformat}
> <workflow-app xmlns='uri:oozie:workflow:0.5' name='PysparkPi-test'>
>         <start to='spark-node' />
>         <action name='spark-node'>
>                 <spark xmlns="uri:oozie:spark-action:0.1">
>                         <job-tracker>${jobTracker}</job-tracker>
>                         <name-node>${nameNode}</name-node>
>                         <master>${master}</master>
>                         <mode>${mode}</mode>
>                         <name>Pysparkpi example</name>
>                         <class></class>
>                         
> <jar>${nameNode}/User/toto/WORK/Oozie/pyspark/lib/pi.py</jar>
>                         <spark-opts>--conf 
> spark.yarn.appMasterEnv.SPARK_HOME=/opt/application/Spark/current --conf 
> spark.executorEnv.SPARK_HOME=/opt/application/Spark/current --conf 
> spark.executorEnv.PYTHONPATH=/opt/application/Spark/current/python</spark-opts>
>                         <arg>100</arg>
>                 </spark>
>                 <ok to="end" />
>                 <error to="fail" />
>         </action>
>         <kill name="fail">
>                 <message>Workflow failed, error 
> message[${wf:errorMessage(wf:lastErrorNode())}]</message>
>         </kill>
>         <end name='end' />
> </workflow-app>
> {noformat}
> I also created a JIRA for Spark: 
> [https://issues.apache.org/jira/browse/SPARK-13679]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (OOZIE-2482) Pyspark job fails with Oozie

Reply via email to