Re: spark-submit can find python?

2018-01-15 Thread Jeff Zhang
legatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > > at java.lang.reflect.Method.invoke(Method.java:498) > > at > org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:782) > > at >

RE: spark-submit can find python?

2018-01-15 Thread Manuel Sopena Ballesteros
p/appcache/application_1512016123441_0045/container_1512016123441_0045_02_01/tmp/1516059780057-0/bin/python? Who copies it and from where? And what do I need to do in order to make my spark-submit job to run? Thank you Manuel From: Manuel Sopena Ballesteros Sent: Tuesday, January 16, 2018

spark-submit can find python?

2018-01-15 Thread Manuel Sopena Ballesteros
Hi all, I am quite new to spark and need some help troubleshooting the execution of an application running on a spark cluster... My spark environment is deployed using Ambari (HDP), YARM is the resource scheduler and hadoop as file system. The application I am trying to run is a python script