Hi Mitch your jar file can be anywhere in the file system, including hdfs.
If using yarn, preferably use cluster mode in terms of deployment. Yarn will distribute the jar to each container. Best Ayan On Tue, 18 Jul 2017 at 2:17 am, Marcelo Vanzin <van...@cloudera.com> wrote: > Spark distributes your application jar for you. > > On Mon, Jul 17, 2017 at 8:41 AM, Mich Talebzadeh > <mich.talebza...@gmail.com> wrote: > > hi guys, > > > > > > an uber/fat jar file has been created to run with spark in CDH yarc > client > > mode. > > > > As usual job is submitted to the edge node. > > > > does the jar file has to be placed in the same directory ewith spark is > > running in the cluster to make it work? > > > > Also what will happen if say out of 9 nodes running spark, 3 have not got > > the jar file. will that job fail or it will carry on on the fremaing 6 > nodes > > that have that jar file? > > > > thanks > > > > Dr Mich Talebzadeh > > > > > > > > LinkedIn > > > https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw > > > > > > > > http://talebzadehmich.wordpress.com > > > > > > Disclaimer: Use it at your own risk. Any and all responsibility for any > > loss, damage or destruction of data or any other property which may arise > > from relying on this email's technical content is explicitly disclaimed. > The > > author will in no case be liable for any monetary damages arising from > such > > loss, damage or destruction. > > > > > > > > -- > Marcelo > > --------------------------------------------------------------------- > To unsubscribe e-mail: user-unsubscr...@spark.apache.org > > -- Best Regards, Ayan Guha
