The purpose of broadcast variable is different. @Malveeka, could you please explain your usecase and issue. If the FAT/ Uber jar is not having required dependent jars then the spark job will fail at the start itself.
What is your scenario in which you want to add new jars? Also, what you mean by adding spark.jars in middle? (Did you mean middle of processing) then you can not change spark jars in middle of execution. However, you can change them for next processing iteration. On Wed, May 23, 2018 at 5:39 PM, kedarsdixit <kedarnath_di...@persistent.com > wrote: > In case of already running jobs, you can make use of broadcasters which > will > broadcast the jars to workers, if you want to change it on the fly you can > rebroadcast it. > > You can explore broadcasters bit more to make use of. > > Regards, > Kedar Dixit > Data Science at Persistent Systems Ltd. > > > > -- > Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/ > > --------------------------------------------------------------------- > To unsubscribe e-mail: user-unsubscr...@spark.apache.org > >
