Thank you for the reply. I am aware of the parameters used when submitting the tasks (--jars is working for us).
But, isn’t there any way how to specify a location (directory) for jars „in global“ - in the spark-defaults.conf?? From: ayan guha [mailto:guha.a...@gmail.com] Sent: Tuesday, November 1, 2016 1:49 PM To: Jan Botorek <jan.boto...@infor.com> Cc: user <user@spark.apache.org> Subject: Re: Add jar files on classpath when submitting tasks to Spark There are options to specify external jars in the form of --jars, --driver-classpath etc depending on spark version and cluster manager.. Please see spark documents for configuration sections and/or run spark submit help to see available options. On 1 Nov 2016 23:13, "Jan Botorek" <jan.boto...@infor.com<mailto:jan.boto...@infor.com>> wrote: Hello, I have a problem trying to add jar files to be available on classpath when submitting task to Spark. In my spark-defaults.conf file I have configuration: spark.driver.extraClassPath = path/to/folder/with/jars all jars in the folder are available in SPARK-SHELL The problem is that jars are not on the classpath for SPARK-MASTER; more precisely – when I submit any job that utilizes any jar from external folder, the java.lang.ClassNotFoundException is thrown. Moving all external jars into the jars folder solves the situation, but we need to keep external files separatedly. Thank you for any help Best regards, Jan
