PM
>
> *To:* Jan Botorek <jan.boto...@infor.com>
> *Cc:* Vinod Mangipudi <vinod...@gmail.com>; user <user@spark.apache.org>
> *Subject:* Re: Add jar files on classpath when submitting tasks to Spark
>
>
>
> If you are using local mode then there is
il.com>; user <user@spark.apache.org>
Subject: Re: Add jar files on classpath when submitting tasks to Spark
If you are using local mode then there is only one JVM. In Linux as below mine
looks like this
${SPARK_HOME}/bin/spark-submit \
--packages ${PACKAGES} \
submit --class com.infor.skyvault.tests.LinearRegressionTest
> --master local C:\_resources\spark-1.0-SNAPSHOT.jar
> -DtrainDataPath="/path/to/model/data"
>
>
>
>
>
>
>
> *From:* Mich Talebzadeh [mailto:mich.talebza...@gmail.com]
> *Sent:* Tuesday, November 1, 2016 2:
2016 2:51 PM
To: Jan Botorek <jan.boto...@infor.com>
Cc: Vinod Mangipudi <vinod...@gmail.com>; user <user@spark.apache.org>
Subject: Re: Add jar files on classpath when submitting tasks to Spark
Are you submitting your job through spark-submit?
Dr Mich Talebzadeh
LinkedI
alebzadeh [mailto:mich.talebza...@gmail.com]
> *Sent:* Tuesday, November 1, 2016 2:18 PM
> *To:* Vinod Mangipudi <vinod...@gmail.com>
>
> *Cc:* user <user@spark.apache.org>
> *Subject:* Re: Add jar files on classpath when submitting tasks to Sp
yan guha [mailto:guha.a...@gmail.com<mailto:guha.a...@gmail.com>]
Sent: Tuesday, November 1, 2016 1:49 PM
To: Jan Botorek <jan.boto...@infor.com<mailto:jan.boto...@infor.com>>
Cc: user <user@spark.apache.org<mailto:user@spark.apache.org>>
Subject: Re: Add jar files on class
s
>> „in global“ - in the spark-defaults.conf??
>>
>>
>>
>>
>>
>> *From:* ayan guha [mailto:guha.a...@gmail.com]
>> *Sent:* Tuesday, November 1, 2016 1:49 PM
>> *To:* Jan Botorek <jan.boto...@infor.com>
>> *Cc:* user <user@s
location (directory) for jars
> „in global“ - in the spark-defaults.conf??
>
>
>
>
>
> *From:* ayan guha [mailto:guha.a...@gmail.com]
> *Sent:* Tuesday, November 1, 2016 1:49 PM
> *To:* Jan Botorek <jan.boto...@infor.com>
> *Cc:* user <user@spark.apache.org>
&g
, November 1, 2016 1:49 PM
To: Jan Botorek <jan.boto...@infor.com>
Cc: user <user@spark.apache.org>
Subject: Re: Add jar files on classpath when submitting tasks to Spark
There are options to specify external jars in the form of --jars,
--driver-classpath etc depending on spark version
There are options to specify external jars in the form of --jars,
--driver-classpath etc depending on spark version and cluster manager..
Please see spark documents for configuration sections and/or run spark
submit help to see available options.
On 1 Nov 2016 23:13, "Jan Botorek"
Hello,
I have a problem trying to add jar files to be available on classpath when
submitting task to Spark.
In my spark-defaults.conf file I have configuration:
spark.driver.extraClassPath = path/to/folder/with/jars
all jars in the folder are available in SPARK-SHELL
The problem is that jars
11 matches
Mail list logo