I guess I misread your message. The archive directory shall contain
only jar files, not tar.gz files...
On 11/14/20 10:11 AM, Artemis User wrote:
Assuming you were using hadoop for your yarn cluster. You can specify
the spark parameters spark.yarn.archive or spark.yarn.jars to contain
the
Assuming you were using hadoop for your yarn cluster. You can specify
the spark parameters spark.yarn.archive or spark.yarn.jars to contain
the jar directory or jar files so that hadoop can find them by default.
See Spark online doc for details
(http://spark.apache.org/docs/latest/running-on-
Hello,
I am submitting a spark application on spark yarn using the cluster
execution mode.
The application itself depends on a couple of jars. I can successfully
submit and run the application using spark-submit --jars option as seen
below:
spark-submit \
--name Yarn-App \
--class \
--properties
