[
https://issues.apache.org/jira/browse/LIVY-487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16760032#comment-16760032
]
Ruslan Dautkhanov commented on LIVY-487:
----------------------------------------
If you specify dependencies to spark-submit as `--jars`, they get distributed
to executors (and driver in yarn-cluster mode). They are stored in a temp
directory specific for that yarn job..
I thought Livy uses the same approach..
How did you add dependencies to your job?
> Livy remote jars and files incorrectly specified on classpath
> -------------------------------------------------------------
>
> Key: LIVY-487
> URL: https://issues.apache.org/jira/browse/LIVY-487
> Project: Livy
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.5.0
> Reporter: Tim Harsch
> Priority: Critical
>
> From a thread in users-livy group:
> If I submit jars using a client machine that is not the same as the server
> (so file system is not shared) and I submit the jar it’s looks like it gets
> transmitted to the server but then I can’t reference any of the classes. *If
> I look at the Spark UI that gets launched for the session then I can see
> references to paths in my client machine*, which aren’t resolvable on the
> server. Is this the scenario you have? I think it might occur in a docker
> env if you are doing client stuff on the host os, and livy is running in a
> container.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
