[ https://issues.apache.org/jira/browse/SPARK-1719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14137176#comment-14137176 ]
Wilfred Spiegelenburg commented on SPARK-1719: ---------------------------------------------- oops, not sure how I missed that last file change :-( > spark.executor.extraLibraryPath isn't applied on yarn > ----------------------------------------------------- > > Key: SPARK-1719 > URL: https://issues.apache.org/jira/browse/SPARK-1719 > Project: Spark > Issue Type: Sub-task > Components: YARN > Affects Versions: 1.0.0 > Reporter: Thomas Graves > Assignee: Guoqiang Li > Fix For: 1.2.0 > > > Looking through the code for spark on yarn I don't see that > spark.executor.extraLibraryPath is being properly applied when it launches > executors. It is using the spark.driver.libraryPath in the ClientBase. > Note I didn't actually test it so its possible I missed something. > I also think better to use LD_LIBRARY_PATH rather then -Djava.library.path. > once java.library.path is set, it doesn't search LD_LIBRARY_PATH. In Hadoop > we switched to use LD_LIBRARY_PATH instead of java.library.path. See > https://issues.apache.org/jira/browse/MAPREDUCE-4072. I'll split this into > separate jira. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org