[ https://issues.apache.org/jira/browse/SPARK-3684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrew Or closed SPARK-3684. ---------------------------- Resolution: Won't Fix > Can't configure local dirs in Yarn mode > --------------------------------------- > > Key: SPARK-3684 > URL: https://issues.apache.org/jira/browse/SPARK-3684 > Project: Spark > Issue Type: Bug > Components: YARN > Affects Versions: 1.1.0 > Reporter: Andrew Or > > We can't set SPARK_LOCAL_DIRS or spark.local.dirs because they're not picked > up in Yarn mode. However, we can't set YARN_LOCAL_DIRS or LOCAL_DIRS either > because these are overridden by Yarn. > I'm trying to set these through SPARK_YARN_USER_ENV. I'm aware that the > default behavior is for Spark to use Yarn's local dirs, but right now there's > no way to change it even if the user wants to. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org