[ https://issues.apache.org/jira/browse/SPARK-11099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jeff Zhang updated SPARK-11099: ------------------------------- Description: spark.driver.extraClassPath doesn't take effect in the latest code, and find the root cause is due to the default conf property file is not loaded The bug is caused by this code snippet in AbstractCommandBuilder {code} Map<String, String> getEffectiveConfig() throws IOException { if (effectiveConfig == null) { if (propertiesFile == null) { effectiveConfig = conf; // return from here if no propertyFile is provided } else { effectiveConfig = new HashMap<>(conf); Properties p = loadPropertiesFile(); // default propertyFile will load here for (String key : p.stringPropertyNames()) { if (!effectiveConfig.containsKey(key)) { effectiveConfig.put(key, p.getProperty(key)); } } } } return effectiveConfig; } {code} was: spark.driver.extraClassPath doesn't take effect in the latest code, and find the root cause is due to the default conf property file is not loaded The bug is caused by this code snippet in AbstractCommandBuilder {code} Map<String, String> getEffectiveConfig() throws IOException { if (effectiveConfig == null) { if (propertiesFile == null) { effectiveConfig = conf; } else { effectiveConfig = new HashMap<>(conf); Properties p = loadPropertiesFile(); for (String key : p.stringPropertyNames()) { if (!effectiveConfig.containsKey(key)) { effectiveConfig.put(key, p.getProperty(key)); } } } } return effectiveConfig; } {code} > Default conf property file is not loaded > ----------------------------------------- > > Key: SPARK-11099 > URL: https://issues.apache.org/jira/browse/SPARK-11099 > Project: Spark > Issue Type: Bug > Components: Spark Shell, Spark Submit > Reporter: Jeff Zhang > Priority: Critical > > spark.driver.extraClassPath doesn't take effect in the latest code, and find > the root cause is due to the default conf property file is not loaded > The bug is caused by this code snippet in AbstractCommandBuilder > {code} > Map<String, String> getEffectiveConfig() throws IOException { > if (effectiveConfig == null) { > if (propertiesFile == null) { > effectiveConfig = conf; // return from here if no propertyFile > is provided > } else { > effectiveConfig = new HashMap<>(conf); > Properties p = loadPropertiesFile(); // default propertyFile > will load here > for (String key : p.stringPropertyNames()) { > if (!effectiveConfig.containsKey(key)) { > effectiveConfig.put(key, p.getProperty(key)); > } > } > } > } > return effectiveConfig; > } > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org