Thomas created SPARK-10375: ------------------------------ Summary: Setting the driver memory with SparkConf().set("spark.driver.memory","1g") does not work Key: SPARK-10375 URL: https://issues.apache.org/jira/browse/SPARK-10375 Project: Spark Issue Type: Bug Components: PySpark Affects Versions: 1.3.0 Environment: Running with yarn Reporter: Thomas Priority: Minor
When running pyspark 1.3.0 with yarn, the following code has no effect: pyspark.SparkConf().set("spark.driver.memory","1g") The Environment tab in yarn shows that the driver has 1g, however, the Executors tab only shows 512 M (the default value) for the driver memory. This issue goes away when the driver memory is specified via the command line (i.e. --driver-memory 1g) -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org