[jira] [Assigned] (SPARK-10839) SPARK_DAEMON_MEMORY has effect on heap size of thriftserver
[ https://issues.apache.org/jira/browse/SPARK-10839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10839: Assignee: (was: Apache Spark) > SPARK_DAEMON_MEMORY has effect on heap size of thriftserver > --- > > Key: SPARK-10839 > URL: https://issues.apache.org/jira/browse/SPARK-10839 > Project: Spark > Issue Type: Bug > Components: Spark Submit >Affects Versions: 1.4.1, 1.5.0 >Reporter: Yun Zhao > > When SPARK_DAEMON_MEMORY in spark-env.sh is setted to modify memory of Master > or Worker, there's an effect on heap size of thriftserver, further, this > effect cannot be modified by spark.driver.memory or --driver-memory. Version > 1.3.1 does not have the same problem. > in org.apache.spark.launcher.SparkSubmitCommandBuilder: > {quote} > String tsMemory = > isThriftServer(mainClass) ? System.getenv("SPARK_DAEMON_MEMORY") : > null; > String memory = firstNonEmpty(tsMemory, > firstNonEmptyValue(SparkLauncher.DRIVER_MEMORY, conf, props), > System.getenv("SPARK_DRIVER_MEMORY"), System.getenv("SPARK_MEM"), > DEFAULT_MEM); > cmd.add("-Xms" + memory); > cmd.add("-Xmx" + memory); > {quote} > SPARK_DAEMON_MEMORY has the highest priority. > It can be modified like this: > {quote} > String memory = firstNonEmpty(firstNonEmptyValue(SparkLauncher.DRIVER_MEMORY, > conf, props), > System.getenv("SPARK_DRIVER_MEMORY"), tsMemory, > System.getenv("SPARK_MEM"), DEFAULT_MEM); > {quote} -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-10839) SPARK_DAEMON_MEMORY has effect on heap size of thriftserver
[ https://issues.apache.org/jira/browse/SPARK-10839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10839: Assignee: Apache Spark > SPARK_DAEMON_MEMORY has effect on heap size of thriftserver > --- > > Key: SPARK-10839 > URL: https://issues.apache.org/jira/browse/SPARK-10839 > Project: Spark > Issue Type: Bug > Components: Spark Submit >Affects Versions: 1.4.1, 1.5.0 >Reporter: Yun Zhao >Assignee: Apache Spark > > When SPARK_DAEMON_MEMORY in spark-env.sh is setted to modify memory of Master > or Worker, there's an effect on heap size of thriftserver, further, this > effect cannot be modified by spark.driver.memory or --driver-memory. Version > 1.3.1 does not have the same problem. > in org.apache.spark.launcher.SparkSubmitCommandBuilder: > {quote} > String tsMemory = > isThriftServer(mainClass) ? System.getenv("SPARK_DAEMON_MEMORY") : > null; > String memory = firstNonEmpty(tsMemory, > firstNonEmptyValue(SparkLauncher.DRIVER_MEMORY, conf, props), > System.getenv("SPARK_DRIVER_MEMORY"), System.getenv("SPARK_MEM"), > DEFAULT_MEM); > cmd.add("-Xms" + memory); > cmd.add("-Xmx" + memory); > {quote} > SPARK_DAEMON_MEMORY has the highest priority. > It can be modified like this: > {quote} > String memory = firstNonEmpty(firstNonEmptyValue(SparkLauncher.DRIVER_MEMORY, > conf, props), > System.getenv("SPARK_DRIVER_MEMORY"), tsMemory, > System.getenv("SPARK_MEM"), DEFAULT_MEM); > {quote} -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org