[jira] [Commented] (SPARK-2744) The configuration spark.history.retainedApplications is invalid
[
https://issues.apache.org/jira/browse/SPARK-2744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14081191#comment-14081191
]
Marcelo Vanzin commented on SPARK-2744:
---
Sort of. The docs say:
{quote}
The number of application UIs to retain.
{quote}
That's still true. Only the configured number of UIs are kept in memory. The
newer will list all available applications, though, while just keeping a few
UIs in memory. So it's more useful, since you're able to browse more history.
The configuration spark.history.retainedApplications is invalid
-
Key: SPARK-2744
URL: https://issues.apache.org/jira/browse/SPARK-2744
Project: Spark
Issue Type: Bug
Components: Spark Core
Reporter: meiyoula
Labels: historyserver
when I set it in spark-env.sh like this:export
SPARK_HISTORY_OPTS=$SPARK_HISTORY_OPTS -Dspark.history.ui.port=5678
-Dspark.history.retainedApplications=1 , the web of historyserver retains
more than one application
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (SPARK-2744) The configuration spark.history.retainedApplications is invalid
[ https://issues.apache.org/jira/browse/SPARK-2744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14079497#comment-14079497 ] Marcelo Vanzin commented on SPARK-2744: --- Are you sure that option means what you think it means? The History Server will list all applications. It will just retain a max number of them *in memory*. That option does not control how many applications are show, it controls how much memory the HS will need. The configuration spark.history.retainedApplications is invalid - Key: SPARK-2744 URL: https://issues.apache.org/jira/browse/SPARK-2744 Project: Spark Issue Type: Bug Components: Spark Core Reporter: meiyoula Labels: historyserver when I set it in spark-env.sh like this:export SPARK_HISTORY_OPTS=$SPARK_HISTORY_OPTS -Dspark.history.ui.port=5678 -Dspark.history.retainedApplications=1 , the web of historyserver retains more than one application -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (SPARK-2744) The configuration spark.history.retainedApplications is invalid
[ https://issues.apache.org/jira/browse/SPARK-2744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14080368#comment-14080368 ] meiyoula commented on SPARK-2744: - Has the meaning been changed? In the release version of spark, it controls how many applications are show. The configuration spark.history.retainedApplications is invalid - Key: SPARK-2744 URL: https://issues.apache.org/jira/browse/SPARK-2744 Project: Spark Issue Type: Bug Components: Spark Core Reporter: meiyoula Labels: historyserver when I set it in spark-env.sh like this:export SPARK_HISTORY_OPTS=$SPARK_HISTORY_OPTS -Dspark.history.ui.port=5678 -Dspark.history.retainedApplications=1 , the web of historyserver retains more than one application -- This message was sent by Atlassian JIRA (v6.2#6252)
