[
https://issues.apache.org/jira/browse/SPARK-32097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gaurangi Saxena updated SPARK-32097:
------------------------------------
Description: Our service dynamically creates short-lived YARN clusters in
cloud. Spark applications run on these dynamically created clusters. Log data
for these applications is stored on a remote file-system. We want a static
instance of SparkHistoryServer to view information on jobs that ran on these
clusters. We use glob because we cannot have a static list of directories where
the log files reside. (was: We would like to configure SparkHistoryServer to
display applications from multiple clusters/environments. Data displayed on
this UI comes from directory configured as log-directory. It would be nice if
this log-directory also accepted regex. This way we will be able to read and
display applications from multiple directories.
)
> Allow reading history log files from multiple directories
> ---------------------------------------------------------
>
> Key: SPARK-32097
> URL: https://issues.apache.org/jira/browse/SPARK-32097
> Project: Spark
> Issue Type: Wish
> Components: Spark Core
> Affects Versions: 2.4.5
> Reporter: Gaurangi Saxena
> Priority: Minor
>
> Our service dynamically creates short-lived YARN clusters in cloud. Spark
> applications run on these dynamically created clusters. Log data for these
> applications is stored on a remote file-system. We want a static instance of
> SparkHistoryServer to view information on jobs that ran on these clusters. We
> use glob because we cannot have a static list of directories where the log
> files reside.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
