[ https://issues.apache.org/jira/browse/SPARK-13914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sean Owen resolved SPARK-13914. ------------------------------- Resolution: Won't Fix > Add functionality to back up spark event logs > --------------------------------------------- > > Key: SPARK-13914 > URL: https://issues.apache.org/jira/browse/SPARK-13914 > Project: Spark > Issue Type: Improvement > Components: Scheduler > Affects Versions: 1.6.0, 1.6.2, 2.0.0 > Reporter: Parag Chaudhari > > Spark event logs are usually stored in HDFS when running Spark on YARN. In a > cloud environment, these HDFS files are often stored on the disks of > ephemeral instances that could go away once the instances are terminated. > Users may want to persist the event logs as the event happens for issue > investigation and performance analysis before and after the cluster is > terminated. The backup path can be managed by the spark users based on their > needs. For example, some users may copy the event logs to a cloud storage > service directly and keep them there forever. While some other users may want > to store the event logs on local disks and back them up to a cloud storage > service from time to time. Other users will not want to use the feature, so > this feature should be off by default; users enable the feature when and only > when they need it. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org