GitHub user paragpc opened a pull request:
https://github.com/apache/spark/pull/11736
[SPARK-13914] [Scheduler] Add functionality to optionally configure a
backup path for the Spark event logs
## What changes were proposed in this pull request?
Added "spark.eventLog.backup.enabled" and "spark.eventLog.backup.dir"
options for spark event logs to allow real time back up of spark event logs.
Added functionality to backup event logs in back up directory as event
happens.
Added corresponding documentation changes.
Users can backup the event logs to a cloud storage service directly and
keep them there forever or in cloud environment, users may want to store the
event logs on local disks and back them up to a cloud storage service from time
to time using cloud backup agents.
## How was this patch tested?
Existing tests passed.
Added unit tests for back up functionality.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/paragpc/spark master_event_log_backup
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/11736.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #11736
----
commit ec9999fcb4aabefd9b26462cb0b3c0914d6a9ea9
Author: Parag Chaudhari <para...@users.noreply.github.com>
Date: 2016-03-15T17:25:46Z
Merge pull request #1 from apache/master
Update from original repository
commit 5af1f8a4abc4ee48e862c052a406407f184e7161
Author: Parag Chaudhari <para...@amazon.com>
Date: 2016-03-14T17:54:17Z
Added functionality to optionally configure a backup path for the Spark
event log.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
