Github user ericvandenbergfb commented on the issue:
https://github.com/apache/spark/pull/18791
See continuation of pull request at
https://github.com/apache/spark/pull/19770
---
-
To unsubscribe, e-mail:
Github user jerryshao commented on the issue:
https://github.com/apache/spark/pull/18791
@ericvandenbergfb please also fix the PR title, thanks.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user jiangxb1987 commented on the issue:
https://github.com/apache/spark/pull/18791
@ericvandenbergfb Could you please rebase this to the latest master so we
can continue review it? Also cc @vanzin @jerryshao
---
Github user ericvandenbergfb commented on the issue:
https://github.com/apache/spark/pull/18791
The default is off, so people can opt-in to more aggressive clean up.
Is this okay to be merged?
---
If your project is set up for it, you can reply to this email and have your
Github user jiangxb1987 commented on the issue:
https://github.com/apache/spark/pull/18791
Yea, I'm just thinking whether it is possible we can have a perfect
approach that we can be confident to turn it on by default.
---
If your project is set up for it, you can reply to this
Github user ajbozarth commented on the issue:
https://github.com/apache/spark/pull/18791
This is something that should be defaulted to off, like I mentioned above
many users like myself only use one log directory and when this is on it would
delete any non-application logs in the log
Github user jiangxb1987 commented on the issue:
https://github.com/apache/spark/pull/18791
QQ: Do we want to default turn on the feature in some future version? If
so, what pre-condition in your mind should be fulfilled? Currently we have had
too much configurations that default to
Github user ajbozarth commented on the issue:
https://github.com/apache/spark/pull/18791
Overall I like this as an option and the code looks good. Personally I use
one log directory for all my logs (not just the SHS) so this wouldn't work for
me, but I also run into dead files
