Github user jerryshao commented on the issue:
https://github.com/apache/spark/pull/18905
I have two questions about the fix:
1. Is it a good idea to change recovery path to other directory? Since
recovery path is configured by user or figured out by yarn, so maybe YARN has
some assumption about this path, if we change to other one, will this introduce
some issues. Also if recovery path is null, should it be guaranteed by user for
the availability.
2. What if the previous bad disk back to normal with orphan data? For
example is dir1 is failed with state V1, and based on this logic we should
another dir2 and state changed to v2. Then after a while if dir1 is back to
normal, then which dirs are we choosing based on your current code?
CC @tgravescs to review.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
