Github user madanadit commented on the issue:
https://github.com/apache/spark/pull/21095
Thanks @liyinan926 for the review and @andrusha for addressing the comments
and moving the work along. I can close this PR once #21260 merges
Github user madanadit commented on the issue:
https://github.com/apache/spark/pull/21095
Hi @foxish, I don't see why the 2 testing concerns should block reviewing
this PR.
1. This PR does not attempt to address non-hostpath volumes (both
implementation and unit tests are hence
Github user madanadit commented on the issue:
https://github.com/apache/spark/pull/21095
Hey @liyinan926, do you think you'll have time to take a look this week?
---
-
To unsubscribe, e-mail: reviews-unsubscr
Github user madanadit commented on the issue:
https://github.com/apache/spark/pull/21095
@liyinan926 Thanks for the comment. I refactored it to address your
concern. The code is simple enough to be easily refactored if the need be by
whoever implements the next volume type
Github user madanadit commented on the issue:
https://github.com/apache/spark/pull/21095
Thanks @foxish for your feedback. As a first time contributor to Spark, I
would like to limit the scope of the changes in this PR. Let me know when
you're ready to review again
Github user madanadit commented on the issue:
https://github.com/apache/spark/pull/21095
@liyinan926 @foxish Please take a look
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
GitHub user madanadit opened a pull request:
https://github.com/apache/spark/pull/21095
[SPARK-23529][K8s] Support mounting hostPath volumes
## What changes were proposed in this pull request?
This PR introduces a new config `spark.kubernetes.driver/executor.volumes
Github user madanadit closed the pull request at:
https://github.com/apache/spark/pull/21032
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user madanadit commented on the issue:
https://github.com/apache/spark/pull/21032
@liyinan926 @foxish I've started a doc
[here](https://docs.google.com/document/d/15-mk7UnOYNTXoF6EKaVlelWYc9DTrTXrYoodwDuAwY4/edit?usp=sharing).
Feel free to edit and circulate to interested
Github user madanadit commented on the issue:
https://github.com/apache/spark/pull/21032
@liyinan926 Sounds good
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user madanadit commented on the issue:
https://github.com/apache/spark/pull/21032
@foxish @liyinan926 Thanks for your comments. To make the configuration
more general, here is an initial proposal.
Context:
- volumeMount requires `[name, mountPath, readWriteMode
Github user madanadit commented on a diff in the pull request:
https://github.com/apache/spark/pull/21032#discussion_r181197979
--- Diff:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala
---
@@ -117,6 +117,13 @@ private[spark] object
Github user madanadit closed the pull request at:
https://github.com/apache/spark/pull/20989
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user madanadit commented on the issue:
https://github.com/apache/spark/pull/21032
@felixcheung, @foxish can you take a look?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user madanadit commented on the issue:
https://github.com/apache/spark/pull/20989
Thanks @felixcheung!. I've re targeted to master in this
[PR](https://github.com/apache/spark/pull/21032). Closing this one
GitHub user madanadit opened a pull request:
https://github.com/apache/spark/pull/21032
[SPARK-23529][K8s] Support mounting hostPath volumes for executors
## What changes were proposed in this pull request?
This PR introduces a new config `spark.kubernetes.executor.volumes
Github user madanadit commented on the issue:
https://github.com/apache/spark/pull/20989
@foxish The failure seems to be because of `SparkRemoteFileTest` missing
from branch-2.3 (only present in master branch). Which branch do you recommend
targeting this PR towards
Github user madanadit commented on the issue:
https://github.com/apache/spark/pull/20989
@foxish Can you take a look?. Thanks!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
GitHub user madanadit opened a pull request:
https://github.com/apache/spark/pull/20989
[SPARK-23529][K8s] Support mounting hostPath volumes for executors
## What changes were proposed in this pull request?
This PR introduces a new config `spark.kubernetes.executor.volumes
19 matches
Mail list logo
