Github user foxish commented on the issue: https://github.com/apache/spark/pull/19468 @skonto, there was some discussion about this on the SPIP. We see them as separate and independent issues with the pluggable API being a long term goal. It would involve a working group of people drawn from all cluster manager integration maintainers - because the changes are extensive and require scalability/performance testing in a variety of environments - probably over 2-3 releases before we can iron out the kinks. In the short term however, adding the package here enables supporting the growing number of K8s users of Spark that currently rely on our fork, and the integration testing makes us confident about not impacting the release process or adding complexity to the maintainers workflow. The K8s community will set up all the requisite testing, and ensure health of this package. The integration testing would also carry forward to when we have pluggable APIs in the future.
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org