The SparkR support PR includes integration testing that can be tested on a local Minikube instance by merely running the distribution with appropriate flags (--r) and running the integration-tests similarly to as you would on any k8s test. Maybe some others could locally test this, if there is any hesitation. I, for one, don't see a problem with merging as is and having the Jenkins build be configured for R distribution support via the Ubuntu update when Shane gets to it, similar to as we did for PySpark. The PR can be found here <https://github.com/apache/spark/pull/21584>.
Furthermore, I am wondering if there is any desire to ensure SparkR support (for k8s) is merged by 2.4 by the community, and if so, then maybe merging this without the Jenkins build seems even more appropriate. Best, Ilan Filonenko On Wed, Aug 15, 2018 at 3:33 PM, Erik Erlandson <eerla...@redhat.com> wrote: > The SparkR support PR is finished, along with integration testing, however > Shane has requested that the integration testing not be enabled until after > the 2.4 release because it requires the OS updates he wants to test *after* > the release. > > The integration testing can be run locally, and so the question at hand > is: would the PMC be willing to consider inclusion of the SparkR for 2.4, > based on local verification of the testing? The PySpark PR was merged under > similar circumstances: the testing was verified locally and the PR was > merged before the testing was enabled for jenkins. > > Cheers, > Erik >
