I fully support merging that SparkR support on k8s. If Ilan and other are willing to manually validate the RC I’m happy to voucher for it (I’m not 100% sure i have capacity to test it that way but certainly will try)
Also +1 on revisiting Jenkins builds. tbh I’m not sure we depend on them too much during the release process due to reason Marcelo listed, for the last 4+ releases... ________________________________ From: Ilan Filonenko <i...@cornell.edu> Sent: Thursday, August 16, 2018 10:27 AM To: shane knapp Cc: Erik Erlandson; Wenchen Fan; Marcelo Vanzin; Reynold?Xin; Spark dev list Subject: Re: [DISCUSS] SparkR support on k8s back-end for Spark 2.4 Okay, if there is a consensus in merging the SparkR feature and its respective tests separately, I will split up the current PR to allow for the feature to be merged while the e2e tests to be used for locally testing until we are ready to merge (similar to PySpark). Will give it another day to hear other opinions, but otherwise working under the impression that the OS upgrade will not happen before the 2.4 cut. On Thu, Aug 16, 2018 at 12:53 PM, shane knapp <skn...@berkeley.edu<mailto:skn...@berkeley.edu>> wrote: On Thu, Aug 16, 2018 at 9:49 AM, Erik Erlandson <eerla...@redhat.com<mailto:eerla...@redhat.com>> wrote: IMO sparkR support makes sense to merge for 2.4, as long as the release wranglers agree that local integration testing is sufficiently convincing. i agree w/this. testing for this stuff specifically will happen within a couple of weeks after the 2.4 cut. Part of the intent here is to allow this to happen without Shane having to reorganize his complex upgrade schedule and make it even more complicated. this. exactly. :) -- Shane Knapp UC Berkeley EECS Research / RISELab Staff Technical Lead https://rise.cs.berkeley.edu
