On Wed, Mar 13, 2019 at 11:49 AM Marcelo Vanzin <van...@cloudera.com> wrote:
> Do the upgraded minikube/k8s versions break the current master client > version too? > > yes. > I'm not super concerned about 2.4 integration tests being broken for a > little bit. It's very uncommon for new PRs to be open against > branch-2.4 that would affect k8s. > > ok. if the 2.4.1 PR is merged at about the same time as the one for master, we won't have to worry about the 2.4.x tests failing. But I really don't want master to break. So if we can upgrade minikube > first, even if that breaks k8s integration tests on branch-2.4 for a > little bit, that would be optimal IMO. > > i have everything staged on the affected jenkins workers and can do the infra upgrade really quickly... shane > On Wed, Mar 13, 2019 at 11:26 AM shane knapp <skn...@berkeley.edu> wrote: > > > > hey everyone... i wanted to break this discussion out of the > mega-threads for the 2.4.1 RC candidates. > > > > the TL;DR is that we've been trying to update the k8s client libs to > something much more modern. however, for us to do this, we need to update > our very old k8s and minikube versions. > > > > the problem here lies in the fact that if we update the client libs on > master, but not the 2.4 branch, then the 2.4 branch k8s integration tests > will fail if we update our backend minikube/k8s versions. > > > > i've done all of the testing locally for the new k8s client libs, and am > ready to pull the trigger on the infrastructure upgrade (which will take > all of ~15 mins). > > > > for this to happen, two PRs will need to be merged... one for 2.4.1 and > one for master. > > > > is there a chance that we can get > https://github.com/apache/spark/pull/23993 merged in for the 2.4.1 > release? this will also require > https://github.com/apache/spark/pull/24002 (for master) to be merged > simultaneously. > > > > both of those PRs are ready to go (tho 23993 was closed w/o merge and > i'm not entirely sure why). > > > > here's the primary jira we're using to track this upgrade: > > https://issues.apache.org/jira/browse/SPARK-26742 > > > > thanks in advance, > > > > shane > > -- > > Shane Knapp > > UC Berkeley EECS Research / RISELab Staff Technical Lead > > https://rise.cs.berkeley.edu > > > > -- > Marcelo > -- Shane Knapp UC Berkeley EECS Research / RISELab Staff Technical Lead https://rise.cs.berkeley.edu
