Scala type checking thread-safety issue, and global locks to resolve it

2019-03-14 Thread Sean Owen
This is worth a look: https://github.com/apache/spark/pull/24085 Scala has a surprising thread-safety bug in the "<:<" operator that's used to check subtypes, which can lead to incorrect results in non-trivial situations. The fix on the table is to introduce a global lock to protect a lot of the

Re: [build system] jenkins wedged again, restarted service.

2019-03-14 Thread shane knapp
things are building again! On Thu, Mar 14, 2019 at 2:02 PM shane knapp wrote: > if performance continues to drop off like this, i will reboot the box soon. > > builds should be back to normal in a few minutes... sorry for the > inconvenience! > > -- > Shane Knapp > UC Berkeley EECS Research /

[build system] jenkins wedged again, restarted service.

2019-03-14 Thread shane knapp
if performance continues to drop off like this, i will reboot the box soon. builds should be back to normal in a few minutes... sorry for the inconvenience! -- Shane Knapp UC Berkeley EECS Research / RISELab Staff Technical Lead https://rise.cs.berkeley.edu

Re: [discuss] 2.4.1-rcX release, k8s client PRs, build system infrastructure update

2019-03-14 Thread shane knapp
about to send mail. On Thu, Mar 14, 2019 at 1:00 PM Marcelo Vanzin wrote: > Not sure if anyone else is having the same issues, but jenkins seems > to be in a weird state. > > I can't connect to the web UI, and it doesn't seem to be responding to > test requests. > > On Wed, Mar 13, 2019 at 3:15

Re: [discuss] 2.4.1-rcX release, k8s client PRs, build system infrastructure update

2019-03-14 Thread Marcelo Vanzin
Not sure if anyone else is having the same issues, but jenkins seems to be in a weird state. I can't connect to the web UI, and it doesn't seem to be responding to test requests. On Wed, Mar 13, 2019 at 3:15 PM shane knapp wrote: > > upgrade completed, jenkins building again... master PR

Re: [discuss] 2.4.1-rcX release, k8s client PRs, build system infrastructure update

2019-03-14 Thread shane knapp
i think we should. the version of k8s we were testing against before was v1.10.x, and we are currently testing master *and* 2.4.1 against v1.13.1. given that we're a couple of weeks away from v.1.14.x i feel that this is a safe and prudent thing to include. On Thu, Mar 14, 2019 at 10:57 AM DB

Re: [discuss] 2.4.1-rcX release, k8s client PRs, build system infrastructure update

2019-03-14 Thread Marcelo Vanzin
IMO we can vote on rc8; that upgrade should not block it. If it fails for some other reason, then rc9 will have it. Supported versions in k8s-land are really weird and don't match well to our release schedule. There isn't a good solution to this at the moment... good thing k8s support is still

Re: [discuss] 2.4.1-rcX release, k8s client PRs, build system infrastructure update

2019-03-14 Thread DB Tsai
Since rc8 was already cut without the k8s client upgrade; the build is ready to vote, and including k8s client upgrade in 2.4.1 implies that we will drop the old-but-not-that-old K8S versions as Sean mentioned, should we include this upgrade in 2.4.2? Thanks. Sincerely, DB Tsai

Re: [discuss] 2.4.1-rcX release, k8s client PRs, build system infrastructure update

2019-03-14 Thread shane knapp
thanks everyone, both PRs are merged. :) On Wed, Mar 13, 2019 at 3:51 PM shane knapp wrote: > btw, let's wait and see if the non-k8s PRB tests pass before merging > https://github.com/apache/spark/pull/23993 in to 2.4.1 > > On Wed, Mar 13, 2019 at 3:42 PM shane knapp wrote: > >> 2.4.1 k8s