Re: Revisiting Online serving of Spark models?

2018-06-11 Thread Holden Karau
So I kicked of a thread on user@ to collect people's feedback there but I'll summarize the offline results later this week too. On Tue, Jun 12, 2018, 5:03 AM Liang-Chi Hsieh wrote: > > Hi, > > It'd be great if there can be any sharing of the offline discussion. > Thanks! > > > > Holden Karau

Re: Revisiting Online serving of Spark models?

2018-06-11 Thread Liang-Chi Hsieh
Hi, It'd be great if there can be any sharing of the offline discussion. Thanks! Holden Karau wrote > We’re by the registration sign going to start walking over at 4:05 > > On Wed, Jun 6, 2018 at 2:43 PM Maximiliano Felice < > maximilianofelice@ >> wrote: > >> Hi! >> >> Do we meet at the

[ANNOUNCE] Announcing Apache Spark 2.3.1

2018-06-11 Thread Marcelo Vanzin
We are happy to announce the availability of Spark 2.3.1! Apache Spark 2.3.1 is a maintenance release, based on the branch-2.3 maintenance branch of Spark. We strongly recommend all 2.3.x users to upgrade to this stable release. To download Spark 2.3.1, head over to the download page:

[build system] DOWNTIME ALERT! jenkins will be down all day july 16th (saturday)

2018-06-11 Thread shane knapp
hey everyone! we have another power "event" for our building on campus... this is to both fix the high-voltage lead that the city of berkeley accidentally cut last year during construction, as well as to install two new UPS systems in one of our on-prem machine rooms. while jenkins will still

Very slow complex type column reads from parquet

2018-06-11 Thread Jakub Wozniak
Hello, We have stumbled upon a quite degraded performance when reading a complex (struct, array) type columns stored in Parquet. A Parquet file is of around 600MB (snappy) with ~400k rows with a field of a complex type { f1: array of ints, f2: array of ints } where f1 array length is 50k