Re: [DISCUSS] Migration to Spark 2.0.0

2016-08-04 Thread Luciano Resende
That was going to be my suggestion... In Zeppelin, we just introduced support for different versions of scala and added support for spark 2.0 based on profiles and a bit of reflections... Do we have to do anything related to Scala versions as well ? On Thursday, August 4, 2016, Matthias Boehm

Re: [DISCUSS] Migration to Spark 2.0.0

2016-08-04 Thread Matthias Boehm
I would recommend to start an investigation if we could support both the 1.x and 2.x lines with a single code base. It seems feasible to refactor the code a bit, compile against 2.0 (or with profiles), and run on either 1.6 or 2.0. For example, by creating a wrapper that implements both Iterable