Re: [discuss] dropping Hadoop 2.2 and 2.3 support in Spark 2.0?

2016-01-14 Thread Sean Owen
I personally support this. I had suggest drawing the line at Hadoop 2.6, but that's minor. More info: Hadoop 2.7: April 2015 Hadoop 2.6: Nov 2014 Hadoop 2.5: Aug 2014 Hadoop 2.4: April 2014 Hadoop 2.3: Feb 2014 Hadoop 2.2: Oct 2013 CDH 5.0/5.1 = Hadoop 2.3 + backports CDH 5.2/5.3 = Hadoop 2.5 +

[discuss] dropping Hadoop 2.2 and 2.3 support in Spark 2.0?

2016-01-13 Thread Reynold Xin
We've dropped Hadoop 1.x support in Spark 2.0. There is also a proposal to drop Hadoop 2.2 and 2.3, i.e. the minimal Hadoop version we support would be Hadoop 2.4. The main advantage is then we'd be able to focus our Jenkins resources (and the associated maintenance of Jenkins) to create builds