Thanks for chiming in. Note that an organization's agility in Spark
upgrades can be very different from Hadoop upgrades.
For many orgs, Hadoop is responsible for cluster resource scheduling (YARN)
and data storage (HDFS). These two are notorious difficult to upgrade. It
is all or nothing for a clu
> On 14 Jan 2016, at 09:28, Steve Loughran wrote:
>>
>
> 2.6.x is still having active releases, likely through 2016. It'll be the only
> hadoop version where problems Spark encounters would get fixed
Correction: minimum Hadoop version
Any problem reported against older versions will probably
> On 14 Jan 2016, at 02:17, Sean Owen wrote:
>
> I personally support this. I had suggest drawing the line at Hadoop
> 2.6, but that's minor. More info:
>
> Hadoop 2.7: April 2015
> Hadoop 2.6: Nov 2014
> Hadoop 2.5: Aug 2014
> Hadoop 2.4: April 2014
> Hadoop 2.3: Feb 2014
> Hadoop 2.2: Oct 201
I personally support this. I had suggest drawing the line at Hadoop
2.6, but that's minor. More info:
Hadoop 2.7: April 2015
Hadoop 2.6: Nov 2014
Hadoop 2.5: Aug 2014
Hadoop 2.4: April 2014
Hadoop 2.3: Feb 2014
Hadoop 2.2: Oct 2013
CDH 5.0/5.1 = Hadoop 2.3 + backports
CDH 5.2/5.3 = Hadoop 2.5 + b
We've dropped Hadoop 1.x support in Spark 2.0.
There is also a proposal to drop Hadoop 2.2 and 2.3, i.e. the minimal
Hadoop version we support would be Hadoop 2.4. The main advantage is then
we'd be able to focus our Jenkins resources (and the associated maintenance
of Jenkins) to create builds fo
