> On 14 Jan 2016, at 09:28, Steve Loughran <ste...@hortonworks.com> wrote: >> > > 2.6.x is still having active releases, likely through 2016. It'll be the only > hadoop version where problems Spark encounters would get fixed
Correction: minimum Hadoop version Any problem reported against older versions will probably get a message saying "upgrade" > > It's also the last iteration of interesting API features —especially in YARN: > timeline server, registry, various other things > > And it has s3a, which, for anyone using S3 for storage, is the only S3 > filesystem binding I'd recommend. Hadoop 2.4 not only has s3n, it's got a > broken one that (HADOOP-10589) > > I believe 2.6 supportsr recent guava versions, even if it is frozen on 11.0 > to avoid surprising people (i.e. all deprecated/removed classes should have > been stripped) > > Finally: it's the only version of Hadoop that works on Java 7, has patches to > support Java8+kerberos (in fact, Java 7u80+ and kerberos). > > For the reason of JVMs and guava alone, I'd abandon Hadoop < 2.6. Those > versions won't work on secure Java 7 clusters, recent guava versions, and > have lots of uncorrected issues. > > Oh, and did I mention the test matrix? The later version of Hadoop you use, > the less versions to test against. > >> My general position is that backwards-compatibility and supporting >> older platforms needs to be a low priority in a major release; it's a >> decision about what to support for users in the next couple years, not >> the preceding couple years. Users on older technologies simply stay on >> the older Spark until ready to update; they are in no sense suddenly >> left behind otherwise. > > > If they are running older versions of Hadoop, they generally have stable apps > which they don't bother upgrading. New clusters => new versions => new apps. > > > B�KKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKKCB�� [��X��ܚX�KK[XZ[�]�][��X��ܚX�P �\�˘\X�K�ܙ�B��܈Y][ۘ[��[X[� �K[XZ[�]�Z[ �\�˘\X�K�ܙ�B� I have no idea what this is or why it made it to the tail of my email. Maybe outlook has changed its signature for me. --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
