Re: Spark 3.0 and S3A

2019-11-01 Thread Steve Loughran
On Mon, Oct 28, 2019 at 3:40 PM Sean Owen wrote: > There will be a "Hadoop 3.x" version of 3.0, as it's essential to get > a JDK 11-compatible build. you can see the hadoop-3.2 profile. > hadoop-aws is pulled in in the hadoop-cloud module I believe, so bears > checking whether the profile

Re: Spark 3.0 and S3A

2019-10-28 Thread Sean Owen
There will be a "Hadoop 3.x" version of 3.0, as it's essential to get a JDK 11-compatible build. you can see the hadoop-3.2 profile. hadoop-aws is pulled in in the hadoop-cloud module I believe, so bears checking whether the profile updates the versions there too. On Mon, Oct 28, 2019 at 10:34 AM

Spark 3.0 and S3A

2019-10-28 Thread Nicholas Chammas
Howdy folks, I have a question about what is happening with the 3.0 release in relation to Hadoop and hadoop-aws . Today, among other builds, we release a build of Spark built against Hadoop 2.7 and another one built