Re: Using hadoop-cloud_2.12 jars

2020-06-22 Thread Rahij Ramsharan
Thanks for the response. If we intend consumers to be able to use this based on the docs I linked, could we publish the jar to maven central? On Mon, Jun 22, 2020 at 12:59 PM Jorge Machado wrote: > You can build it from source. > > Clone the spark git repo and run: ./build/mvn clean package

Re: Using hadoop-cloud_2.12 jars

2020-06-22 Thread Jorge Machado
You can build it from source. Clone the spark git repo and run: ./build/mvn clean package -DskipTests -Phadoop-3.2 -Pkubernetes -Phadoop-cloud Regards > On 22. Jun 2020, at 11:00, Rahij Ramsharan wrote: > > Hello, > > I am trying to use the new S3 committers >

Using hadoop-cloud_2.12 jars

2020-06-22 Thread Rahij Ramsharan
Hello, I am trying to use the new S3 committers ( https://spark.apache.org/docs/latest/cloud-integration.html#committing-work-into-cloud-storage-safely-and-fast) in spark 3.0.0. As per https://spark.apache.org/docs/latest/cloud-integration.html#installation, I need to include