Re: Path style access fs.s3a.path.style.access property is not working in spark code

Recommend to use v2.9.x, there are lot of optimizations that makes life much easier while accessing from Spark. Thanks. -Samik On 05-05-2020 01:55 am, Aniruddha P Tekade wrote: Hello User, I got the solution to this. If you are writing to a custom s3 url, then use the hadoop-aws-2.8.0.jar as

Re: Path style access fs.s3a.path.style.access property is not working in spark code

Hello User, I got the solution to this. If you are writing to a custom s3 url, then use the hadoop-aws-2.8.0.jar as the separate flag was introduced to enable path style access. Best, Aniruddha --- ᐧ On Fri, May 1, 2020 at 5:08 PM Aniruddha P Tekade wrote: > Hello Users, > > I am

Path style access fs.s3a.path.style.access property is not working in spark code

Hello Users, I am using on-premise object storage and able to perform operations on different bucket using aws-cli. However, when I am trying to use the same path from my spark code, it fails. Here are the details - Addes dependencies in build.sbt - - hadoop-aws-2.7.4.ja -