Thanks Jason! I hadn't noticed the config property for S3. I tried this out now, and feel it is a lot more easier now.
And yes, we should definitely update the docs. There have been quite a few threads related to S3 config. On Wed, Apr 20, 2016 at 8:19 AM, Jason Altekruse <[email protected]> wrote: > I don't believe there is any way in which a particular bucket has a > property of being s3, s3n or s3a. As I understand it, this only change the > client library that is used to interface with S3. We have included the jars > necessary for s3a with Drill, which is the newest and most performant > option available. > > I need to open a doc JIRA for this, but there is one way in which the s3 > experience was improved recently to prevent the need to restart Drill to > add your S3 credentials. When you create a connection to an S3 bucket, you > can now specify your credentials in a property named "config" in the > storage plugin. This allows you to set any filesystem properties, which we > previously was only possible to set with a core-site.xml file on the > classpath when starting Drill. > > Example: > { > "type": "file", > "enabled": true, > "connection": "s3a://address.of.your.bucket/", > "config": { > "fs.s3a.access.key": "<PUT YOUR ACCESS KEY HERE>", > "fs.s3a.secret.key": "<PUT YOUR SECRET KEY HERE>" > }, > "workspaces": { > "root": { > "location": "/", > "writable": false, > "defaultInputFormat": null > } > }, > "formats": { > "psv": { > "type": "text", > "extensions": [ > "tbl" > ], > "delimiter": "|" > }, ... > > > Jason Altekruse > Software Engineer at Dremio > Apache Drill Committer > > On Wed, Apr 20, 2016 at 7:40 AM, Nick Monetta <[email protected]> wrote: > > > Hi, > > Does Drill v1.6 still support s3n connections or just s3a? > > > > I have a s3n S3 bucket that I'm trying to connect to and it will not > work. > > My config is: > > > > { > > "type": "file", > > "enabled": true, > > "connection": "s3n://inrixprod-tapp/", > > "workspaces": { > > "root": { > > "location": "/", > > "writable": false, > > "defaultInputFormat": null > > }, > > > > Nick Monetta | INRIX |[email protected] |Movement Intelligence | > > www.inrix.com | mobile +1 646-248-4105 | > > > > > > >
