Vicente, The S3 configuration is similar to a file configuration. At the bottom of this email is a configuration that works (set 'connection' to 's3://bucketname'). A couple of other steps are required on each drill node: - Download the jets3t-*.jar file from: https://jets3t.s3.amazonaws.com/downloads.html and copy it into $DRILL_HOME/opt/drill/jars/3rdparty/ - Add your S3 credentials to $DRILL_HOME/conf/core-site.xml -- see my example below -- you need to define these properties: - fs.s3.awsAccessKeyId - fs.s3.awsSecretAccessKey - fs.s3n.awsAccessKeyId - fs.s3n.awsSecretAccessKey - Edit $DRILL_HOME/bin/hadoop_excludes.txt and remove 'jets3t'
You should be able to run queries after that... Nick Example storage plugin configuration: { "type": "file", "enabled": true, "connection": "s3://bucketname", "workspaces": { "default": { "location": "/", "writable": false, "storageformat": null }, "root": { "location": "/", "writable": true, "storageformat": "json" } }, "formats": { "psv": { "type": "text", "extensions": [ "tbl" ], "delimiter": "|" }, "csv": { "type": "text", "extensions": [ "csv" ], "delimiter": "," }, "tsv": { "type": "text", "extensions": [ "tsv" ], "delimiter": "\t" }, "parquet": { "type": "parquet" }, "json": { "type": "json" } } } Example entries in core-site.xml: <configuration> <property> <name>fs.s3.awsAccessKeyId</name> <value>ID</value> </property> <property> <name>fs.s3.awsSecretAccessKey</name> <value>SECRET</value> </property> <property> <name>fs.s3n.awsAccessKeyId</name> <value>ID</value> </property> <property> <name>fs.s3n.awsSecretAccessKey</name> <value>SECRET</value> </property> </configuration> -----Original Message----- From: Vicente Arteaga [mailto:vicentearte...@hotmail.com] Sent: Tuesday, December 02, 2014 3:41 AM To: drill-...@incubator.apache.org Subject: Configuration example for a s3:// source for Drill Hi! I've attended the nosqlmatters2014, and am eager to test Drill! Good work!I've played around a bit, but now I would like to connect to s3, where I currently store the files. Could you please provide me with a configuration example? I don't see where to fill in the AWS credentials, and I haven't found in the wiki nor documentation any example for that. Thanks! Best Regards,Vicente