RE: Apache Drill Question

2017-06-14 Thread Kunal Khatua
Not familiar with SSHFS or GlusterFS specs, but It should, in theory, work out of the box. You can start off Drill with having the underlying storage plugins talk to a localFS. I'm presuming SSHFS / GlusterFS can expose the files through a local NFS-like mount. However, if your three nodes al

Apache Drill Question

2017-06-14 Thread Edgardo Robles
Hi, I setup a 3 node zooker/drill cluster but would like to test parquet files but do not want to setup hdfs. Would drill work if I used sshfs or glusterfs to store the parquet files and the cluster be able to query the parquet file with similar performance as hdfs or is using sshfs or gluste

Re: QUESTION: Drill Configuration to access S3 buckets

2017-06-14 Thread Uwe L. Korn
The current Drill releases use the hadoop-io libraries from the 2.7.x series. Locally I have built against the 3.0.0 alpha (2.8 should also work) and can access the regions with newer signature versions. But you should be careful with that as I had to do some code changes to have it built with t

Re: QUESTION: Drill Configuration to access S3 buckets

2017-06-14 Thread Shankar Mane
aws new regions uses only signature version 4 protocol for S3. Other regions has both V2 and V4 compatible. Drill works very well if regions has both signature versions. By adding endpoints, same problem persists. May be Drill API doesn't have support to V4 protocol yet. This V4 problems is also

Re: QUESTION: Drill Configuration to access S3 buckets

2017-06-14 Thread Jack Ingoldsby
Useful to know, thanks. Also having problems with Ohio. Will try another region On Wed, Jun 14, 2017, 19:46 Сергей Боровик wrote: > Hi! > I have an AWS EC2 instance with apache drill 1-10.0.and configured IAM > Role. > > And I am able to access and query S3 bucket in US East (N. Virginia) > regi

Re: Increasing store.parquet.block-size

2017-06-14 Thread Padma Penumarthy
Sure. I will check and try to fix them as well. Thanks, Padma > On Jun 14, 2017, at 3:12 AM, Khurram Faraaz wrote: > > Thanks Padma. There are some more related failures reported in DRILL-2478, do > you think we should fix them too, if it is an easy fix. > > > Regards, > > Khurram > >

QUESTION: Drill Configuration to access S3 buckets

2017-06-14 Thread Сергей Боровик
Hi! I have an AWS EC2 instance with apache drill 1-10.0.and configured IAM Role. And I am able to access and query S3 bucket in US East (N. Virginia) region, but not able to access/query buckets in US East (Ohio) region, it fails with "error: system error: amazons3exception: status code 400, AWS S

RE: JDBC help

2017-06-14 Thread Kunal Khatua
You need to use the org.apache.drill.jdbc.Driver class. The drill-jdbc-all dependency should be enough. Are you trying to connect to an HDFS cluster's zookeeper? -Original Message- From: Aspen Hsu [mailto:a...@xactlycorp.com] Sent: Wednesday, June 14, 2017 12:07 PM To: user@drill.apac

JDBC help

2017-06-14 Thread Aspen Hsu
Hello, I am working on a maven project that relies on drill-JDBC, for a stand alone project, I used this dependency (and it connected/ran fine) org.apache.drill.exec drill-jdbc-all 1.10.0 For my actual project, I am using the same dependency and getting this error. WARN oadd.org.apache.zooke

Re: Using Apache Drill with AirBnB SuperSet

2017-06-14 Thread John Omernik
Be warned it's a hack together thing I and some others (Waves at Charles) did to prove it out... we'd love it if MapR sought to extend it and put some programming rigor around it... *hint hint :) On Wed, Jun 14, 2017 at 3:04 PM, Kunal Khatua wrote: > The Superset project looks pretty neat. > >

RE: Using Apache Drill with AirBnB SuperSet

2017-06-14 Thread Kunal Khatua
The Superset project looks pretty neat. Didn't realize that there is also a Python Driver for Drill. I'd think that would be useful too. -Original Message- From: John Omernik [mailto:j...@omernik.com] Sent: Wednesday, June 14, 2017 3:45 AM To: user Subject: Using Apache Drill with Ai

Re: Drill Summit/Conference Proposal

2017-06-14 Thread Julian Hyde
I like the idea of co-hosting a conference. ApacheCon in particular is a good venue, and they explicitly encourage sub-conferences (there are “Big Data” and “IoT” tracks, and this year there were sub-conferences for Tomcat and CloudStack). DrillCon was part of ApacheCon, people could attend a w

Trouble connecting to S3 - but can connect with AWS utility using same credentials

2017-06-14 Thread Jack Ingoldsby
Hi, I set up a demo bucket.but am having some trouble connecting using Drill on Windows, embedded I created an S3 storage plugin "s3citibike" The keys below are real keys (don't get too excited, read only, no rights to change permissions, only contains public data) { "type": "file", "enab

Re: Drill Summit/Conference Proposal

2017-06-14 Thread Charles Givre
Hi Bob, Good to hear from you. I agree that there could be value in having a joint Presto/Drill/Redshift conference, but how would you describe the overall theme? In essence (not looking to start a flame war here...) these tools are similar in terms of what the user experiences and I can definitel

Re: Drill Summit/Conference Proposal

2017-06-14 Thread Bob Rudis
I grok this is the Drill list and I'm also a big user of Drill (and have made some UDFs) but there might be some efficacy in expanding the scope to the Presto and Redshift Spectrum communities. I'm not claiming there's 100% equivalence, but the broader view of being able to access multiple types of

Re: Connecting to S3 bucket which does not seem to require a key

2017-06-14 Thread Jack Ingoldsby
I think I will add this as a second thread, as I think it's kind of moved on from my original question. It sounds like one cannot connect without a key even if the bucket is public, which is good to know. Thx Jack On Mon, Jun 12, 2017 at 2:25 PM, Jack Ingoldsby wrote: > Thanks, but unfortunatel

Drill Summit/Conference Proposal

2017-06-14 Thread Charles Givre
Hello fellow Drill users and developers, I've been involved with the Drill community for some time, and I was thinking that it might be time to start exploring the idea of a Drill Summit or Conference. If you're interested, please send me a note and I'll start having some conversations about what'

Using Apache Drill with AirBnB SuperSet

2017-06-14 Thread John Omernik
Hey all, I've had some success getting Drill working with Superset (Visualization tool) via work done by a few people (not just me) I thought I'd share what I was using and how it was working and see if others benefited from it, or if issues occur they can be posted to the issue to improve things.

Re: Increasing store.parquet.block-size

2017-06-14 Thread Khurram Faraaz
Thanks Padma. There are some more related failures reported in DRILL-2478, do you think we should fix them too, if it is an easy fix. Regards, Khurram From: Padma Penumarthy Sent: Wednesday, June 14, 2017 11:43:16 AM To: user@drill.apache.org Subject: Re: Incr