Re: Announcing new committer: Hanifi Gunes

Congratulations, Hanifi! > On Apr 16, 2015, at 2:29 PM, Jacques Nadeau wrote: > > The Apache Drill PMC is very pleased to announce Hanifi Gunes as a new > committer. > > He has been providing great contributions over the past six months. > Additionally, has quickly become the go to expert for v

Re: Drill logical plan optimization

Hi Rajkumar, Here are some links: http://drill.apache.org/docs/performance-tuning-introduction/ (Performance Tuning Guide) http://drill.apache.org/docs/query-plans/ Did you mean optimize

Re: Monitoring long / stuck CTAS

See below: > On May 27, 2015, at 12:17 PM, Matt wrote: > > Attempting to create a Parquet backed table with a CTAS from an 44GB tab > delimited file in HDFS. The process seemed to be running, as CPU and IO was > seen on all 4 nodes in this cluster, and .parquet files being created in the > ex

Re: JAVA API for Drill

Adding to Hanifi’s comment. Loot at QueryWrapper#run method and QueryWrapper$Listener https://github.com/hnfgns/incubator-drill/blob/master/exec/java-exec/src/main/java/org/apache/drill/exec/server/rest/QueryWrapper.java

Re: Issues on querying hbase table on REST

Hi, Are you sending the correct Content-Type in the request header? Here’s a link that might help you: http://docs.brightcove.com/en/video-cloud/player-management/guides/postman.html and the Drill REST API doc:

Re: Set Drill Response Format to CSV Through Rest APIs

Currently we support only JSON through REST API. Thank you, Sudheesh > On Jul 15, 2015, at 9:26 PM, Usman Ali wrote: > > Hi, > Is there any way to set response format of drill to csv instead of > json using Rest APIs? If yes, then what other response formats are > available in drill. > >

Re: Flatten Output Json

Does this help? http://drill.apache.org/docs/flatten/ Thank you, Sudheesh > On Jul 15, 2015, at 10:55 PM, Usman Ali wrote: > > Hi, > Drill sqlline displays output in a nice format. I am guessing it must > be flattening the output json before printing

Re: using the REST API

See inline. > On Jul 16, 2015, at 4:36 AM, Stefán Baxter wrote: > > Hi, > > I have a few questions regarding the rest API. > > - Is it possible that the rest api (query.json) should return numeric > values as strings? There is a ticket for this: https://issues.apache.org/jira/browse/DRILL

Re: Rest API

See inline. > On Jul 16, 2015, at 4:03 AM, Preetham Nadig > wrote: > > Hello, > I am planning to use DRILL for one of my projects and I have been working > with it for couple of weeks That’s awesome! > One of the things I would like to do is access DRILL over a REST API, I have > successful

Re: empty results

Is it returning empty results only though REST API? Did you try sqlline? Do you have a simple repro? If so, can you file a ticket? Thank you, Sudheesh > On Jul 16, 2015, at 10:25 AM, Stefán Baxter wrote: > > Hi, > > What can be happening if a drillbit (local) starts returning empty results >

Re: Operation category READ is not supported in state standby at org.apache.hadoop.hdfs.server.namenode.ha.Standby

Can you try just “thrift://nn2:9083 ” (and not include the failover namenode) for “hive.metastore.uris” property? Thank you, Sudheesh > On Jul 16, 2015, at 1:43 AM, Arthur Chan wrote: > > Anyone has idea what I would be wrong in setup Drill? > > On Tue, Jul 14, 2015 at 4:21 PM, Arthur Chan >

Hangout happening now!

Come join the Drill community as we discuss what has been happening lately and what is in the pipeline. All are welcome, if you know about Drill, want to know more or just want to listen in. Link: https://plus.google.com/hangouts/_/event/ci4rdiju8bv04a64efj5fedd0lc

Re: pending queries jamming the system

Hi Stefan, Can you create a JIRA for this? Please attach to the JIRA: (1) thread dumps of the three Drillbits (you can get this using jstack), and (2) json query profiles of the PENDING queries (you can get this from the "Full JSON Profile" at the bottom of the profile page). Thank you, Sudhees

Resetting an option

Hey y‘all, Re DRILL-1065 , at system level (ALTER system RESET …), resetting an option would mean changing the value to the default provided by Drill. But, at session level (ALTER session RESET …), would resetting an option mean: (a) changing th

Re: Resetting an option

Correction: currently any user can SET or RESET an option for session and system. > On Aug 10, 2015, at 2:20 PM, Sudheesh Katkam wrote: > > Hey y‘all, > > Re DRILL-1065 <https://issues.apache.org/jira/browse/DRILL-1065>, at system > level (ALTER system RESET …), r

Re: Help with optimizing a query

Hi Yousef, If possible, could you put the profile in a publicly accessible location (Dropbox, etc) and post the link here? Thank you, Sudheesh > On Aug 10, 2015, at 3:43 PM, Yousef Lasi wrote: > > We're running a 4 file join on a set of parquet files, the largest of which > is about 20 GB in

Re: Drill view schema returned in results set

HI Michael, The error message was fixed as part of DRILL-3583 . The problem is that the query uses the SUM function on a column with string or boolean type. Thank you, Sudheesh > On Sep 22, 2015, at 3:54 AM, > wrote: > > I actually solved

Re: NullPointers in type conversions

See below: > On Sep 21, 2015, at 8:22 AM, USC wrote: > > Hi, > This is a system wide setting. Meaning, you need to say > > Alter system set `drill.exec.functions.cast_empty_string_to_null` = true; To clarify, all options available through sys.options (except some used for testing) can be set

Re: How to unsubscribe from the mail group ?

You can get digests; see https://mail-archives.apache.org/mod_mbox/drill-user/ > On Nov 15, 2015, at 6:24 PM, Kim Chew wrote: > > https://drill.apache.org/mailinglists/ > > On Sun, Nov 15, 2015 at 4:03 PM, ganesh wrote: > >> Hi, >> >>

Re: Announcing new committer: Kristine Hahn

Congratulations and welcome, Kris! > On Dec 4, 2015, at 9:19 AM, Jacques Nadeau wrote: > > The Apache Drill PMC is very pleased to announce Kristine Hahn as a new > committer. > > Kris has worked tirelessly on creating and improving the Drill > documentation. She has been extraordinary in her e

Re: SYSTEM ERROR: StackOverflowError on querying a Hive table

1) What version of Drill are you using? 2) Can you enable verbose messaging (ALTER SESSION SET `exec.errors.verbose` = true;) and post the resulting error message? Also, please check if there are any relevant ERROR messages in the log file. Thank you, Sudheesh > On Dec 6, 2015, at 11:28 PM, Dur

Re: Rest & web authentication

Hi Niko, Web authentication is not available yet; DRILL-3201 is being reviewed. The doc pages are ahead. Thank you, Sudheesh > On Dec 7, 2015, at 7:06 AM, Keys Botzum wrote: > > I think this answers your question (the answer is yes according

Re: Rest & web authentication

on. >> >> See the discussion thread here for more info. >> >> http://search-hadoop.com/m/qRVAXir9Do1qHni72&subj=How+to+setup+user+authentication+for+the+WebUI+ >> < >> http://search-hadoop.com/m/qRVAXir9Do1qHni72&subj=How+to+setup+user+authen

[DISCUSS] New Feature: Drill Client Impersonation

Hey y’all, I plan to work on DRILL-4281 : support for inbound/client impersonation. Please review the design document , which is open for comments. There is also a

Re: [DISCUSS] New Feature: Drill Client Impersonation

he application servers blinded them to the identity of >>> the >>>> end user accessing important corporate data. When JEE application >> servers >>>> and databases finally added the ability to impersonate that addressed a >>> lot >>>> of securi

Re: [DISCUSS] New Feature: Drill Client Impersonation

> Do you have an interface proposal? I didn't see that. > > Also, what do you think about my comment and Keys response about moving > pooling to the Driver and then making "connection" lightweight. > > -- > Jacques Nadeau > CTO and Co-Founder, Dremio &g

Re: Hangout Happening Now!

Same here, I am unable to connect. I get a “Requesting to join the video call..” and then nothing happens. > On Feb 23, 2016, at 10:12 AM, Parth Chandra wrote: > > Hey Jason, > > We're unable to connect to the hangout. Should we try restarting the > hangout? > > Parth > > On Tue, Feb 23, 20

Re: [DISCUSS] New Feature: Drill Client Impersonation

the >> application. I believe some other databases like SQLServer let you change >> the user via SQL. >> >> With regards to interfacing the impersonation feature, it looks like all >> you need is the username, which is already being pass down from the >

Re: ConnectTimeoutException when starting drill

Drill 1.6 introduces support for Java 1.8, which is generally available today. Download here . Thank you, Sudheesh > On Mar 3, 2016, at 1:40 PM, Rob Terpilowski wrote: > > I am attempting to get

Re: high latency of querying hive datasource

Can you gather the query profile from the web UI [1]? This mailing list does not accept attachments; please put the profile in a shareable location (Dropbox?), and post a link. Thank you, Sudheesh [1] https://drill.apache.org/docs/query-profiles/

Re: Impersonation

Hi Lunen, Is the drillbit process running? If not, look for errors in logs in */opt/apache-drill-1.6.0/log* In embedded mode, Zookeeper is not required. Try: *bin/sqlline –u "jdbc:drill:zk=local" -n user1 -p user1PW* This command starts and connects to a drillbit (which is embedded in the sqllin

Hangout Topics For 05/31/16

Hey y’all, Let’s use this thread to pre-populate a list of topics to discuss on Tuesday’s hangout (05/31/16), so people can attend if they are interested in the mentioned topics. I will also collect topics at the beginning of the hangout. + DRILL-4280: Kerberos Authentication (Sudheesh) Thank

Re: Oracle Query Problem

Can you enable verbose logging and post the resulting error message? You can do this by executing the following statement, and then the failing query. SET `exec.errors.verbose` = true; Thank you, Sudheesh > On May 31, 2016, at 11:27 AM, SanjiV SwaraJ wrote: > > Hello I have Oralce Query for S

Suggestions for Hangout Topics for 06/28/16

If you have any suggestions for hangout topics for tomorrow, you can add it to this thread. We will also ask around at the beginning of the hangout for any topics. The goal is to try to cover whatever possible during the 1 hr. Thank you, Sudheesh

Hangout Starting Now

Hangout starting now: https://hangouts.google.com/hangouts/_/event/ci4rdiju8bv04a64efj5fedd0lc Join us! Thank you, Sudheesh

Re: Drill on Kerberos (Hive Storage plugin)

Can you share the Hive storage plugin configuration? Thank you, Sudheesh > On Jul 3, 2016, at 7:00 PM, Santosh Kulkarni > wrote: > > Hi, > > Drill query gives the following error for Hive storage plugin. > > show databases; > > Error: SYSTEM ERROR: GSSException: No valid credentials provide

Re: Drill - Hive - Kerberos

Can you set the following property to false in the Hive storage plugin configuration, and try again? "hive.server2.enable.doAs": “false" Thank you, Sudheesh > On Jul 5, 2016, at 11:29 AM, Joseph Swingle wrote: > > Yes. Impersonation is enabled.: > > drill.exec: { > > cluster-id: "hhe", >

Re: Olingo plugin

Hi Steve, AFAIK, no such plans. Would you like to open a ticket, and work on it? Thank you, Sudheesh > On Jul 12, 2016, at 10:10 AM, Steve Warren wrote: > > Are there plans to release an Olingo (odata) plugin? > > https://olingo.apache.org/ > > -- > Confidentiality Notice and Disclaimer: T

Re: Olingo plugin

ng plugins. Is > there any available? > > On Tue, Jul 12, 2016 at 10:55 AM, Sudheesh Katkam > wrote: > >> Hi Steve, >> >> AFAIK, no such plans. Would you like to open a ticket, and work on it? >> >> Thank you, >> Sudheesh >> >>> On J

Re: Olingo plugin

t; > Thanks! I'll have a look, I had found the contrib in github. > > On Tue, Jul 12, 2016 at 11:12 AM, Sudheesh Katkam <mailto:skat...@maprtech.com>> > wrote: > >> There is some documentation available [1]. >> >> There are five implementations i

Re: Pushdown Capabilities with RDBMS

Hi Marcus, I am glad that you are exploring Drill! Per RDBMS storage plugin documentation [1], join pushdown is supported. So the scenario you described is likely a bug; can you open a ticket [2] with the details on how to reproduce the issue? Thank you, Sudheesh [1] https://drill.apache.org/d

[DISCUSS] New Feature: Kerberos Authentication

Hi all, I plan to work on DRILL-4280: Kerberos Authentication for Clients [1]. The design document [2] is attached to the ticket. Please read and comment! Thank you, Sudheesh [1] https://issues.apache.org/jira/browse/DRILL-4280 [2] https://docs.google.com/document/d/1qSBV2Hi3KwaDFADZJm9me2Nq4Rqn

Changes in Launch Scripts

Hi all, I just committed DRILL-4581 [1] that changes launch scripts. The patch should be backward compatible. This email is just an FYI to start using the new style of drill-env.sh file. The major usability change is that Drill defaults have been moved from conf/drill-env.sh to bin/drill-config.s

Re: tmp noexec

epoll was supposed to be disabled as part of this PR [1] pending perf. tests; IIRC test clusters were busy then, and I lost track of this change. I’ll post an update soon. Thank you, Sudheesh [1] https://github.com/apache/drill/pull/486 > On Jul 26,

Re: Connecting Drill to Azure Data Lake

What failure(s) do you see? Thank you, Sudheesh > On Jul 29, 2016, at 4:07 PM, Kevin Verhoeven > wrote: > > Hi Drill Community, > > Has anyone attempted to connect Drill to the Azure Data Lake? Microsoft has > implemented a WebHDFS API over Azure Data Lake, so Drill should be able to > conn

Re: [Drill-Issues] Drill-1.6.0: Drillbit not starting

Can you check if there are any errors in the drillbit.out file? This file should be in the same directory as the log file. Thank you, Sudheesh > On Aug 4, 2016, at 4:25 AM, Shankar Mane wrote: > > I am getting this error infrequently. Most of the time drill starts > normally and sometimes this

Re: Fetch queries status from drill prompt

Profiles of running queries are stored in Zookeeper (or the configured transient store). Thank you, Sudheesh > On Aug 18, 2016, at 11:23 AM, Anup Tiwari wrote: > > Thanks chun for info.. > > But can you tell me from where, running queries status come on profile > user-interface(UI)? Because i

Re: Query hangs on planning

That setting is for off-heap memory. The earlier case hit heap memory limit. > On Sep 1, 2016, at 11:36 AM, Zelaine Fong wrote: > > One other thing ... have you tried tuning the planner.memory_limit > parameter? Based on the earlier stack trace, you're hitting a memory limit > during query plan

Re: NULL values for DATE type columns using JDBC connector

Hi Dan, Per documentation , I don’t think sql4es is in the list of actively tested JDBC drivers. But a “WHERE something IS NOT NULL” returning NULL looks like a bug to me. Can you open a ticket ? T

Re: Authentication with PAM fails with Centos 7

Hi Pradeeban, I am not entirely sure what the problem is. The important part from the stack trace is here: > Invalid user credentials: PAM profile 'pkathi2' validation failed From code

Re: Right outer join fails

Hi Kathir, I tried the same query in embedded mode, and I got a different error. java.lang.IndexOutOfBoundsException: index: 0, length: 8 (expected: range(0, 0)) at io.netty.buffer.DrillBuf.checkIndexD(DrillBuf.java:123) at io.netty.buffer.DrillBuf.chk(DrillBuf.java:147) a

Re: Quering Hbase with TimeRange, Version, Timestamp,etc

AFAIK, not possible currently. Thank you, Sudheesh > On Sep 14, 2016, at 5:15 AM, Ulf Andreasson @ MapR > wrote: > > I am also curious regarding this, is it possible ? > > > > Ulf Andreasson | Ericsson Global Alliance Solution Engineer, MapR.com | +46 > 72 700 2295 > > > On Fri, M

Re: Queries over Swift?

AFAIK, there is no documentation. I am not sure anyone has tried it before. That said, from [1], Swift enables Apache Hadoop applications - including MapReduce jobs, read and write data to and from instances of the OpenStack Swift object store. And Drill uses the HDFS client library. So using Sw

Re: Support for Alluxio

Based on the documentation [1], Alluxio not only uses HDFS as a underlying storage system but also provides a “Hadoop API” to clients, and Drill uses this API. So it should be possible. Create a storage plugin named “alluxio” whose contents match “dfs” plugin, and then make changes. Not sure wh

Re: drill rest api converts all data types to string

Hi Niek, That is a bug; thank you for digging out the exact location. Please open a ticket . Let’s start tracking the details of the fix in that ticket. Thank you, Sudheesh > On Sep 15, 2016, at 2:49 AM, Niek Bartholomeus wrote: > > Hi, > > I'm u

Re: Drill 1.8.0 Error: RESOURCE ERROR: Failed to create schema tree.

This is how to get a verbose error: First set the option: > SET `exec.errors.verbose` = true; And then run the query. The detailed output will point us to where the error occurred. Thank you, Sudheesh > On Sep 15, 2016, at 9:12 PM, Abhishek Girish > wrote: > > Hi Kartik, > > Can you take a

Re: drill rest api converts all data types to string

e/DRILL-2373 > > -- > Jacques Nadeau > CTO and Co-Founder, Dremio > > On Fri, Sep 16, 2016 at 8:56 AM, Sudheesh Katkam > wrote: > >> Hi Niek, >> >> That is a bug; thank you for digging out the exact location. Please open a >> ticket <https://is

Re: Right outer join fails

IndexOutOfBoundsException. Somehow, the >>> IndexOutOfBoundsException >>> log is getting suppressed and only the QueryDataBatch error is logged. But >>> thats a separate issue. >>> >>> I did run it from WebUI and its in RUNNING state forever (actually i

[HANGOUT] Topics for 10/04/16

Hi drillers, Our bi-weekly hangout is tomorrow (10/04/16, 10 AM PT). If you have any suggestions for hangout topics, you can add them to this thread. We will also ask around at the beginning of the hangout for topics. Thank you, Sudheesh

Re: [HANGOUT] Topics for 10/04/16

oin tomorrow hangout in case people have questions about > those. > > Cheers, > > Laurent > > On Mon, Oct 3, 2016 at 10:28 AM, Subbu Srinivasan > wrote: > >> Can we close on https://github.com/apache/drill/pull/518 ? >> >> On Mon, Oct 3, 2016 at 10:

Hangout starting now..

Link: https://hangouts.google.com/hangouts/_/maprtech.com/drillbi-weeklyhangout Thank you, Sudheesh

Re: Apache Drill Hangout Minutes - 11/1/16

Hi Laurent, That's right; this was mentioned in the design document. I am piggybacking on previous changes that break the "newer clients talking to older servers" compatibility. For example, as I understand, some resolved sub-tasks of DRILL-4714 [1] *implicitly* break this compatibility; say the

Re: Apache Drill Hangout Minutes - 11/1/16

> > > > > > > -- > Jacques Nadeau > CTO and Co-Founder, Dremio > > On Tue, Nov 1, 2016 at 8:44 PM, Sudheesh Katkam > wrote: > > > Hi Laurent, > > > > That's right; this was mentioned in the design document. > > > > I am

[ANNOUNCE] Apache Drill 1.9.0 Released

On behalf of the Apache Drill community, I am happy to announce the release of Apache Drill 1.9.0. For information about Apache Drill, and to get involved, visit the project website: https://drill.apache.org/ This release introduces new features and enhancements, including asynchronous Parquet r

Re: [Drill 1.9.0] : [CONNECTION ERROR] :- (user client) closed unexpectedly. Drillbit down?

Is the drillbit service (running on datanodeN/10.*.*.5:31010) actually down when the error is seen? If not, try lowering parallelism using these two session options, before running the queries: planner.width.max_per_node (decrease this) planner.slice_target (increase this) Thank you, Sudheesh

Re: [Drill 1.9.0] : [CONNECTION ERROR] :- (user client) closed unexpectedly. Drillbit down?

act to query and do i have to set this at > session level OR system level? > > > > Regards, > *Anup Tiwari* > > On Tue, Dec 20, 2016 at 11:59 PM, Chun Chang wrote: > >> I am pretty sure this is the same as DRILL-4708. >> >> On Tue, Dec 20, 2016 at 10

Re: Impersonation with Drill Web Console or REST API

Maybe the doc should say that Drill supports impersonation through web console. These clients use Java client library, just like JDBC. Note that *inbound* impersonation is not supported yet because Drill does not expose an “impersonation_target” field through the web login form. Thank you, Sudh

[HANGOUT] Topics for 01/23/17

Hi drillers, Our bi-weekly hangout is tomorrow (01/23/17, 10 AM PT). If you have any suggestions for hangout topics, you can add them to this thread. We will also ask around at the beginning of the hangout for topics. Thank you, Sudheesh

Re: [HANGOUT] Topics for 01/24/17

I meant 01/24/17, 10 AM PT. > On Jan 23, 2017, at 12:43 PM, Sudheesh Katkam wrote: > > Hi drillers, > > Our bi-weekly hangout is tomorrow (01/23/17, 10 AM PT). If you have any > suggestions for hangout topics, you can add them to this thread. We will also > ask around at

Re: [HANGOUT] Topics for 01/24/17

Join us here: https://plus.google.com/hangouts/_/event/ci4rdiju8bv04a64efj5fedd0lc On Jan 23, 2017, at 6:43 PM, Sudheesh Katkam mailto:skat...@mapr.com>> wrote: I meant 01/24/17, 10 AM PT. On Jan 23, 2017, at 12:43 PM, Sudheesh Katkam mailto:skat...@mapr.com>> wrote: Hi drill

Re: questions about storage plugin

Can you check for relevant info in the drillbit.log file (either /var/log/drill/ or ${DRILL_HOME}/log)? > On Jan 31, 2017, at 1:50 PM, Pravin Joshi wrote: > > I am trying to create storage plugin for snowflake and am getting Please > retry: error (unable to create/ update storage) error. JDBC c

Re: S3 using IAM roles

Glad you could figure this out! Can you open a ticket with details? https://issues.apache.org/jira/browse/DRILL - Sudheesh On Apr 3, 2017, at 12:42 PM, Knapp, Michael mailto:michael.kn...@capitalone.com>> wrote: All, In case others are in the same situation as I am, I will tell you how I solv

Re: Strange results with date_trunc 'QUARTER'

Looks like a bug to me. Please open a ticket. A simple repro would be very useful. https://issues.apache.org/jira/browse/DRILL - Sudheesh On Apr 3, 2017, at 2:11 PM, Joel Wilsson mailto:joel.wils...@gmail.com>> wrote: Hi, I'm seeing some strange results when trying to group by date_trunc('QU

[HANGOUT] Topics for 04/18/17

Hi drillers, Our bi-weekly hangout is tomorrow (04/18/17, 10 AM PT). If you have any suggestions for hangout topics, you can add them to this thread. We will also ask around at the beginning of the hangout for topics. Hangout link: https://plus.google.com/hangouts/_/event/ci4rdiju8bv04a64efj5f

Re: multiple users and passwords

Drill supports impersonation (“outbound”) to HDFS and Hive; this works because the client API allows for inbound impersonation. In your use-case, does the backend database allow Drill to impersonate the end user (“Joe”) i.e. does the database support inbound impersonation? If so, there still ne

Re: multiple users and passwords

To clarify: Passing *end user* credentials from one service to another *to reach the target service* is messy. On May 2, 2017, at 1:06 PM, Sudheesh Katkam mailto:skat...@mapr.com>> wrote: Drill supports impersonation (“outbound”) to HDFS and Hive; this works because the client API allo

Re: setting an administrator

There are system options that define the list of users and list of groups that are considered administrators. By default, the user running the drillbit is the administrator. System options can only be changed by administrators. So login as an administrator through sqlline and run “ALTER SYSTEM