Re: show tables taking long time on hive plugin

2016-08-18 Thread Jinfeng Ni
You are right that there is improvement room for "Show table" command. Today, Drill process "Show table" as a query against INFORMATION_SCHEMA.TABLE WHERE schema = 'Hive_schema', which will not only get the table names but also table instances. In fact, "Show Table" should only return table names.

Re: EMR + Error closing operators

2016-08-18 Thread Jinfeng Ni
Glad to see you figure out the cause of the problem! I thought the drillbit which run out of spilling disk space would capture the error stack trace in it's drillbit.log. If the query is executed on multiple nodes, you probably need concat drillbit.log from all the nodes and search through the en

Odd behaviour with use dfs

2016-08-18 Thread Joe Vullo
Hi there I’ve noticed some odd behaviour when using drill and am unsure of where to ask about it. When using the web page displayed by drill on localhost:8047/query (by default) running the following commands fail: use dfs.mydfs; and then: show files; Then I receive this error:

Re: Fetch queries status from drill prompt

2016-08-18 Thread Sudheesh Katkam
Profiles of running queries are stored in Zookeeper (or the configured transient store). Thank you, Sudheesh > On Aug 18, 2016, at 11:23 AM, Anup Tiwari wrote: > > Thanks chun for info.. > > But can you tell me from where, running queries status come on profile > user-interface(UI)? Because i

Re: Fetch queries status from drill prompt

2016-08-18 Thread Anup Tiwari
Thanks chun for info.. But can you tell me from where, running queries status come on profile user-interface(UI)? Because if it's coming on profile UI then it must have some back end file or something like that.. On 18-Aug-2016 11:37 PM, "Chun Chang" wrote: Anup, I believe only when a query is

Re: Fetch queries status from drill prompt

2016-08-18 Thread Chun Chang
Anup, I believe only when a query is in a "terminal", i.e. cancelled/completed/failed state, then it is written to the drillbit_queries.json file on the foreman node. If what you want to do is monitoring queries running on your cluster, your best bet is to configure your cluster to store profile i

Re: EMR + Error closing operators

2016-08-18 Thread Stefan Sedich
Ok got it! looks like it was the spil directory, http://www.openkb.info/2016/04/how-to-use-mapr-local-volume-as-spill.html created this and I am good to go! Is there any reason why the EMR template does not do this? considering the root volume on these EMR instances seems to be very small. On Thu

Re: EMR + Error closing operators

2016-08-18 Thread Stefan Sedich
Spun up a fresh cluster and I can see a useful error now, looks like it is due to no space, I assume it is due to the root volume being almost full (this is a fresh EMR install). When running in EMR is there any drill config I should be changing to avoid this? [Error Id: 650c9f25-760e-498d-9f2c-

Drill config convenience: new site directory

2016-08-18 Thread Paul Rogers
Recent changes to the Drill launch scripts give us a new option for simplifying Drill upgrade in Drill 1.8 and later. The new scripts support a “site directory” that holds all our site-specific files separate from the Drill product directory. Before now, we could put our config files in a confi

Re: show tables taking long time on hive plugin

2016-08-18 Thread Sungwook Yoon
Yes, but... It has not much to do with taking 2mins reading tables compared to 1 second from beeline... Beeline is not fast due to caching.. Sungwook On Thu, Aug 18, 2016 at 7:49 AM, Zelaine Fong wrote: > Have you tried tweaking Drill's Hive metastore cache parameters? See > https://drill.ap

Re: show tables taking long time on hive plugin

2016-08-18 Thread Zelaine Fong
Have you tried tweaking Drill's Hive metastore cache parameters? See https://drill.apache.org/docs/hive-metadata-caching/. -- Zelaine On Wed, Aug 17, 2016 at 10:02 PM, Sungwook Yoon wrote: > Hi, > > Drill 1.6.0 > MapR 3.1.1 > Hive 0.13 > HS2 SQL Authorization > Hive Meta storage authorization

Fetch queries status from drill prompt

2016-08-18 Thread Anup Tiwari
Hi All, We want to see all types of queries which ran on drill cluster or currently running from drill prompt, Can someone help us on this? To achieve above , we read the drill documentation and set up a storage plugin to access local file system and able to query *"drillbit_queries.json"* log fi