[Drill-Issues] Drill-1.6.0: CTAS - No table found in case select returns empty set.

[Shankar Mane]

Is there any configuration to do, To force create table even when SELECT
statement return empty data ?


0: jdbc:drill:> create table glv_abcookie_min_newsid as
. . . . . . . > select
. . . . . . . > app_sessionid,
. . . . . . . > new_sessionid,
. . . . . . . > min(serverTime) as min_serverTime
. . . . . . . > from
. . . . . . . > base_newsid
. . . . . . . > where
. . . . . . . > STRPOS(cookie, 'Opera') > 0
. . . . . . . > group by
. . . . . . . > app_sessionid,
. . . . . . . > new_sessionid ;
+---++
| Fragment  | Number of records written  |
+---++
| 1_2   | 0  |
| 1_1   | 0  |
| 1_3   | 0  |
| 1_0   | 0  |
| 1_5   | 0  |
| 1_4   | 0  |
| 1_6   | 0  |
| 1_7   | 0  |
| 1_8   | 0  |
| 1_10  | 0  |
| 1_11  | 0  |
| 1_12  | 0  |
| 1_14  | 0  |
| 1_9   | 0  |
| 1_15  | 0  |
| 1_18  | 0  |
| 1_13  | 0  |
| 1_19  | 0  |
| 1_16  | 0  |
| 1_17  | 0  |
| 1_20  | 0  |
| 1_21  | 0  |
| 1_22  | 0  |
+---++
23 rows selected (39.99 seconds)

0: jdbc:drill:> select * from glv_abcookie_min_newsid limit 1 ;
Error: VALIDATION ERROR: From line 1, column 15 to line 1, column 37: Table
'glv_abcookie_min_newsid' not found

SQL Query null

[Error Id: de854901-99a3-4237-90be-0162f5d995d2 on datanode2:31010]
(state=,code=0)

Re: [Drill-Issues] Drill-1.6.0: Drillbit not starting

[Sudheesh Katkam]

Can you check if there are any errors in the drillbit.out file? This file 
should be in the same directory as the log file.

Thank you,
Sudheesh

> On Aug 4, 2016, at 4:25 AM, Shankar Mane  wrote:
> 
> I am getting this error infrequently. Most of the time drill starts
> normally and sometimes this gives below error. I am running drill 1.6.0 in
> cluster mode. ZK has also setup.
> 
> Could some one please explain where the issue is ?
> 
> 
> 
> 2016-08-04 03:45:15,870 [main] INFO  o.a.d.e.s.s.PersistentStoreRegistry -
> Using the configured PStoreProvider class: 'org.apache.drill.exec.store.sy
> s.store.provider.ZookeeperPersistentStoreProvider'.
> 2016-08-04 03:45:16,430 [main] INFO  o.apache.drill.exec.server.Drillbit -
> Construction completed (1294 ms).
> 2016-08-04 03:45:28,250 [main] WARN  o.apache.drill.exec.server.Drillbit -
> Failure on close()
> java.lang.NullPointerException: null
>at
> org.apache.drill.exec.work.WorkManager.close(WorkManager.java:157)
> ~[drill-java-exec-1.6.0.jar:1.6.0]
>at
> org.apache.drill.common.AutoCloseables.close(AutoCloseables.java:76)
> ~[drill-common-1.6.0.jar:1.6.0]
>at
> org.apache.drill.common.AutoCloseables.close(AutoCloseables.java:64)
> ~[drill-common-1.6.0.jar:1.6.0]
>at org.apache.drill.exec.server.Drillbit.close(Drillbit.java:149)
> [drill-java-exec-1.6.0.jar:1.6.0]
>at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:283)
> [drill-java-exec-1.6.0.jar:1.6.0]
>at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:261)
> [drill-java-exec-1.6.0.jar:1.6.0]
>at org.apache.drill.exec.server.Drillbit.main(Drillbit.java:257)
> [drill-java-exec-1.6.0.jar:1.6.0]
> 2016-08-04 03:45:28,250 [main] INFO  o.apache.drill.exec.server.Drillbit -
> Shutdown completed (1819 ms).

[Drill-Issues] Drill-1.6.0: Drillbit not starting

[Shankar Mane]

I am getting this error infrequently. Most of the time drill starts
normally and sometimes this gives below error. I am running drill 1.6.0 in
cluster mode. ZK has also setup.

Could some one please explain where the issue is ?



2016-08-04 03:45:15,870 [main] INFO  o.a.d.e.s.s.PersistentStoreRegistry -
Using the configured PStoreProvider class: 'org.apache.drill.exec.store.sy
s.store.provider.ZookeeperPersistentStoreProvider'.
2016-08-04 03:45:16,430 [main] INFO  o.apache.drill.exec.server.Drillbit -
Construction completed (1294 ms).
2016-08-04 03:45:28,250 [main] WARN  o.apache.drill.exec.server.Drillbit -
Failure on close()
java.lang.NullPointerException: null
at
org.apache.drill.exec.work.WorkManager.close(WorkManager.java:157)
~[drill-java-exec-1.6.0.jar:1.6.0]
at
org.apache.drill.common.AutoCloseables.close(AutoCloseables.java:76)
~[drill-common-1.6.0.jar:1.6.0]
at
org.apache.drill.common.AutoCloseables.close(AutoCloseables.java:64)
~[drill-common-1.6.0.jar:1.6.0]
at org.apache.drill.exec.server.Drillbit.close(Drillbit.java:149)
[drill-java-exec-1.6.0.jar:1.6.0]
at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:283)
[drill-java-exec-1.6.0.jar:1.6.0]
at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:261)
[drill-java-exec-1.6.0.jar:1.6.0]
at org.apache.drill.exec.server.Drillbit.main(Drillbit.java:257)
[drill-java-exec-1.6.0.jar:1.6.0]
2016-08-04 03:45:28,250 [main] INFO  o.apache.drill.exec.server.Drillbit -
Shutdown completed (1819 ms).

Re: Drill Issues

[Khurram Faraaz]

You can use CONVERT_TO and CONVERT_FROM functions, to get HDFS bytes
read/write from drill queries.

https://drill.apache.org/docs/supported-data-types/#data-types-for-convert_to-and-convert_from-functions

On Mon, May 23, 2016 at 11:28 AM, vinita.go...@siemens.com <
vinita.go...@siemens.com> wrote:

> Hi
>
> How do I get HDFS bytes read and write information for drill queries?
>
>
>
> Thanks and Regards,
> Vinita Goyal
>
>
>
>
>

Re: Drill Issues

[Tushar Pathare]

Hello Vinita,
A simple storage plugins for Hive looks like this.

{
  "type": "hive",
  "enabled": true,
  "configProps": {
"hive.metastore.uris": "thrift://scflexnode09:9083",
"hive.metastore.sasl.enabled": "false",
"fs.default.name": "hdfs://scflexnode09:8020/"
  }
}


Hopes this helps


Tushar B Pathare
High Performance Computing (HPC) Administrator
General Parallel File System
Scientific Computing
Bioinformatics Division
Research

Sidra Medical and Research Centre
Sidra OPC Building
PO Box 26999  |  Doha, Qatar
Near QNCC,5th Floor
Office 4003  ext 37443 | M +974 74793547
tpath...@sidra.org | www.sidra.org 



Follow us on:
 
 
 
 
 











On 5/23/16, 9:00 AM, "vinita.go...@siemens.com"  
wrote:

>Hi..
>
>How can I use Hive tables for running drill queries. What will be the storage 
>plug-in? Please help I tried it a lot But could not get how to do.
>
>Thanks and Regards
>Vinita Goyal
Disclaimer: This email and its attachments may be confidential and are intended 
solely for the use of the individual to whom it is addressed. If you are not 
the intended recipient, any reading, printing, storage, disclosure, copying or 
any other action taken in respect of this e-mail is prohibited and may be 
unlawful. If you are not the intended recipient, please notify the sender 
immediately by using the reply function and then permanently delete what you 
have received. Any views or opinions expressed are solely those of the author 
and do not necessarily represent those of Sidra Medical and Research Center.

RE: Drill Issues

[vinita.go...@siemens.com]

Hi..

How can I use Hive tables for running drill queries. What will be the storage 
plug-in? Please help I tried it a lot But could not get how to do.

Thanks and Regards
Vinita Goyal

Fwd: Drill Issues

[Sanjiv Kumar]

Hello
  I am facing some problem while using drill. I have also posted
earlier my problem one by one, but didn't get any proper solution for that.
This time again i am posting my problem.

1) Join Query Problem (for details check in April 16  with *subject*:-
Regarding
Join Query Problem)

2) Multiple Column having same name & same data type problem (for details
check in April 16 *Subject*:- Multiple Column having same name & same data
type problem)

3) Authentication in Window Operating System (for details check in April 16
*SUBJECT*:- Regarding Drill Authentication)


I hope  this time i get some proper solution from Drill.



 ..
  Thanks & Regards
  *Sanjiv Kumar*



-- 
 ..
  Thanks & Regards
  *Sanjiv Kumar*

Re: Drill Issues

[Chunhui Shi]

Hi Sanjiv,

3)  As to authentication in windows, I think you already got a proper
reply, that is you have to implement your own authenticator. Depends on
your windows environment, you can decide either call windows API to check
user+password, or connect Active Directory either through LDAP or Kerberos
to get authenticated, I think there are some example around in mail lists.

2) As to this one, I think you did not provide the requested information --
the plan of the problematic query, you can get the plan by running "explain
plan for " in sqlline.

1) For 1 and 2, I think you can file a JIRA for each of the issues, and
post plan and attach drillbit.log to the JIRA.

On Tue, May 17, 2016 at 11:41 PM, Khurram Faraaz 
wrote:

> Hello Sanjiv,
>
> I work with the Drill team. I don't have your previous email (I joined the
> user group recently). I can take a look at your join query problem and your
> multiple columns having same name and same type issue.
>
> Please share/resend your earlier email, and I can take a look.
> Two things (1) please share the explain plan for your join query problem,
> if you can get one. (2) please share information from drillbit.log related
> to your same name same type many columns issue.
>
> Also I assume your Drillbits are setup on a Linux/UNIX environment.
>
> Thanks,
> Khurram
>
> On Wed, May 18, 2016 at 11:37 AM, Sanjiv Kumar 
> wrote:
>
> > Hello
> >   I am facing some problem while using drill. I have also posted
> > earlier my problem one by one, but didn't get any proper solution for
> that.
> > This time again i am posting my problem.
> >
> > 1) Join Query Problem (for details check in April 16  with *subject*:-
> > Regarding
> > Join Query Problem)
> >
> > 2) Multiple Column having same name & same data type problem (for details
> > check in April 16 *Subject*:- Multiple Column having same name & same
> data
> > type problem)
> >
> > 3) Authentication in Window Operating System (for details check in April
> 16
> > *SUBJECT*:- Regarding Drill Authentication)
> >
> >
> > I hope  this time i get some proper solution from Drill.
> >
> >
> >
> >  ..
> >   Thanks & Regards
> >   *Sanjiv Kumar*
> >
>

Re: Drill Issues

[Khurram Faraaz]

Hello Sanjiv,

I work with the Drill team. I don't have your previous email (I joined the
user group recently). I can take a look at your join query problem and your
multiple columns having same name and same type issue.

Please share/resend your earlier email, and I can take a look.
Two things (1) please share the explain plan for your join query problem,
if you can get one. (2) please share information from drillbit.log related
to your same name same type many columns issue.

Also I assume your Drillbits are setup on a Linux/UNIX environment.

Thanks,
Khurram

On Wed, May 18, 2016 at 11:37 AM, Sanjiv Kumar  wrote:

> Hello
>   I am facing some problem while using drill. I have also posted
> earlier my problem one by one, but didn't get any proper solution for that.
> This time again i am posting my problem.
>
> 1) Join Query Problem (for details check in April 16  with *subject*:-
> Regarding
> Join Query Problem)
>
> 2) Multiple Column having same name & same data type problem (for details
> check in April 16 *Subject*:- Multiple Column having same name & same data
> type problem)
>
> 3) Authentication in Window Operating System (for details check in April 16
> *SUBJECT*:- Regarding Drill Authentication)
>
>
> I hope  this time i get some proper solution from Drill.
>
>
>
>  ..
>   Thanks & Regards
>   *Sanjiv Kumar*
>

Drill Issues

[Sanjiv Kumar]

Hello
  I am facing some problem while using drill. I have also posted
earlier my problem one by one, but didn't get any proper solution for that.
This time again i am posting my problem.

1) Join Query Problem (for details check in April 16  with *subject*:-
Regarding
Join Query Problem)

2) Multiple Column having same name & same data type problem (for details
check in April 16 *Subject*:- Multiple Column having same name & same data
type problem)

3) Authentication in Window Operating System (for details check in April 16
*SUBJECT*:- Regarding Drill Authentication)


I hope  this time i get some proper solution from Drill.



 ..
  Thanks & Regards
  *Sanjiv Kumar*

Re: Apache Drill issues

[Paul Rogers]

Hi Ananda,

I’m somewhat new to Drill and I asked the same questions. Here’s what I 
understand (and I hope others will offer any needed corrections.)

Drill uses a flow-based DAG model with no intermediate caching or checkpoints. 
That is a fancy way of saying that data streams from scanners to aggregators to 
your client. There is no way to recover/restart any fragment and preserve query 
semantics because Drill has no means of knowing which rows have already been 
sent upstream by that fragment.

As a result, the failure of any fragment fails the entire query; the recovery 
solution is to rerun the query.

Any Drillbit can act as a foreman; one per query. The Foreman for a query is 
the Drillbit to which your client happens to connect. Each Drillbit uses 
ZooKeeper to monitor the status of all other Drillbits. If a Drillbit dies (or 
stops its ZK heartbeat), the Drillbit drops out of ZK and is assumed dead. Each 
Foreman fails any queries that were active on the failed Drillbit. If it is the 
foreman that dies, then the client handles the failure (I’m a bit unsure of the 
details in this particular case.)

Thanks,

- Paul


> On Apr 5, 2016, at 2:20 AM, Ananda Samal  wrote:
> 
> Hi Team,
> 
> I went through the Architecture of Drill and have couple of questions are
> in my mind . Can you please help me here :
> 
> 1- what is the recovery model/process of Drill ?
> ( If one of the drillbit went down from cluster while processing the data
> , how to recover them .)
> 
> 2- If any of the minor Fragment id went down , Is Foreman able to recover
> that automatically or how it will manage ?
> 
> 3- Is the Foreman keep track of other drillbits which are involved on the
> query executions.If yes how ? If no , then how it will manage if other
> Drillbits went down .?
> 
> Can some one help here .
> 
> -- 
> 
> 
> Thanks ,
> Ananda Samal .
> 
> 
> 
> -- 
> 
> 
> Thanks ,
> Ananda Samal .

Fwd: Apache Drill issues

[Ananda Samal]

Hi Team,

I went through the Architecture of Drill and have couple of questions are
in my mind . Can you please help me here :

1- what is the recovery model/process of Drill ?
 ( If one of the drillbit went down from cluster while processing the data
, how to recover them .)

2- If any of the minor Fragment id went down , Is Foreman able to recover
that automatically or how it will manage ?

3- Is the Foreman keep track of other drillbits which are involved on the
query executions.If yes how ? If no , then how it will manage if other
Drillbits went down .?

Can some one help here .

-- 


Thanks ,
Ananda Samal .



-- 


Thanks ,
Ananda Samal .