XML support in Drill 1.x or 2.0

[Edmon Begoli]

What do you guys think about opening up a JIRA and a design discussion for
supporting XML in Drill in some of the upcoming releases?

I think it will have to be done, sooner or later, in order to cover the
whole spectrum.

I've started doing a little bit of research in what could be the options,
possibly suitable OSS tools, etc.

Re: Apache MetaModel - an interesting future integration target

[Kasper Sørensen]

Hi Jacques,

I think we can best answer that together. I didn't yet dive into all the
details of Drill, but from a high level it seems that we could maybe
provide connectivity to a wide set of databases and file formats, which you
could leverage. The query syntax for all MetaModel connectors is exactly
the same, only the schema/table model may differ. But that means to me that
if you plug in one connector, you probably have them all. The main
complication as I see it is how to mediate the Drill queries to MetaModel
queries. Your query language seems to be richer than MetaModel's,
containing more functions and operators. So how can we best ensure that the
maximum amount of the query gets passed on to MetaModel, without passing on
stuff that MetaModel wouldn't support.

Best regards,
Kasper

2015-09-30 13:43 GMT+02:00 Jacques Nadeau :

> I think I can speak for everyone here that the Drill community is also
> interested in working with other projects, especially Apache products.
> There definitely seems to be some overlap. When we discussed this on the
> list here a couple weeks ago, I think the question was what would be a good
> first step. Do you have any ideas on that front?
>
> --
> Jacques Nadeau
> CTO and Co-Founder, Dremio
>
> On Wed, Sep 30, 2015 at 6:57 AM, Kasper Sørensen <
> i.am.kasper.soren...@gmail.com> wrote:
>
> > Hi guys,
> >
> > I'm PMC of Apache MetaModel and was just watching Tugdual Grall's talk
> here
> > at Apache: Big Data Europe. I have the feeling that we should try to work
> > together, or at least explore some cooperation, since the way I see it
> > there's a big overlap with what MetaModel tries to archieve and what
> Drill.
> > And some differences of course. It seems MetaModel is "metadata first"
> > while Drill allows for a lot more dynamic querying capabilities.
> >
> > Either way, some way of integrating the two projects would be very
> > interesting I think. I'm very open for helping out and giving my input.
> >
> > Someone noticed the lower activity-level we have. Yes, MetaModel isn't
> the
> > biggest project. Basically this conference (Apache: Big Data) is the
> first
> > "PR move" we ever did. I did get a lot of interesting feedback from my
> talk
> > here and I expect that we will attract more people as the word spreads.
> > Integrating with other projects such as Drill is also instrumental for us
> > in doing that, obviously. The other reason might be that Apache MetaModel
> > actually has a pretty long history already, first being founded in
> > 2010-2011 outside of Apache, it's certainly already quite mature and is
> > powering both other open source products (such as DataCleaner) and
> > commercial ones (Stratio Datavis, Human Inference DataHub). Probably more
> > that I don't know of.
> >
> > Anyways... Hope you guys might be interested to do some experimentation
> > here. I think both projects could benefit.
> >
> > Best regards,
> > Kasper Sørensen
> >
>

[GitHub] drill pull request: DRILL-3229: Implement Union type vector

[StevenMPhillips]

GitHub user StevenMPhillips opened a pull request:

https://github.com/apache/drill/pull/180

DRILL-3229: Implement Union type vector



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/StevenMPhillips/drill drill-3229

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/drill/pull/180.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #180


commit e5a529d7a276597f2b62cdcb9a1cab2fae8bc52f
Author: Steven Phillips 
Date:   2015-10-01T10:26:34Z

DRILL-3229: Implement Union type vector




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Improving S3 query performance with cache capabilities

[Stefán Baxter]

Hi Jocelyn,

I have used Tachyon as a caching layer for accessing S3 from Drill. It has
limitations but the latest release of Tachyon solves many of them (I have
not yet tested this).

I'm quite interested in what you are doing and it's relevant for my project
as well.

Please let me know if there is any development on your side and if you are
interested then I will share with you some basic documentation regarding
what needs to be done to connect the 3.

Regards,
 -Stefán

On Thu, Oct 1, 2015 at 9:37 AM, DEMOY, Jocelyn 
wrote:

> Hi all,
>
> I am an architect of a PAAS BI solution and gave apache drill a test
> during a week.
>
> On our current solution, we store our customer data on S3 column files and
> perform in memory computation on a home made nosql engine. We have TB of
> data on S3, but since it's a multi tenant solution, when a end customer
> perform queries it's only on a subset of data in separated S3 folder with
> let's say 1 gb of data max.
>
> We try to have the best response time for real time analytics queries
> (things like 1 sec response time for 500K row aggregation on 4 cols with
> joins). To do so, we load only the necessary columns (we have one file per
> column, not per table) and cache the columns value in local JVM for xx mins.
>
> I built a POC with drill & parquet to replace our computation engine.
> Local execution time are fine and mach our needs. I am quite happy with the
> capability to query on S3 with real SQL syntax. My main problem is the
> latency with S3 (from AWS instances) :  for every query I have to pay the
> download cost of the parquet file from S3, this makes the query response
> time too long for a "real time" solution.
>
> I would like to know if you have plan anything in the roadmap to enable
> some native caching capabilities on the data itself (not only metadata
> caching).
>
> I saw the AbstractStoragePlugin and AbstractRecordReader classes. Would it
> be possible (and a good idea) for us to create a decorator for the classic
> file provider (or a totally new custom S3 provider) with memory cache
> capability. How would this make sense in a drill cluster and in the drill
> philosophy ?
>
> Thanks in advance
>
>
> Jocelyn Demoy
> BI Architect, R and strategy
> Sage
>

Re: XML support in Drill 1.x or 2.0

[Tugdual Grall]

+1
to start the discussion as I see more and more questions about XML in the
context

Tug

On Thu, Oct 1, 2015 at 12:51 PM, Edmon Begoli  wrote:

> What do you guys think about opening up a JIRA and a design discussion for
> supporting XML in Drill in some of the upcoming releases?
>
> I think it will have to be done, sooner or later, in order to cover the
> whole spectrum.
>
> I've started doing a little bit of research in what could be the options,
> possibly suitable OSS tools, etc.
>

Improving S3 query performance with cache capabilities

[DEMOY, Jocelyn]

Hi all,

I am an architect of a PAAS BI solution and gave apache drill a test during a 
week.

On our current solution, we store our customer data on S3 column files and 
perform in memory computation on a home made nosql engine. We have TB of  data 
on S3, but since it's a multi tenant solution, when a end customer perform 
queries it's only on a subset of data in separated S3 folder with let's say 1 
gb of data max.

We try to have the best response time for real time analytics queries (things 
like 1 sec response time for 500K row aggregation on 4 cols with joins). To do 
so, we load only the necessary columns (we have one file per column, not per 
table) and cache the columns value in local JVM for xx mins.

I built a POC with drill & parquet to replace our computation engine. Local 
execution time are fine and mach our needs. I am quite happy with the 
capability to query on S3 with real SQL syntax. My main problem is the latency 
with S3 (from AWS instances) :  for every query I have to pay the download cost 
of the parquet file from S3, this makes the query response time too long for a 
"real time" solution.

I would like to know if you have plan anything in the roadmap to enable some 
native caching capabilities on the data itself (not only metadata caching).

I saw the AbstractStoragePlugin and AbstractRecordReader classes. Would it be 
possible (and a good idea) for us to create a decorator for the classic file 
provider (or a totally new custom S3 provider) with memory cache capability. 
How would this make sense in a drill cluster and in the drill philosophy ?

Thanks in advance


Jocelyn Demoy
BI Architect, R and strategy
Sage

[jira] [Resolved] (DRILL-3877) drill version 1.0.0 is shown in drill shell for 1.1.0 during start up

[Deneche A. Hakim (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-3877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deneche A. Hakim resolved DRILL-3877.
-
Resolution: Duplicate

> drill  version 1.0.0 is shown in drill shell for 1.1.0 during start up 
> ---
>
> Key: DRILL-3877
> URL: https://issues.apache.org/jira/browse/DRILL-3877
> Project: Apache Drill
>  Issue Type: Improvement
>Reporter: Devender Yadav 
>Priority: Trivial
>
> When I started shell for drill 1.1.0:
> impadmin@ubuntu:~/softwares/apache-drill-1.1.0/bin$ ./drill-embedded 
> Oct 01, 2015 2:18:49 AM org.glassfish.jersey.server.ApplicationHandler 
> initialize
> INFO: Initiating Jersey application, version Jersey: 2.8 2014-04-29 
> 01:25:26...
> apache drill 1.0.0 
> "a drill is a terrible thing to waste"
> 0: jdbc:drill:zk=local> 
> apache drill 1.0.0  should be  replaced with apache drill 1.1.0



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Custom SerDes in Drill

[michael.england]

Hi,

Is it possible to use custom SerDe in Drill? From what I can see it is not, but 
there are some proprietary data types that I am interested in querying that 
need a SerDe. I am guessing a Drill query over a Hive table (set up to use a 
custom SerDe) will not work?

Thanks,
Mike


This e-mail (including any attachments) is private and confidential, may 
contain proprietary or privileged information and is intended for the named 
recipient(s) only. Unintended recipients are strictly prohibited from taking 
action on the basis of information in this e-mail and must contact the sender 
immediately, delete this e-mail (and all attachments) and destroy any hard 
copies. Nomura will not accept responsibility or liability for the accuracy or 
completeness of, or the presence of any virus or disabling code in, this 
e-mail. If verification is sought please request a hard copy. Any reference to 
the terms of executed transactions should be treated as preliminary only and 
subject to formal written confirmation by Nomura. Nomura reserves the right to 
retain, monitor and intercept e-mail communications through its networks 
(subject to and in accordance with applicable laws). No confidentiality or 
privilege is waived or lost by Nomura by any mistransmission of this e-mail. 
Any reference to "Nomura" is a reference to any entity in the Nomura Holdings, 
Inc. group. Please read our Electronic Communications Legal Notice which forms 
part of this e-mail: http://www.Nomura.com/email_disclaimer.htm

Re: Timing for 1.2 release, ideas around things to include

[Abdel Hakim Deneche]

I can manage the release.

On Wed, Sep 30, 2015 at 6:10 PM, Aman Sinha  wrote:

> +1 for starting the process of finalizing the 1.2 release and 'bounce
> everything not critical (and not in-progress or reviewable) off of this
> release'.
>
> Aman
>
> On Wed, Sep 30, 2015 at 5:36 PM, Jacques Nadeau 
> wrote:
>
> > Hey All,
> >
> > It seems like we should finalize on the 1.2 release pretty soon. I
> > currently see something like 174 bugs currently open. Of those, ~20 are
> > reviewable and ~20 are in progress. I'm inclined to bounce everything not
> > critical (and not in-progress or reviewable) off of this release. Any
> > thoughts on when everyone would like to do a release? Also, anyone want
> to
> > be release manager?
> >
> > Jacques
> >
> > --
> > Jacques Nadeau
> > CTO and Co-Founder, Dremio
> >
> > On Tue, Aug 18, 2015 at 7:58 PM, Parth Chandra 
> wrote:
> >
> > > A couple of reasons -
> > >   a) We definitely need more QA time for the features that are ready.
> > >   b) Some features, in particular, Parquet metadata caching (Steven)
> and
> > > Faster Hive reads (Jason) are close enough to completion that we really
> > > should include them. Again the team could use the extra time to QA
> these.
> > >
> > >
> > > On Tue, Aug 18, 2015 at 4:32 PM, Chris Westin  >
> > > wrote:
> > >
> > > > Unfortunately, we've had to modify our plans, and are now thinking of
> > > > aiming for a vote on September 10th.
> > > >
> > > > On Thu, Aug 13, 2015 at 1:29 PM, Chris Westin <
> chriswesti...@gmail.com
> > >
> > > > wrote:
> > > >
> > > > > We were thinking of targetting for a vote on August 26th.
> > > > >
> > > > > On Thu, Aug 13, 2015 at 1:19 PM, Jacques Nadeau <
> jacq...@dremio.com>
> > > > > wrote:
> > > > >
> > > > >> Following up since we haven't discussed the 1.2 release in a
> while.
> > > How
> > > > >> are people feeling about trying to get a candidate out in the next
> > > week
> > > > or
> > > > >> two?
> > > > >>
> > > > >> thanks,
> > > > >> Jacques
> > > > >>
> > > > >> --
> > > > >> Jacques Nadeau
> > > > >> CTO and Co-Founder, Dremio
> > > > >>
> > > > >>
> > > > >>
> > > > >> > -- Forwarded message --
> > > > >> > From: Parth Chandra 
> > > > >> > Date: Thu, Jul 9, 2015 at 11:29 AM
> > > > >> > Subject: Re: Timing for 1.2 release, ideas around things to
> > include
> > > > >> > To: dev@drill.apache.org
> > > > >> >
> > > > >> >
> > > > >> > I just moved DRILL-3201 to 1.2. It's one of the candidates for
> > > > inclusion
> > > > >> > into 1.2
> > > > >> >
> > > > >> > On Thu, Jul 9, 2015 at 7:04 AM, 
> > wrote:
> > > > >> >
> > > > >> > > I'm not sure what the appetite is for pushing this into 1.2 vs
> > the
> > > > >> > planned
> > > > >> > > 1.3, but Drill UI Authentication would be really great for
> > > > >> > companies/users
> > > > >> > > that run multi-tenant Hadoop clusters with Drill. At the
> moment
> > > any
> > > > >> user
> > > > >> > > can change storage plugins and cancel running queries, even if
> > > they
> > > > >> did
> > > > >> > not
> > > > >> > > submit them. This is a bit of a blocker for Drill on
> > multi-tenant
> > > > >> > > production clusters for now.
> > > > >> > >
> > > > >> > > JIRA link:
> > > > >> > > https://issues.apache.org/jira/browse/DRILL-3201
> > > > >> > >
> > > > >> > >
> > > > >> > >
> > > > >> > > -Original Message-
> > > > >> > > From: Jacques Nadeau [mailto:jacq...@apache.org]
> > > > >> > > Sent: 08 July 2015 17:19
> > > > >> > > To: dev@drill.apache.org
> > > > >> > > Subject: Timing for 1.2 release, ideas around things to
> include
> > > > >> > >
> > > > >> > > Hey Guys,
> > > > >> > >
> > > > >> > > What do you think about planning the 1.2 feature cutoff around
> > Aug
> > > > >> 21st
> > > > >> > > and target a release vote a week or so after that?
> > > > >> > >
> > > > >> > > What are different features people want to get into 1.2?
> > > > >> > >
> > > > >> > > Here are couple of things that I'd like to help facilitate
> > > > >> contribution
> > > > >> > > and/or inclusion:
> > > > >> > >
> > > > >> > > - Mongo improvements (including extended type fixes 2879 ,
> test
> > > > cases
> > > > >> > 1666,
> > > > >> > > etc)
> > > > >> > > - httpd log parser format plugin (3423)
> > > > >> > > - rpc enhancements (local 2941 and off-thread 3242)
> > > > >> > > - First merge of Magnus' JDBC connector (3180)
> > > > >> > > - New allocator (1942)
> > > > >> > >
> > > > >> > > Other key goals?  What do people think about the proposed
> > timing?
> > > > >> > >
> > > > >> > >
> > > > >> > > This e-mail (including any attachments) is private and
> > > confidential,
> > > > >> may
> > > > >> > > contain proprietary or privileged information and is intended
> > for
> > > > the
> > > > >> > named
> > > > >> > > recipient(s) only. Unintended recipients are strictly
> prohibited
> > > > from
> > > 

Re: Review Request 32248: DRILL-2139: Star is not expanded correctly in "select distinct" query

[Sean Hsuan-Yi Chu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32248/
---

(Updated Oct. 1, 2015, 3:18 p.m.)


Review request for drill, Aman Sinha, Jacques Nadeau, and Jinfeng Ni.


Changes
---

addressed comments


Bugs: DRILL-2139
https://issues.apache.org/jira/browse/DRILL-2139


Repository: drill-git


Description
---

Expand * at the run time


Diffs (updated)
-

  
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/aggregate/AggregateUtils.java
 PRE-CREATION 
  
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/aggregate/HashAggBatch.java
 a033a8e 
  
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/aggregate/StreamingAggBatch.java
 2ab1e66 
  
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/xsort/ExternalSortBatch.java
 49a64cf 
  exec/java-exec/src/test/java/org/apache/drill/TestDistinctStar.java 
PRE-CREATION 
  
exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/agg/TestHashAggr.java
 3786bfd 
  exec/java-exec/src/test/resources/store/text/data/repeatedRows.json 
PRE-CREATION 
  
exec/java-exec/src/test/resources/testframework/testDistinctStar/testSelectDistinct.tsv
 PRE-CREATION 
  
exec/java-exec/src/test/resources/testframework/testDistinctStar/testSelectDistinctExpression.tsv
 PRE-CREATION 
  
exec/java-exec/src/test/resources/testframework/testDistinctStar/testSelectDistinctOverJoin.tsv
 PRE-CREATION 

Diff: https://reviews.apache.org/r/32248/diff/


Testing
---

Unit and all QA tests passed.


Thanks,

Sean Hsuan-Yi Chu

[GitHub] drill pull request: DRILL-3874: flattening large JSON objects uses...

[cwestin]

GitHub user cwestin opened a pull request:

https://github.com/apache/drill/pull/181

DRILL-3874: flattening large JSON objects uses too much direct memory

- add getBufferSizeFor() to ValueVector interface
- add implememtations of getBufferSizeFor() for all ValueVector derivatives
- add adaptive algorithm for adjusting batch size to flatten operator

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/cwestin/incubator-drill flatten-memory

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/drill/pull/181.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #181


commit ca9bccb8657169849e4965e180cdad75e0652c9e
Author: Chris Westin 
Date:   2015-09-29T23:05:00Z

DRILL-3874: flattening large JSON objects uses too much direct memory
- add getBufferSizeFor() to ValueVector interface
- add implememtations of getBufferSizeFor() for all ValueVector derivatives
- add adaptive algorithm for adjusting batch size to flatten operator




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (DRILL-3878) Support XML Querying (selects/projections, no writing)

[Edmon Begoli (JIRA)]

Edmon Begoli created DRILL-3878:
---

 Summary: Support XML Querying (selects/projections, no writing)
 Key: DRILL-3878
 URL: https://issues.apache.org/jira/browse/DRILL-3878
 Project: Apache Drill
  Issue Type: New Feature
Affects Versions: Future
Reporter: Edmon Begoli
 Fix For: Future


Support querying of the XML documents (as read-only selects, 
Writing should be implemented as a different feature that brings its own set of 
challenges.)

To consider is reading of the trivial, schema-less, XML documents, DTD-oriented 
ones and also of schema-defined ones.

Also, we should consider direct querying vs. using converter tools to change 
the representation from XML to JSON, CSV, etc.


Design and Implementation discussion, notes, ideas and implementation 
suggestions should be captured here:
https://docs.google.com/document/d/1oS-cObSaTlAmuW_XghDLmHbBEorLl0z-axaHnjy7vg0/edit?usp=sharing
 
(no vandalism, please)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Custom SerDes in Drill

[Jacques Nadeau]

As long as the custom serde dependencies are also on the Drill classpath,
this should work fine.
On Oct 1, 2015 5:59 AM,  wrote:

> Hi,
>
> Is it possible to use custom SerDe in Drill? From what I can see it is
> not, but there are some proprietary data types that I am interested in
> querying that need a SerDe. I am guessing a Drill query over a Hive table
> (set up to use a custom SerDe) will not work?
>
> Thanks,
> Mike
>
>
> This e-mail (including any attachments) is private and confidential, may
> contain proprietary or privileged information and is intended for the named
> recipient(s) only. Unintended recipients are strictly prohibited from
> taking action on the basis of information in this e-mail and must contact
> the sender immediately, delete this e-mail (and all attachments) and
> destroy any hard copies. Nomura will not accept responsibility or liability
> for the accuracy or completeness of, or the presence of any virus or
> disabling code in, this e-mail. If verification is sought please request a
> hard copy. Any reference to the terms of executed transactions should be
> treated as preliminary only and subject to formal written confirmation by
> Nomura. Nomura reserves the right to retain, monitor and intercept e-mail
> communications through its networks (subject to and in accordance with
> applicable laws). No confidentiality or privilege is waived or lost by
> Nomura by any mistransmission of this e-mail. Any reference to "Nomura" is
> a reference to any entity in the Nomura Holdings, Inc. group. Please read
> our Electronic Communications Legal Notice which forms part of this e-mail:
> http://www.Nomura.com/email_disclaimer.htm
>
>

Re: XML support in Drill 1.x or 2.0

[Edmon Begoli]

I will open JIRA and attach a Google Doc to record design thoughts there.

On Thu, Oct 1, 2015 at 9:33 AM, Jacques Nadeau  wrote:

> Sounds good. The biggest question to me is how to map XML to our json data
> model. I suggest the discussion start there.
> On Oct 1, 2015 6:01 AM, "Tugdual Grall"  wrote:
>
> > +1
> > to start the discussion as I see more and more questions about XML in the
> > context
> >
> > Tug
> >
> > On Thu, Oct 1, 2015 at 12:51 PM, Edmon Begoli  wrote:
> >
> > > What do you guys think about opening up a JIRA and a design discussion
> > for
> > > supporting XML in Drill in some of the upcoming releases?
> > >
> > > I think it will have to be done, sooner or later, in order to cover the
> > > whole spectrum.
> > >
> > > I've started doing a little bit of research in what could be the
> options,
> > > possibly suitable OSS tools, etc.
> > >
> >
>

[jira] [Created] (DRILL-3880) sqlline does not allow for a password prompt - security issue

[John Omernik (JIRA)]

John Omernik created DRILL-3880:
---

 Summary: sqlline does not allow for a password prompt - security 
issue
 Key: DRILL-3880
 URL: https://issues.apache.org/jira/browse/DRILL-3880
 Project: Apache Drill
  Issue Type: Improvement
  Components: Client - CLI
Affects Versions: 1.1.0
Reporter: John Omernik
 Fix For: Future


When authentication is enabled in drill, and using sqlline, there is no way to 
get the sqlline client to prompt for a password. The only option is to specify 
the password at the command line (-n user -p password) or to log in and then 
connect.  

This is a security risk, in that now the .bash_history contains the user's 
password, defeating accountability on the system.  Hive and MYSQL both allow 
for a -p flag with no value to trigger a prompt for the password that is not 
logged by .bash_history. 

One work around is to connect after starting sqlline, however, if the sqlline 
command offers a way to specify the username/password, we should do it in a way 
that doesn't violate security principles. 





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (DRILL-3879) sqlline user authentication provides poor feedback on authentication

[John Omernik (JIRA)]

John Omernik created DRILL-3879:
---

 Summary: sqlline user authentication provides poor feedback on 
authentication
 Key: DRILL-3879
 URL: https://issues.apache.org/jira/browse/DRILL-3879
 Project: Apache Drill
  Issue Type: Improvement
  Components: Client - CLI
Affects Versions: 1.1.0
Reporter: John Omernik
Priority: Minor
 Fix For: Future


When authenticating with sqlline using the -n and -p flags, if you provide an 
incorrect user/name and password, instead of just failing with a bad user/pass 
type message, there is a whole page of java stack trace.  This should be a 
simple trapped message, because for users setting up authentication, the stack 
trace may be overwhelming and they then think the setup is wrong rather than 
setup is correct and user/pass is incorrect. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

RE: Custom SerDes in Drill

[michael.england]

Interesting, I'll give it a go and reply to the mail list to see how I get on.

Thanks,
Mike

-Original Message-
From: Jacques Nadeau [mailto:jacq...@dremio.com] 
Sent: 01 October 2015 14:35
To: dev@drill.apache.org
Subject: Re: Custom SerDes in Drill

As long as the custom serde dependencies are also on the Drill classpath, this 
should work fine.
On Oct 1, 2015 5:59 AM,  wrote:

> Hi,
>
> Is it possible to use custom SerDe in Drill? From what I can see it is 
> not, but there are some proprietary data types that I am interested in 
> querying that need a SerDe. I am guessing a Drill query over a Hive 
> table (set up to use a custom SerDe) will not work?
>
> Thanks,
> Mike
>
>
> This e-mail (including any attachments) is private and confidential, 
> may contain proprietary or privileged information and is intended for 
> the named
> recipient(s) only. Unintended recipients are strictly prohibited from 
> taking action on the basis of information in this e-mail and must 
> contact the sender immediately, delete this e-mail (and all 
> attachments) and destroy any hard copies. Nomura will not accept 
> responsibility or liability for the accuracy or completeness of, or 
> the presence of any virus or disabling code in, this e-mail. If 
> verification is sought please request a hard copy. Any reference to 
> the terms of executed transactions should be treated as preliminary 
> only and subject to formal written confirmation by Nomura. Nomura 
> reserves the right to retain, monitor and intercept e-mail 
> communications through its networks (subject to and in accordance with 
> applicable laws). No confidentiality or privilege is waived or lost by 
> Nomura by any mistransmission of this e-mail. Any reference to 
> "Nomura" is a reference to any entity in the Nomura Holdings, Inc. group. 
> Please read our Electronic Communications Legal Notice which forms part of 
> this e-mail:
> http://www.Nomura.com/email_disclaimer.htm
>
>


This e-mail (including any attachments) is private and confidential, may 
contain proprietary or privileged information and is intended for the named 
recipient(s) only. Unintended recipients are strictly prohibited from taking 
action on the basis of information in this e-mail and must contact the sender 
immediately, delete this e-mail (and all attachments) and destroy any hard 
copies. Nomura will not accept responsibility or liability for the accuracy or 
completeness of, or the presence of any virus or disabling code in, this 
e-mail. If verification is sought please request a hard copy. Any reference to 
the terms of executed transactions should be treated as preliminary only and 
subject to formal written confirmation by Nomura. Nomura reserves the right to 
retain, monitor and intercept e-mail communications through its networks 
(subject to and in accordance with applicable laws). No confidentiality or 
privilege is waived or lost by Nomura by any mistransmission of this e-mail. 
Any reference to "Nomura" is a reference to any entity in the Nomura Holdings, 
Inc. group. Please read our Electronic Communications Legal Notice which forms 
part of this e-mail: http://www.Nomura.com/email_disclaimer.htm

New Slack setup for Devs and Users

[Jacques Nadeau]

Hey Guys,

We've been using Slack a lot internally and have found it very useful. I
setup a new slack for Drill developers and users. I've set it to
automatically accept new users from @apache.org as well as @mapr.com, @
maprtech.com and @dremio.com. I'm more than happy to whitelist other
domains (but slack won't let me enable general domains such as gmail and
yahoo).

If you aren't on one the whitelisted domains, just send me an email and
I'll invite you (or add your domain as appropriate).

Remember, these channels should be used for help as opposed to making
design decisions. My goal is also to post a digest once a week from the
channels back on to the list so that the information is publicly available.

I set up two initial channels: #user and #dev.

Let's see if this makes things easier for everybody.

To Join, go to: https://drillers.slack.com/signup


--
Jacques Nadeau
CTO and Co-Founder, Dremio

[jira] [Created] (DRILL-3881) Rowkey filter does not get pushed into Scan

[Khurram Faraaz (JIRA)]

Khurram Faraaz created DRILL-3881:
-

 Summary: Rowkey filter does not get pushed into Scan
 Key: DRILL-3881
 URL: https://issues.apache.org/jira/browse/DRILL-3881
 Project: Apache Drill
  Issue Type: Bug
  Components: Execution - Flow
Affects Versions: 1.2.0
Reporter: Khurram Faraaz
Assignee: Smidth Panchamia


Rowkey filter does not get pushed down into Scan

4 node cluster CentOS
Drill master commit ID: b9afcf8f

case 1) Rowkey filter does not get pushed into Scan

{code}
0: jdbc:drill:schema=dfs.tmp> explain plan for select 
CONVERT_FROM(ROW_KEY,'FLOAT_OB') AS RK,CONVERT_FROM(T.`colfam1`.`qual1`,'UTF8') 
FROM flt_Tbl T WHERE ROW_KEY = CAST('3.0838087E38' AS FLOAT);
+--+--+
| text | json |
+--+--+
| 00-00Screen
00-01  Project(RK=[CONVERT_FROMFLOAT_OB($0)], 
EXPR$1=[CONVERT_FROMUTF8(ITEM($1, 'qual1'))])
00-02SelectionVectorRemover
00-03  Filter(condition=[=($0, CAST('3.0838087E38'):FLOAT NOT NULL)])
00-04Scan(groupscan=[HBaseGroupScan [HBaseScanSpec=HBaseScanSpec 
[tableName=flt_Tbl, startRow=null, stopRow=null, filter=null], columns=[`*`]]])
{code}

case 2) Rowkey filter does not get pushed into Scan

{code}
0: jdbc:drill:schema=dfs.tmp> explain plan for select 
CONVERT_FROM(ROW_KEY,'FLOAT_OB') AS RK,CONVERT_FROM(T.`colfam1`.`qual1`,'UTF8') 
FROM flt_Tbl T WHERE CONVERT_FROM(ROW_KEY,'FLOAT_OB') = CAST('3.0838087E38' AS 
FLOAT) AND CONVERT_FROM(T.`colfam1`.`qual1`,'UTF8') LIKE '%30838087473969088%' 
order by CONVERT_FROM(ROW_KEY,'FLOAT_OB') ASC;
+--+--+
| text | json |
+--+--+
| 00-00Screen
00-01  Project(RK=[$0], EXPR$1=[$1])
00-02SelectionVectorRemover
00-03  Sort(sort0=[$0], dir0=[ASC])
00-04Project(RK=[CONVERT_FROMFLOAT_OB($0)], 
EXPR$1=[CONVERT_FROMUTF8(ITEM($1, 'qual1'))])
00-05  SelectionVectorRemover
00-06Filter(condition=[AND(=(CONVERT_FROM($0, 'FLOAT_OB'), 
CAST('3.0838087E38'):FLOAT NOT NULL), LIKE(CONVERT_FROM(ITEM($1, 'qual1'), 
'UTF8'), '%30838087473969088%'))])
00-07  Scan(groupscan=[HBaseGroupScan 
[HBaseScanSpec=HBaseScanSpec [tableName=flt_Tbl, startRow=, stopRow=, 
filter=SingleColumnValueFilter (colfam1, qual1, EQUAL, 
^.*\x5CQ30838087473969088\x5CE.*$)], columns=[`*`]]])
{code}

Same as case (2) just that ASC is missing in order by clause.
{code}
0: jdbc:drill:schema=dfs.tmp> explain plan for select 
CONVERT_FROM(ROW_KEY,'FLOAT_OB') AS RK,CONVERT_FROM(T.`colfam1`.`qual1`,'UTF8') 
FROM flt_Tbl T WHERE CONVERT_FROM(ROW_KEY,'FLOAT_OB') = CAST('3.0838087E38' AS 
FLOAT) AND CONVERT_FROM(T.`colfam1`.`qual1`,'UTF8') LIKE '%30838087473969088%' 
order by CONVERT_FROM(ROW_KEY,'FLOAT_OB');
+--+--+
| text | json |
+--+--+
| 00-00Screen
00-01  Project(RK=[$0], EXPR$1=[$1])
00-02SelectionVectorRemover
00-03  Sort(sort0=[$0], dir0=[ASC])
00-04Project(RK=[CONVERT_FROMFLOAT_OB($0)], 
EXPR$1=[CONVERT_FROMUTF8(ITEM($1, 'qual1'))])
00-05  SelectionVectorRemover
00-06Filter(condition=[AND(=(CONVERT_FROM($0, 'FLOAT_OB'), 
CAST('3.0838087E38'):FLOAT NOT NULL), LIKE(CONVERT_FROM(ITEM($1, 'qual1'), 
'UTF8'), '%30838087473969088%'))])
00-07  Scan(groupscan=[HBaseGroupScan 
[HBaseScanSpec=HBaseScanSpec [tableName=flt_Tbl, startRow=, stopRow=, 
filter=SingleColumnValueFilter (colfam1, qual1, EQUAL, 
^.*\x5CQ30838087473969088\x5CE.*$)], columns=[`*`]]])
{code}

Snippet that creates and inserts data into HBase table.

{code}
public static void main(String args[]) throws IOException {
Configuration conf = HBaseConfiguration.create();
conf.set("hbase.zookeeper.property.clientPort","5181");
HBaseAdmin admin = new HBaseAdmin(conf);
if (admin.tableExists("flt_Tbl")) {
admin.disableTable("flt_Tbl");
admin.deleteTable("flt_Tbl");
}

HTableDescriptor tableDesc = new
HTableDescriptor(TableName.valueOf("flt_Tbl"));

tableDesc.addFamily(new HColumnDescriptor("colfam1"));
admin.createTable(tableDesc);

HTable table  = new HTable(conf, "flt_Tbl");

//for (float i = (float)0.5; i <= 100.00; i += 0.75) {
for (float i = (float)1.4E-45; i <= Float.MAX_VALUE; i += 
Float.MAX_VALUE / 64) {
byte[] bytes = new byte[5];
org.apache.hadoop.hbase.util.PositionedByteRange br =
  new org.apache.hadoop.hbase.util.SimplePositionedByteRange(bytes, 
0, 5);
org.apache.hadoop.hbase.util.OrderedBytes.encodeFloat32(br, i,
  org.apache.hadoop.hbase.util.Order.DESCENDING);

Put p = new Put(bytes);

p.add(Bytes.toBytes("colfam1"),Bytes.toBytes("qual1"),String.format("value 
%05f", i).getBytes());
table.put(p);
}
table.flushCommits();
table.close();
}
{code}

Re: Timing for 1.2 release, ideas around things to include

[Parth Chandra]

+1 on the release. I'm all for starting the process ASAP.



On Thu, Oct 1, 2015 at 8:17 AM, Abdel Hakim Deneche 
wrote:

> I can manage the release.
>
> On Wed, Sep 30, 2015 at 6:10 PM, Aman Sinha  wrote:
>
> > +1 for starting the process of finalizing the 1.2 release and 'bounce
> > everything not critical (and not in-progress or reviewable) off of this
> > release'.
> >
> > Aman
> >
> > On Wed, Sep 30, 2015 at 5:36 PM, Jacques Nadeau 
> > wrote:
> >
> > > Hey All,
> > >
> > > It seems like we should finalize on the 1.2 release pretty soon. I
> > > currently see something like 174 bugs currently open. Of those, ~20 are
> > > reviewable and ~20 are in progress. I'm inclined to bounce everything
> not
> > > critical (and not in-progress or reviewable) off of this release. Any
> > > thoughts on when everyone would like to do a release? Also, anyone want
> > to
> > > be release manager?
> > >
> > > Jacques
> > >
> > > --
> > > Jacques Nadeau
> > > CTO and Co-Founder, Dremio
> > >
> > > On Tue, Aug 18, 2015 at 7:58 PM, Parth Chandra 
> > wrote:
> > >
> > > > A couple of reasons -
> > > >   a) We definitely need more QA time for the features that are ready.
> > > >   b) Some features, in particular, Parquet metadata caching (Steven)
> > and
> > > > Faster Hive reads (Jason) are close enough to completion that we
> really
> > > > should include them. Again the team could use the extra time to QA
> > these.
> > > >
> > > >
> > > > On Tue, Aug 18, 2015 at 4:32 PM, Chris Westin <
> chriswesti...@gmail.com
> > >
> > > > wrote:
> > > >
> > > > > Unfortunately, we've had to modify our plans, and are now thinking
> of
> > > > > aiming for a vote on September 10th.
> > > > >
> > > > > On Thu, Aug 13, 2015 at 1:29 PM, Chris Westin <
> > chriswesti...@gmail.com
> > > >
> > > > > wrote:
> > > > >
> > > > > > We were thinking of targetting for a vote on August 26th.
> > > > > >
> > > > > > On Thu, Aug 13, 2015 at 1:19 PM, Jacques Nadeau <
> > jacq...@dremio.com>
> > > > > > wrote:
> > > > > >
> > > > > >> Following up since we haven't discussed the 1.2 release in a
> > while.
> > > > How
> > > > > >> are people feeling about trying to get a candidate out in the
> next
> > > > week
> > > > > or
> > > > > >> two?
> > > > > >>
> > > > > >> thanks,
> > > > > >> Jacques
> > > > > >>
> > > > > >> --
> > > > > >> Jacques Nadeau
> > > > > >> CTO and Co-Founder, Dremio
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > >> > -- Forwarded message --
> > > > > >> > From: Parth Chandra 
> > > > > >> > Date: Thu, Jul 9, 2015 at 11:29 AM
> > > > > >> > Subject: Re: Timing for 1.2 release, ideas around things to
> > > include
> > > > > >> > To: dev@drill.apache.org
> > > > > >> >
> > > > > >> >
> > > > > >> > I just moved DRILL-3201 to 1.2. It's one of the candidates for
> > > > > inclusion
> > > > > >> > into 1.2
> > > > > >> >
> > > > > >> > On Thu, Jul 9, 2015 at 7:04 AM, 
> > > wrote:
> > > > > >> >
> > > > > >> > > I'm not sure what the appetite is for pushing this into 1.2
> vs
> > > the
> > > > > >> > planned
> > > > > >> > > 1.3, but Drill UI Authentication would be really great for
> > > > > >> > companies/users
> > > > > >> > > that run multi-tenant Hadoop clusters with Drill. At the
> > moment
> > > > any
> > > > > >> user
> > > > > >> > > can change storage plugins and cancel running queries, even
> if
> > > > they
> > > > > >> did
> > > > > >> > not
> > > > > >> > > submit them. This is a bit of a blocker for Drill on
> > > multi-tenant
> > > > > >> > > production clusters for now.
> > > > > >> > >
> > > > > >> > > JIRA link:
> > > > > >> > > https://issues.apache.org/jira/browse/DRILL-3201
> > > > > >> > >
> > > > > >> > >
> > > > > >> > >
> > > > > >> > > -Original Message-
> > > > > >> > > From: Jacques Nadeau [mailto:jacq...@apache.org]
> > > > > >> > > Sent: 08 July 2015 17:19
> > > > > >> > > To: dev@drill.apache.org
> > > > > >> > > Subject: Timing for 1.2 release, ideas around things to
> > include
> > > > > >> > >
> > > > > >> > > Hey Guys,
> > > > > >> > >
> > > > > >> > > What do you think about planning the 1.2 feature cutoff
> around
> > > Aug
> > > > > >> 21st
> > > > > >> > > and target a release vote a week or so after that?
> > > > > >> > >
> > > > > >> > > What are different features people want to get into 1.2?
> > > > > >> > >
> > > > > >> > > Here are couple of things that I'd like to help facilitate
> > > > > >> contribution
> > > > > >> > > and/or inclusion:
> > > > > >> > >
> > > > > >> > > - Mongo improvements (including extended type fixes 2879 ,
> > test
> > > > > cases
> > > > > >> > 1666,
> > > > > >> > > etc)
> > > > > >> > > - httpd log parser format plugin (3423)
> > > > > >> > > - rpc enhancements (local 2941 and off-thread 3242)
> > > > > >> > > - First merge of Magnus' JDBC connector (3180)
> > > > > >> > > - New allocator (1942)
> > > > 

[GitHub] drill pull request: Fix for DRILL-3869

[aleph-zero]

Github user aleph-zero commented on the pull request:

https://github.com/apache/drill/pull/178#issuecomment-144817688
  
Addressed above comments by trimming whitespace from the query string prior 
to trimming any trailing ';' character.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill pull request: Fix for DRILL-3869

[jaltekruse]

Github user jaltekruse commented on the pull request:

https://github.com/apache/drill/pull/178#issuecomment-144819302
  
The change looks good, but thinking about this a little more it might make 
sense to fix this a little higher up the chain. If we fix this where queries 
are actually accepted by the Drill server, then this same fix would also apply 
to the JDBC and C++ interfaces. I believe the only reason this happens to work 
with sqlline is because it is stripping it off on the client before sending the 
query. As long as we're making a change on the server, I think we should put it 
as close to actual execution as possible.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill pull request: Issues/drill 3791

[aleph-zero]

Github user aleph-zero commented on a diff in the pull request:

https://github.com/apache/drill/pull/177#discussion_r40960796
  
--- Diff: 
contrib/storage-jdbc/src/main/java/org/apache/drill/exec/store/jdbc/JdbcRecordReader.java
 ---
@@ -364,13 +368,15 @@ void copy(int index) throws SQLException {
 
   private class DateCopier extends Copier {
 
+Calendar calendar = Calendar.getInstance(TimeZone.getTimeZone("UTC"));
--- End diff --

Fixed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill pull request: Issues/drill 3791

[aleph-zero]

Github user aleph-zero commented on a diff in the pull request:

https://github.com/apache/drill/pull/177#discussion_r40960820
  
--- Diff: 
contrib/storage-jdbc/src/main/java/org/apache/drill/exec/store/jdbc/JdbcRecordReader.java
 ---
@@ -364,13 +368,15 @@ void copy(int index) throws SQLException {
 
   private class DateCopier extends Copier {
 
+Calendar calendar = Calendar.getInstance(TimeZone.getTimeZone("UTC"));
+
 public DateCopier(int columnIndex, ResultSet result, 
NullableDateVector.Mutator mutator) {
   super(columnIndex, result, mutator);
 }
 
 @Override
 void copy(int index) throws SQLException {
-  Date date = result.getDate(columnIndex);
+  Date date = result.getDate(columnIndex, calendar);
--- End diff --

Fixed


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

RE: Channel closed exception

[Sudip Mukherjee]

Ohh..Sorry about that,
Could you please try again? Or I'll put it in github
https://drive.google.com/drive/folders/0ByRlB6iLvhRvSm9UQWJ4b2ZjVDQ

Thanks,
Sudip

-Original Message-
From: Sudheesh Katkam [mailto:skat...@maprtech.com] 
Sent: 02 October 2015 AM 02:04
To: dev@drill.apache.org
Subject: Re: Channel closed exception

I cannot open the shared documents (“Whoops! There was a problem previewing 
this document.”) nor can I download them ("We're sorry, but you do not have 
access to this page.”). The folder link in this thread is also broken (“The 
open folder can no longer be viewed”).

- Sudheesh

> On Oct 1, 2015, at 12:43 PM, Sudip Mukherjee  wrote:
> 
> I uploaded to google drive. Should it be okay?
> 
> https://drive.google.com/drive/folders/0ByRlB6iLvhRvSm9UQWJ4b2ZjVDQ
> 
> Thanks,
> Sudip
> 
> -Original Message-
> From: Sudheesh Katkam [mailto:skat...@maprtech.com]
> Sent: 01 October 2015 AM 10:01
> To: dev@drill.apache.org
> Subject: Re: Channel closed exception
> 
> Ah there, I assumed you were not on Windows. We do not have the drillbit.out 
> file for Windows (I think).
> 
> Anyway, the error messages you see are symptoms of the real issue, which is 
> most likely an “out of memory” scenario. I think that the drillbit did not 
> crash either (in which case you might see a JVM core dump). Instead the 
> drillbit gracefully shutdown unable to handle the error. Could you share the 
> query (if possible), data set details, and configuration?
> 
> It would also help if you put the entire log files somewhere (Github gist or 
> Dropbox) and shared the links.
> 
> - Sudheesh
> 
>> On Sep 30, 2015, at 9:10 PM, Sudip Mukherjee  
>> wrote:
>> 
>> Yes. I was running drill in embedded mode to try it out with  3.9 
>> lakhs records , near about 50+ columns.[ sqlline.bat -u 
>> "jdbc:drill:zk=local" ] I see sqlline.log and sqlline_queries.log under log 
>> folder. Did I miss some steps?
>> 
>> But was able run queries from the command prompt after that
>> 
>> Thanks,
>> Sudip
>> 
>> -Original Message-
>> From: Sudheesh Katkam [mailto:skat...@maprtech.com]
>> Sent: 01 October 2015 AM 09:24
>> To: dev@drill.apache.org
>> Subject: Re: Channel closed exception
>> 
>> Are you running drill in embedded mode? The directory containing the 
>> drillbit.log file contains the drillbit.out file too. There might be a JVM 
>> core dump in the /tmp directory.
>> 
>> Usually "Received shutdown request” message means that’s the drillbit ran 
>> out of memory. Given that you mentioned the query is on a large document 
>> set, this scenario is likely.
>> 
>> - Sudheesh
>> 
>>> On Sep 30, 2015, at 8:46 PM, Sudip Mukherjee  
>>> wrote:
>>> 
>>> Hi,
>>> This is the excerpt of exception in sqlline.log. Could you please refer me 
>>> to the location of drillbit.out file?
>>> 
>>> 
>>> 2015-09-30 22:23:36,286 [BitServer-4] WARN  
>>> o.a.d.exec.rpc.control.ControlServer - Message of mode REQUEST of rpc type 
>>> 6 took longer than 500ms.  Actual duration was 16047ms.
>>> 2015-09-30 22:24:13,581
>>> [29f36725-9de3-b370-7f38-50ebbe677bf7:frag:0:0] INFO 
>>> o.a.d.e.w.fragment.FragmentExecutor -
>>> 29f36725-9de3-b370-7f38-50ebbe677bf7:0:0: State change requested 
>>> CANCELLATION_REQUESTED --> FINISHED
>>> 2015-09-30 22:24:13,581
>>> [29f36725-9de3-b370-7f38-50ebbe677bf7:frag:0:0] INFO 
>>> o.a.d.e.w.f.FragmentStatusReporter -
>>> 29f36725-9de3-b370-7f38-50ebbe677bf7:0:0: State to report: CANCELLED
>>> 2015-09-30 22:24:13,581 [BitServer-4] INFO 
>>> o.a.drill.exec.work.foreman.Foreman - State change requested.
>>> CANCELLATION_REQUESTED --> COMPLETED
>>> 2015-09-30 22:24:13,581 [BitServer-4] INFO  
>>> o.a.drill.exec.work.foreman.Foreman - foreman cleaning up.
>>> 2015-09-30 22:24:13,597 [UserServer-1] INFO  
>>> o.a.drill.exec.work.foreman.Foreman - Failure while trying communicate 
>>> query result to initiating client. This would happen if a client is 
>>> disconnected before response notice can be sent.
>>> org.apache.drill.exec.rpc.ChannelClosedException: null
>>> at 
>>> org.apache.drill.exec.rpc.CoordinationQueue$RpcListener.operationComplete(CoordinationQueue.java:89)
>>>  [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
>>> at 
>>> org.apache.drill.exec.rpc.CoordinationQueue$RpcListener.operationComplete(CoordinationQueue.java:67)
>>>  [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
>>> at 
>>> io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:680)
>>>  [netty-common-4.0.27.Final.jar:4.0.27.Final]
>>> at 
>>> io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:603)
>>>  [netty-common-4.0.27.Final.jar:4.0.27.Final]
>>> at 
>>> io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:563)
>>>  [netty-common-4.0.27.Final.jar:4.0.27.Final]
>>> at
>>> io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.ja
>>> v
>>> 

RE: Channel closed exception

[Sudip Mukherjee]

I uploaded to google drive. Should it be okay?

https://drive.google.com/drive/folders/0ByRlB6iLvhRvSm9UQWJ4b2ZjVDQ

Thanks,
Sudip

-Original Message-
From: Sudheesh Katkam [mailto:skat...@maprtech.com] 
Sent: 01 October 2015 AM 10:01
To: dev@drill.apache.org
Subject: Re: Channel closed exception

Ah there, I assumed you were not on Windows. We do not have the drillbit.out 
file for Windows (I think).

Anyway, the error messages you see are symptoms of the real issue, which is 
most likely an “out of memory” scenario. I think that the drillbit did not 
crash either (in which case you might see a JVM core dump). Instead the 
drillbit gracefully shutdown unable to handle the error. Could you share the 
query (if possible), data set details, and configuration?

It would also help if you put the entire log files somewhere (Github gist or 
Dropbox) and shared the links.

- Sudheesh

> On Sep 30, 2015, at 9:10 PM, Sudip Mukherjee  wrote:
> 
> Yes. I was running drill in embedded mode to try it out with  3.9 
> lakhs records , near about 50+ columns.[ sqlline.bat -u "jdbc:drill:zk=local" 
> ] I see sqlline.log and sqlline_queries.log under log folder. Did I miss some 
> steps?
> 
> But was able run queries from the command prompt after that
> 
> Thanks,
> Sudip
> 
> -Original Message-
> From: Sudheesh Katkam [mailto:skat...@maprtech.com]
> Sent: 01 October 2015 AM 09:24
> To: dev@drill.apache.org
> Subject: Re: Channel closed exception
> 
> Are you running drill in embedded mode? The directory containing the 
> drillbit.log file contains the drillbit.out file too. There might be a JVM 
> core dump in the /tmp directory.
> 
> Usually "Received shutdown request” message means that’s the drillbit ran out 
> of memory. Given that you mentioned the query is on a large document set, 
> this scenario is likely.
> 
> - Sudheesh
> 
>> On Sep 30, 2015, at 8:46 PM, Sudip Mukherjee  
>> wrote:
>> 
>> Hi,
>> This is the excerpt of exception in sqlline.log. Could you please refer me 
>> to the location of drillbit.out file?
>> 
>> 
>> 2015-09-30 22:23:36,286 [BitServer-4] WARN  
>> o.a.d.exec.rpc.control.ControlServer - Message of mode REQUEST of rpc type 6 
>> took longer than 500ms.  Actual duration was 16047ms.
>> 2015-09-30 22:24:13,581
>> [29f36725-9de3-b370-7f38-50ebbe677bf7:frag:0:0] INFO 
>> o.a.d.e.w.fragment.FragmentExecutor -
>> 29f36725-9de3-b370-7f38-50ebbe677bf7:0:0: State change requested 
>> CANCELLATION_REQUESTED --> FINISHED
>> 2015-09-30 22:24:13,581
>> [29f36725-9de3-b370-7f38-50ebbe677bf7:frag:0:0] INFO 
>> o.a.d.e.w.f.FragmentStatusReporter -
>> 29f36725-9de3-b370-7f38-50ebbe677bf7:0:0: State to report: CANCELLED
>> 2015-09-30 22:24:13,581 [BitServer-4] INFO 
>> o.a.drill.exec.work.foreman.Foreman - State change requested.
>> CANCELLATION_REQUESTED --> COMPLETED
>> 2015-09-30 22:24:13,581 [BitServer-4] INFO  
>> o.a.drill.exec.work.foreman.Foreman - foreman cleaning up.
>> 2015-09-30 22:24:13,597 [UserServer-1] INFO  
>> o.a.drill.exec.work.foreman.Foreman - Failure while trying communicate query 
>> result to initiating client. This would happen if a client is disconnected 
>> before response notice can be sent.
>> org.apache.drill.exec.rpc.ChannelClosedException: null
>>  at 
>> org.apache.drill.exec.rpc.CoordinationQueue$RpcListener.operationComplete(CoordinationQueue.java:89)
>>  [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
>>  at 
>> org.apache.drill.exec.rpc.CoordinationQueue$RpcListener.operationComplete(CoordinationQueue.java:67)
>>  [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
>>  at 
>> io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:680)
>>  [netty-common-4.0.27.Final.jar:4.0.27.Final]
>>  at 
>> io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:603)
>>  [netty-common-4.0.27.Final.jar:4.0.27.Final]
>>  at 
>> io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:563)
>>  [netty-common-4.0.27.Final.jar:4.0.27.Final]
>>  at
>> io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.jav
>> a
>> :424) [netty-common-4.0.27.Final.jar:4.0.27.Final]
>> 
>> Thanks,
>> Sudip
>> 
>> -Original Message-
>> From: Sudheesh Katkam [mailto:skat...@maprtech.com]
>> Sent: 01 October 2015 AM 09:09
>> To: dev@drill.apache.org
>> Subject: Re: Channel closed exception
>> 
>> Can you check if there are any error messages in the drillbit.out file?
>> 
>> - Sudheesh
>> 
>>> On Sep 30, 2015, at 8:35 PM, Sudip Mukherjee  
>>> wrote:
>>> 
>>> Hi Devs,
>>> 
>>> Getting into a channel closed exception when I tried to connect drill to 
>>> tableau and query a large document set. Could it be because tableau dropped 
>>> the connection due to datasize. Any thoughts?
>>> 
>>> 2015-09-30 21:13:43,716
>>> [29f3777a-d407-295a-d45e-243fb70efc07:frag:0:0] INFO 
>>> o.a.d.e.w.fragment.FragmentExecutor -
>>> 

Re: Channel closed exception

[Sudheesh Katkam]

I cannot open the shared documents (“Whoops! There was a problem previewing 
this document.”) nor can I download them ("We're sorry, but you do not have 
access to this page.”). The folder link in this thread is also broken (“The 
open folder can no longer be viewed”).

- Sudheesh

> On Oct 1, 2015, at 12:43 PM, Sudip Mukherjee  wrote:
> 
> I uploaded to google drive. Should it be okay?
> 
> https://drive.google.com/drive/folders/0ByRlB6iLvhRvSm9UQWJ4b2ZjVDQ
> 
> Thanks,
> Sudip
> 
> -Original Message-
> From: Sudheesh Katkam [mailto:skat...@maprtech.com] 
> Sent: 01 October 2015 AM 10:01
> To: dev@drill.apache.org
> Subject: Re: Channel closed exception
> 
> Ah there, I assumed you were not on Windows. We do not have the drillbit.out 
> file for Windows (I think).
> 
> Anyway, the error messages you see are symptoms of the real issue, which is 
> most likely an “out of memory” scenario. I think that the drillbit did not 
> crash either (in which case you might see a JVM core dump). Instead the 
> drillbit gracefully shutdown unable to handle the error. Could you share the 
> query (if possible), data set details, and configuration?
> 
> It would also help if you put the entire log files somewhere (Github gist or 
> Dropbox) and shared the links.
> 
> - Sudheesh
> 
>> On Sep 30, 2015, at 9:10 PM, Sudip Mukherjee  
>> wrote:
>> 
>> Yes. I was running drill in embedded mode to try it out with  3.9 
>> lakhs records , near about 50+ columns.[ sqlline.bat -u 
>> "jdbc:drill:zk=local" ] I see sqlline.log and sqlline_queries.log under log 
>> folder. Did I miss some steps?
>> 
>> But was able run queries from the command prompt after that
>> 
>> Thanks,
>> Sudip
>> 
>> -Original Message-
>> From: Sudheesh Katkam [mailto:skat...@maprtech.com]
>> Sent: 01 October 2015 AM 09:24
>> To: dev@drill.apache.org
>> Subject: Re: Channel closed exception
>> 
>> Are you running drill in embedded mode? The directory containing the 
>> drillbit.log file contains the drillbit.out file too. There might be a JVM 
>> core dump in the /tmp directory.
>> 
>> Usually "Received shutdown request” message means that’s the drillbit ran 
>> out of memory. Given that you mentioned the query is on a large document 
>> set, this scenario is likely.
>> 
>> - Sudheesh
>> 
>>> On Sep 30, 2015, at 8:46 PM, Sudip Mukherjee  
>>> wrote:
>>> 
>>> Hi,
>>> This is the excerpt of exception in sqlline.log. Could you please refer me 
>>> to the location of drillbit.out file?
>>> 
>>> 
>>> 2015-09-30 22:23:36,286 [BitServer-4] WARN  
>>> o.a.d.exec.rpc.control.ControlServer - Message of mode REQUEST of rpc type 
>>> 6 took longer than 500ms.  Actual duration was 16047ms.
>>> 2015-09-30 22:24:13,581
>>> [29f36725-9de3-b370-7f38-50ebbe677bf7:frag:0:0] INFO 
>>> o.a.d.e.w.fragment.FragmentExecutor -
>>> 29f36725-9de3-b370-7f38-50ebbe677bf7:0:0: State change requested 
>>> CANCELLATION_REQUESTED --> FINISHED
>>> 2015-09-30 22:24:13,581
>>> [29f36725-9de3-b370-7f38-50ebbe677bf7:frag:0:0] INFO 
>>> o.a.d.e.w.f.FragmentStatusReporter -
>>> 29f36725-9de3-b370-7f38-50ebbe677bf7:0:0: State to report: CANCELLED
>>> 2015-09-30 22:24:13,581 [BitServer-4] INFO 
>>> o.a.drill.exec.work.foreman.Foreman - State change requested.
>>> CANCELLATION_REQUESTED --> COMPLETED
>>> 2015-09-30 22:24:13,581 [BitServer-4] INFO  
>>> o.a.drill.exec.work.foreman.Foreman - foreman cleaning up.
>>> 2015-09-30 22:24:13,597 [UserServer-1] INFO  
>>> o.a.drill.exec.work.foreman.Foreman - Failure while trying communicate 
>>> query result to initiating client. This would happen if a client is 
>>> disconnected before response notice can be sent.
>>> org.apache.drill.exec.rpc.ChannelClosedException: null
>>> at 
>>> org.apache.drill.exec.rpc.CoordinationQueue$RpcListener.operationComplete(CoordinationQueue.java:89)
>>>  [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
>>> at 
>>> org.apache.drill.exec.rpc.CoordinationQueue$RpcListener.operationComplete(CoordinationQueue.java:67)
>>>  [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
>>> at 
>>> io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:680)
>>>  [netty-common-4.0.27.Final.jar:4.0.27.Final]
>>> at 
>>> io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:603)
>>>  [netty-common-4.0.27.Final.jar:4.0.27.Final]
>>> at 
>>> io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:563)
>>>  [netty-common-4.0.27.Final.jar:4.0.27.Final]
>>> at
>>> io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.jav
>>> a
>>> :424) [netty-common-4.0.27.Final.jar:4.0.27.Final]
>>> 
>>> Thanks,
>>> Sudip
>>> 
>>> -Original Message-
>>> From: Sudheesh Katkam [mailto:skat...@maprtech.com]
>>> Sent: 01 October 2015 AM 09:09
>>> To: dev@drill.apache.org
>>> Subject: Re: Channel closed exception
>>> 
>>> Can you check if there are any error messages 

[GitHub] drill pull request: DRILL-3874: flattening large JSON objects uses...

[jaltekruse]

Github user jaltekruse commented on a diff in the pull request:

https://github.com/apache/drill/pull/181#discussion_r40963410
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/BaseRepeatedValueVector.java
 ---
@@ -123,6 +123,15 @@ public int getBufferSize() {
   }
 
   @Override
+  public int getBufferSizeFor(int valueCount) {
+if (getAccessor().getValueCount() == 0) {
+  return 0;
+}
+
+return offsets.getBufferSizeFor(valueCount) + 
vector.getBufferSizeFor(valueCount);
--- End diff --

This first one should be passed valueCount +1 right? As was done in 
VariableLengthVectors.java



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill pull request: Issues/drill 3791

[aleph-zero]

Github user aleph-zero commented on a diff in the pull request:

https://github.com/apache/drill/pull/177#discussion_r40960677
  
--- Diff: contrib/storage-jdbc/pom.xml ---
@@ -15,62 +15,241 @@
  See the License for the specific language governing permissions and
  limitations under the License.
 -->
-http://maven.apache.org/POM/4.0.0 
http://maven.apache.org/xsd/maven-4.0.0.xsd; 
xmlns="http://maven.apache.org/POM/4.0.0;
-xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;>
-  4.0.0
-  
-drill-contrib-parent
-org.apache.drill.contrib
-1.2.0-SNAPSHOT
-  
+http://maven.apache.org/POM/4.0.0 
http://maven.apache.org/xsd/maven-4.0.0.xsd;
+ xmlns="http://maven.apache.org/POM/4.0.0;
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;>
+4.0.0
+
+drill-contrib-parent
+org.apache.drill.contrib
+1.2.0-SNAPSHOT
+
 
-  drill-jdbc-storage
+drill-jdbc-storage
 
-  contrib/jdbc-storage-plugin
+contrib/jdbc-storage-plugin
 
-  
+
+5.1.36
+2
--- End diff --

Yes. I found a maven plugin to handle this. Will push the change.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill pull request: Issues/drill 3791

[aleph-zero]

Github user aleph-zero commented on a diff in the pull request:

https://github.com/apache/drill/pull/177#discussion_r40960694
  
--- Diff: contrib/storage-jdbc/pom.xml ---
@@ -15,62 +15,241 @@
  See the License for the specific language governing permissions and
  limitations under the License.
 -->
-http://maven.apache.org/POM/4.0.0 
http://maven.apache.org/xsd/maven-4.0.0.xsd; 
xmlns="http://maven.apache.org/POM/4.0.0;
-xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;>
-  4.0.0
-  
-drill-contrib-parent
-org.apache.drill.contrib
-1.2.0-SNAPSHOT
-  
+http://maven.apache.org/POM/4.0.0 
http://maven.apache.org/xsd/maven-4.0.0.xsd;
+ xmlns="http://maven.apache.org/POM/4.0.0;
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;>
+4.0.0
+
+drill-contrib-parent
+org.apache.drill.contrib
+1.2.0-SNAPSHOT
+
 
-  drill-jdbc-storage
+drill-jdbc-storage
 
-  contrib/jdbc-storage-plugin
+contrib/jdbc-storage-plugin
 
-  
+
+5.1.36
+2
+20001
+drill_derby_test
+drill_mysql_test
+
 
+
+
+org.apache.drill.exec
+drill-java-exec
+${project.version}
+
 
-
-  org.apache.drill.exec
-  drill-java-exec
-  ${project.version}
-
-
-
-
-  org.apache.drill.exec
-  drill-java-exec
-  tests
-  ${project.version}
-  test
-
-
-  org.apache.drill
-  drill-common
-  tests
-  ${project.version}
-  test
-
-
-  com.yammer.metrics
-  metrics-core
-  2.1.1
-  test
-
-
-  org.apache.derby
-  derbyclient
-  10.11.1.1
-  test
-
-
-  org.apache.derby
-  derbynet
-  10.11.1.1
-  test
-
-  
+
+
--- End diff --

Fixed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Apache_Drill_Exception - Invitation to view

[Sudip Mukherjee (via Google Drive)]

I've shared an item with you:

Apache_Drill_Exception
https://drive.google.com/folderview?id=0ByRlB6iLvhRvSm9UQWJ4b2ZjVDQ=sharing=CPOnotUP=560d87a6

It's not an attachment -- it's stored online. To open this item, just click  
the link above.


Apache drill exception while getting data to tableau..

logs and data source info shared.

Re: Timing for 1.2 release, ideas around things to include

[Jacques Nadeau]

On Thu, Oct 1, 2015 at 11:17 AM, Abdel Hakim Deneche 
 wrote:

> I can manage the release.
>

Great! I use a manual variation of these steps [1] typically. Julian Hyde
made a much nicer set of steps (for Incubator) here: [2] if you want to use
those.

On Thu, Oct 1, 2015 at 2:32 PM, Parth Chandra  wrote:

> +1 on the release. I'm all for starting the process ASAP.
>

I've moved all the non-critical, non-active jiras out of 1.2. However, that
still leaves 63 issues open. Do you want to do a quick hangout to try to
figure what we really want to include versus bump?

J

[1]
https://gist.github.com/jacques-n/be0d2f584171d832aedb#file-release_commands
[2]
https://calcite.incubator.apache.org/docs/howto.html#making-a-release-for-calcite-committers



>
>
>
> On Thu, Oct 1, 2015 at 8:17 AM, Abdel Hakim Deneche  >
> wrote:
>
> > I can manage the release.
> >
> > On Wed, Sep 30, 2015 at 6:10 PM, Aman Sinha  wrote:
> >
> > > +1 for starting the process of finalizing the 1.2 release and 'bounce
> > > everything not critical (and not in-progress or reviewable) off of this
> > > release'.
> > >
> > > Aman
> > >
> > > On Wed, Sep 30, 2015 at 5:36 PM, Jacques Nadeau 
> > > wrote:
> > >
> > > > Hey All,
> > > >
> > > > It seems like we should finalize on the 1.2 release pretty soon. I
> > > > currently see something like 174 bugs currently open. Of those, ~20
> are
> > > > reviewable and ~20 are in progress. I'm inclined to bounce everything
> > not
> > > > critical (and not in-progress or reviewable) off of this release. Any
> > > > thoughts on when everyone would like to do a release? Also, anyone
> want
> > > to
> > > > be release manager?
> > > >
> > > > Jacques
> > > >
> > > > --
> > > > Jacques Nadeau
> > > > CTO and Co-Founder, Dremio
> > > >
> > > > On Tue, Aug 18, 2015 at 7:58 PM, Parth Chandra 
> > > wrote:
> > > >
> > > > > A couple of reasons -
> > > > >   a) We definitely need more QA time for the features that are
> ready.
> > > > >   b) Some features, in particular, Parquet metadata caching
> (Steven)
> > > and
> > > > > Faster Hive reads (Jason) are close enough to completion that we
> > really
> > > > > should include them. Again the team could use the extra time to QA
> > > these.
> > > > >
> > > > >
> > > > > On Tue, Aug 18, 2015 at 4:32 PM, Chris Westin <
> > chriswesti...@gmail.com
> > > >
> > > > > wrote:
> > > > >
> > > > > > Unfortunately, we've had to modify our plans, and are now
> thinking
> > of
> > > > > > aiming for a vote on September 10th.
> > > > > >
> > > > > > On Thu, Aug 13, 2015 at 1:29 PM, Chris Westin <
> > > chriswesti...@gmail.com
> > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > We were thinking of targetting for a vote on August 26th.
> > > > > > >
> > > > > > > On Thu, Aug 13, 2015 at 1:19 PM, Jacques Nadeau <
> > > jacq...@dremio.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > >> Following up since we haven't discussed the 1.2 release in a
> > > while.
> > > > > How
> > > > > > >> are people feeling about trying to get a candidate out in the
> > next
> > > > > week
> > > > > > or
> > > > > > >> two?
> > > > > > >>
> > > > > > >> thanks,
> > > > > > >> Jacques
> > > > > > >>
> > > > > > >> --
> > > > > > >> Jacques Nadeau
> > > > > > >> CTO and Co-Founder, Dremio
> > > > > > >>
> > > > > > >>
> > > > > > >>
> > > > > > >> > -- Forwarded message --
> > > > > > >> > From: Parth Chandra 
> > > > > > >> > Date: Thu, Jul 9, 2015 at 11:29 AM
> > > > > > >> > Subject: Re: Timing for 1.2 release, ideas around things to
> > > > include
> > > > > > >> > To: dev@drill.apache.org
> > > > > > >> >
> > > > > > >> >
> > > > > > >> > I just moved DRILL-3201 to 1.2. It's one of the candidates
> for
> > > > > > inclusion
> > > > > > >> > into 1.2
> > > > > > >> >
> > > > > > >> > On Thu, Jul 9, 2015 at 7:04 AM,  >
> > > > wrote:
> > > > > > >> >
> > > > > > >> > > I'm not sure what the appetite is for pushing this into
> 1.2
> > vs
> > > > the
> > > > > > >> > planned
> > > > > > >> > > 1.3, but Drill UI Authentication would be really great for
> > > > > > >> > companies/users
> > > > > > >> > > that run multi-tenant Hadoop clusters with Drill. At the
> > > moment
> > > > > any
> > > > > > >> user
> > > > > > >> > > can change storage plugins and cancel running queries,
> even
> > if
> > > > > they
> > > > > > >> did
> > > > > > >> > not
> > > > > > >> > > submit them. This is a bit of a blocker for Drill on
> > > > multi-tenant
> > > > > > >> > > production clusters for now.
> > > > > > >> > >
> > > > > > >> > > JIRA link:
> > > > > > >> > > https://issues.apache.org/jira/browse/DRILL-3201
> > > > > > >> > >
> > > > > > >> > >
> > > > > > >> > >
> > > > > > >> > > -Original Message-
> > > > > > >> > > From: Jacques Nadeau [mailto:jacq...@apache.org]
> > > > > > >> > > Sent: 08 July 2015 

[GitHub] drill pull request: Fix for DRILL-3869

[jaltekruse]

Github user jaltekruse commented on the pull request:

https://github.com/apache/drill/pull/178#issuecomment-144823271
  
I just saw that the conversation fragmented on JIRA, Jacques point makes 
sense to me +1


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill pull request: [DRILL-3791] JDBC Storage Plugin Testing

[aleph-zero]

GitHub user aleph-zero opened a pull request:

https://github.com/apache/drill/pull/182

[DRILL-3791] JDBC Storage Plugin Testing

Improves testing for the JDBC storage plugin.

- Adds tests for MySQL
- Simplifies testing by having maven start/stop Derby and MySQL servers
- Moves test data into .sql files under src/test/resources/

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/aleph-zero/drill issues/DRILL-3791

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/drill/pull/182.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #182


commit 371943cc4c204ea4e83d1c44d1f2e30c326b6731
Author: aleph-zero 
Date:   2015-10-01T20:23:41Z

[DRILL-3791] JDBC Storage Plugin Testing

Improves testing for the JDBC storage plugin.

- Adds tests for MySQL
- Simplifies testing by having maven start/stop Derby and MySQL servers
- Moves test data into .sql files under src/test/resources/




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (DRILL-3882) Time and Timestamp vectors should not return timezone-based data

[Andrew (JIRA)]

Andrew created DRILL-3882:
-

 Summary: Time and Timestamp vectors should not return 
timezone-based data
 Key: DRILL-3882
 URL: https://issues.apache.org/jira/browse/DRILL-3882
 Project: Apache Drill
  Issue Type: Bug
  Components: Storage - Other
Affects Versions: 1.1.0
Reporter: Andrew
Assignee: Andrew
 Fix For: 1.2.0


TimeVector, NullableTimeVector, TimestampVector, and NullableTimestampVector 
should not return values that contain timezone information. Each of these 
classes implements:
{code}
public DateTime getObject()
{code}

I believe the correct method should be
{code}
public LocalTime getObject()
{code}

The rational for this change is that the "time" and "timestamp" types are not 
timezone-aware and therefore are more closely modeled by Joda's LocalTime class.

Additionally, the way it is now makes testing harder b/c 
{code}TestBuilder{code} wants to use {code}DateTime{code} objects to compare 
results from JDBC storage engines, but the storage engine's return no timezone 
information for such types.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] drill pull request: DRILL-3874: flattening large JSON objects uses...

[jaltekruse]

Github user jaltekruse commented on the pull request:

https://github.com/apache/drill/pull/181#issuecomment-144846430
  
Overall, the changes look solid.

I am thinking it may be worth trying to remove the bodies of the existing 
getBufferSize() methods and replace them with calls into these new methods, 
passing in the current valueCount that is expected to be set with a call to 
setValueCount(int) before getBufferSize() is used. It seems weird to have two 
different ways to compute the same values, although I see the reason for it, 
previously we were using the writerIndex() for all of the fixed length 
primitives, which is also set in setValueCount().

I don't consider it a must fix to check in, I just wanted to know if you 
had thought about this and ruled it out.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

RE: Channel closed exception

[Sudip Mukherjee]

Uploaded the entire log.

Thanks,
Sudip

-Original Message-
From: Sudheesh Katkam [mailto:skat...@maprtech.com] 
Sent: 02 October 2015 AM 02:40
To: dev@drill.apache.org
Subject: Re: Channel closed exception

Could you share the entire log file?

- Sudheesh

> On Oct 1, 2015, at 1:57 PM, Sudip Mukherjee  wrote:
> 
> Ohh..Sorry about that,
> Could you please try again? Or I'll put it in github 
> https://drive.google.com/drive/folders/0ByRlB6iLvhRvSm9UQWJ4b2ZjVDQ
> 
> Thanks,
> Sudip
> 
> -Original Message-
> From: Sudheesh Katkam [mailto:skat...@maprtech.com]
> Sent: 02 October 2015 AM 02:04
> To: dev@drill.apache.org
> Subject: Re: Channel closed exception
> 
> I cannot open the shared documents (“Whoops! There was a problem previewing 
> this document.”) nor can I download them ("We're sorry, but you do not have 
> access to this page.”). The folder link in this thread is also broken (“The 
> open folder can no longer be viewed”).
> 
> - Sudheesh
> 
>> On Oct 1, 2015, at 12:43 PM, Sudip Mukherjee  
>> wrote:
>> 
>> I uploaded to google drive. Should it be okay?
>> 
>> https://drive.google.com/drive/folders/0ByRlB6iLvhRvSm9UQWJ4b2ZjVDQ
>> 
>> Thanks,
>> Sudip
>> 
>> -Original Message-
>> From: Sudheesh Katkam [mailto:skat...@maprtech.com]
>> Sent: 01 October 2015 AM 10:01
>> To: dev@drill.apache.org
>> Subject: Re: Channel closed exception
>> 
>> Ah there, I assumed you were not on Windows. We do not have the drillbit.out 
>> file for Windows (I think).
>> 
>> Anyway, the error messages you see are symptoms of the real issue, which is 
>> most likely an “out of memory” scenario. I think that the drillbit did not 
>> crash either (in which case you might see a JVM core dump). Instead the 
>> drillbit gracefully shutdown unable to handle the error. Could you share the 
>> query (if possible), data set details, and configuration?
>> 
>> It would also help if you put the entire log files somewhere (Github gist or 
>> Dropbox) and shared the links.
>> 
>> - Sudheesh
>> 
>>> On Sep 30, 2015, at 9:10 PM, Sudip Mukherjee  
>>> wrote:
>>> 
>>> Yes. I was running drill in embedded mode to try it out with  3.9 
>>> lakhs records , near about 50+ columns.[ sqlline.bat -u 
>>> "jdbc:drill:zk=local" ] I see sqlline.log and sqlline_queries.log under log 
>>> folder. Did I miss some steps?
>>> 
>>> But was able run queries from the command prompt after that
>>> 
>>> Thanks,
>>> Sudip
>>> 
>>> -Original Message-
>>> From: Sudheesh Katkam [mailto:skat...@maprtech.com]
>>> Sent: 01 October 2015 AM 09:24
>>> To: dev@drill.apache.org
>>> Subject: Re: Channel closed exception
>>> 
>>> Are you running drill in embedded mode? The directory containing the 
>>> drillbit.log file contains the drillbit.out file too. There might be a JVM 
>>> core dump in the /tmp directory.
>>> 
>>> Usually "Received shutdown request” message means that’s the drillbit ran 
>>> out of memory. Given that you mentioned the query is on a large document 
>>> set, this scenario is likely.
>>> 
>>> - Sudheesh
>>> 
 On Sep 30, 2015, at 8:46 PM, Sudip Mukherjee  
 wrote:
 
 Hi,
 This is the excerpt of exception in sqlline.log. Could you please refer me 
 to the location of drillbit.out file?
 
 
 2015-09-30 22:23:36,286 [BitServer-4] WARN  
 o.a.d.exec.rpc.control.ControlServer - Message of mode REQUEST of rpc type 
 6 took longer than 500ms.  Actual duration was 16047ms.
 2015-09-30 22:24:13,581
 [29f36725-9de3-b370-7f38-50ebbe677bf7:frag:0:0] INFO 
 o.a.d.e.w.fragment.FragmentExecutor -
 29f36725-9de3-b370-7f38-50ebbe677bf7:0:0: State change requested 
 CANCELLATION_REQUESTED --> FINISHED
 2015-09-30 22:24:13,581
 [29f36725-9de3-b370-7f38-50ebbe677bf7:frag:0:0] INFO 
 o.a.d.e.w.f.FragmentStatusReporter -
 29f36725-9de3-b370-7f38-50ebbe677bf7:0:0: State to report: 
 CANCELLED
 2015-09-30 22:24:13,581 [BitServer-4] INFO 
 o.a.drill.exec.work.foreman.Foreman - State change requested.
 CANCELLATION_REQUESTED --> COMPLETED
 2015-09-30 22:24:13,581 [BitServer-4] INFO  
 o.a.drill.exec.work.foreman.Foreman - foreman cleaning up.
 2015-09-30 22:24:13,597 [UserServer-1] INFO  
 o.a.drill.exec.work.foreman.Foreman - Failure while trying communicate 
 query result to initiating client. This would happen if a client is 
 disconnected before response notice can be sent.
 org.apache.drill.exec.rpc.ChannelClosedException: null
at 
 org.apache.drill.exec.rpc.CoordinationQueue$RpcListener.operationComplete(CoordinationQueue.java:89)
  [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at 
 org.apache.drill.exec.rpc.CoordinationQueue$RpcListener.operationComplete(CoordinationQueue.java:67)
  [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at 
 

[GitHub] drill pull request: [DRILL-3723] Remove unused RemoteServiceSet me...

[aleph-zero]

GitHub user aleph-zero opened a pull request:

https://github.com/apache/drill/pull/184

[DRILL-3723] Remove unused RemoteServiceSet method

RemoteServiceSet.getServiceSetWithFullCache() was not using any of the
arguments passed to it. It was identical to getLocalServiceSet().

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/aleph-zero/drill issues/DRILL-3723

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/drill/pull/184.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #184


commit fc79dfda2c4922b9ab8d5db99743b3eb1055fbb5
Author: aleph-zero 
Date:   2015-10-01T22:44:54Z

[DRILL-3723] Remove unused RemoteServiceSet method

RemoteServiceSet.getServiceSetWithFullCache() was not using any of the
arguments passed to it. It was identical to getLocalServiceSet().




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Partial aggregation in Drill-on-Phoenix

[Julian Hyde]

Phoenix is able to perform quite a few relational operations on the
region server: scan, filter, project, aggregate, sort (optionally with
limit). However, the sort and aggregate are necessarily "local". They
can only deal with data on that region server, and there needs to be a
further operation to combine the results from the region servers.

The question is how to plan such queries. I think the answer is an
AggregateExchangeTransposeRule.

The rule would spot an Aggregate on a data source that is split into
multiple locations (partitions) and split it into a partial Aggregate
that computes sub-totals and a summarizing Aggregate that combines
those totals.

How does the planner know that the Aggregate needs to be split? Since
the data's distribution has changed, there would need to be an
Exchange operator. It is the Exchange operator that triggers the rule
to fire.

There are some special cases. If the data is sorted as well as
partitioned (say because the local aggregate uses a sort-based
algorithm) we could maybe use a more efficient plan. And if the
partition key is the same as the aggregation key we don't need a
summarizing Aggregate, just a Union.

It turns out not to be very Phoenix-specific. In the Drill-on-Phoenix
scenario, once the Aggregate has been pushed through the Exchange
(i.e. onto the drill-bit residing on the region server) we can then
push the DrillAggregate across the drill-to-phoenix membrane and make
it into a PhoenixServerAggregate that executes in the region server.

Related issues:
* https://issues.apache.org/jira/browse/DRILL-3840
* https://issues.apache.org/jira/browse/CALCITE-751

Julian

[GitHub] drill pull request: Fix for DRILL-3869

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/drill/pull/178


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill pull request: Refresh metadata bugs 1

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/drill/pull/179


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Resolved] (DRILL-3817) Refresh metadata does not work when used with sub schema

[Aman Sinha (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-3817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aman Sinha resolved DRILL-3817.
---
Resolution: Fixed

Fixed in f9cefd5e7b2de52812ac6a0813237479d9a6a167.

> Refresh metadata does not work when used with sub schema  
> --
>
> Key: DRILL-3817
> URL: https://issues.apache.org/jira/browse/DRILL-3817
> Project: Apache Drill
>  Issue Type: Bug
>Reporter: Mehant Baid
>Assignee: Mehant Baid
> Fix For: 1.2.0
>
> Attachments: DRILL-3817.patch
>
>
> refresh table metadata dfs.tmp.`lineitem` does not work, hit the following 
> exception
> org.apache.drill.common.exceptions.UserRemoteException: PARSE ERROR: 
> org.apache.calcite.sql.SqlBasicCall cannot be cast to 
> org.apache.calcite.sql.SqlIdentifier
> If the sub schema is removed it works.
> refresh table metadata dfs.`/tmp/lineitem`



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] drill pull request: [DRILL-3300] Bug fix: Unused method parameter

[aleph-zero]

GitHub user aleph-zero opened a pull request:

https://github.com/apache/drill/pull/183

[DRILL-3300] Bug fix: Unused method parameter

Fixed a bug whereby HBaseGroupScan.setFilterPushedDown(boolean) was not 
using the value
passed to it to set an internal flag. Instead the method was always
setting the value to 'true'.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/aleph-zero/drill issues/DRILL-3300

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/drill/pull/183.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #183


commit 1089cf6e8543ad1659c350123a77599594cae108
Author: aleph-zero 
Date:   2015-10-01T22:30:24Z

[DRILL-3300] Bug fix: Unused method parameter

Fixed a bug whereby setFilterPushedDown(boolean) was not using the value
passed to it to set an internal flag. Instead the method was always
setting the value to 'true'.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill pull request: DRILL-3884: Fix lower parallelization issues w...

[vkorukanti]

GitHub user vkorukanti opened a pull request:

https://github.com/apache/drill/pull/185

DRILL-3884: Fix lower parallelization issues with Hive's native scan.

@amansinha100 Could you please review the patch?

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/vkorukanti/drill DRILL-3884

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/drill/pull/185.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #185


commit cdd71f1f2aa224f65147e084d75eacbc4ff7285b
Author: vkorukanti 
Date:   2015-10-01T22:15:12Z

DRILL-3884: Fix lower parallelization issues with Hive's native scan.




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Resolved] (DRILL-3565) Add support for Avro UNION type

[Andrew (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-3565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew resolved DRILL-3565.
---
Resolution: Fixed

> Add support for Avro UNION type
> ---
>
> Key: DRILL-3565
> URL: https://issues.apache.org/jira/browse/DRILL-3565
> Project: Apache Drill
>  Issue Type: Improvement
>  Components: Storage - Other
>Affects Versions: 1.1.0, 1.2.0
>Reporter: Abhishek Girish
> Fix For: Future
>
> Attachments: divolte.avro, drillbit.log.txt
>
>
> Running a simple select * from an avro file fails. 
> {code:sql}
> > select count(*) from `divolte.avro`;
> Error: SYSTEM ERROR: UnsupportedOperationException: Unimplemented type: UNION
> Fragment 0:0
> [Error Id: c7c1ed87-cd85-4146-844d-4addc227128b on abhi1:31010] 
> (state=,code=0)
> {code}
> Plan:
> {code}
> 00-00Screen
> 00-01  Project(*=[$0])
> 00-02Scan(groupscan=[EasyGroupScan 
> [selectionRoot=maprfs:/tmp/divolte.avro, numFiles=1, columns=[`*`], 
> files=[maprfs:///tmp/divolte.avro]]])
>  {code}
> Log & data file attached. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (DRILL-3883) Trailing characters are ignored when casting from string to date

[Victoria Markman (JIRA)]

Victoria Markman created DRILL-3883:
---

 Summary: Trailing characters are ignored when casting from string 
to date
 Key: DRILL-3883
 URL: https://issues.apache.org/jira/browse/DRILL-3883
 Project: Apache Drill
  Issue Type: Bug
  Components: Query Planning & Optimization
Affects Versions: 1.1.0, 1.2.0
Reporter: Victoria Markman
Priority: Minor


Explicit cast:
{code}
0: jdbc:drill:schema=dfs> select cast('2015-01-03 xxx' as date) 
from sys.options limit 1;
+-+
|   EXPR$0|
+-+
| 2015-01-03  |
+-+
1 row selected (0.279 seconds)
{code}

Implicit cast:
{code}
0: jdbc:drill:schema=dfs> select * from t1;
+---++-+
|  a1   |   b1   | c1  |
+---++-+
| 1 | a  | 2015-01-01  |
| 2 | b  | 2015-01-02  |
| 3 | c  | 2015-01-03  |
| 4 | null   | 2015-01-04  |
| 5 | e  | 2015-01-05  |
| 6 | f  | 2015-01-06  |
| 7 | g  | 2015-01-07  |
| null  | h  | 2015-01-08  |
| 9 | i  | null|
| 10| j  | 2015-01-10  |
+---++-+
10 rows selected (0.271 seconds)

0: jdbc:drill:schema=dfs> select * from t1 where c1 between cast('2015-01-01' 
as date) and '2015-01-03 xxx';
+-++-+
| a1  |   b1   | c1  |
+-++-+
| 1   | a  | 2015-01-01  |
| 2   | b  | 2015-01-02  |
| 3   | c  | 2015-01-03  |
+-++-+
3 rows selected (0.222 seconds)
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (DRILL-3819) Remove redundant filter for files start with "."

[Aman Sinha (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-3819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aman Sinha resolved DRILL-3819.
---
Resolution: Fixed

Fixed in edd10df4026ebe3e4b0f4ec314760453cecfe756

> Remove redundant filter for files start with "."
> 
>
> Key: DRILL-3819
> URL: https://issues.apache.org/jira/browse/DRILL-3819
> Project: Apache Drill
>  Issue Type: Bug
>Reporter: Mehant Baid
>Assignee: Mehant Baid
> Fix For: 1.2.0
>
> Attachments: DRILL-3819.patch
>
>
> Due to a minor issue in resolving merge conflict between drop table and 
> refresh metadata, we now have two checks for the same filter (files starting 
> with "."). 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 32248: DRILL-2139: Star is not expanded correctly in "select distinct" query

[Sean Hsuan-Yi Chu]

> On Oct. 1, 2015, 2:11 a.m., Jacques Nadeau wrote:
> > exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/aggregate/AggregateUtils.java,
> >  line 32
> > 
> >
> > Can you add a Javadoc here?

Done


> On Oct. 1, 2015, 2:11 a.m., Jacques Nadeau wrote:
> > exec/java-exec/src/test/java/org/apache/drill/TestDistinctStar.java, line 25
> > 
> >
> > Can you add a test for something that has a known schema select * 
> > distinct?

add two; one for streamagg; the other for hash agg;


- Sean Hsuan-Yi


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32248/#review101186
---


On Oct. 1, 2015, 3:18 p.m., Sean Hsuan-Yi Chu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/32248/
> ---
> 
> (Updated Oct. 1, 2015, 3:18 p.m.)
> 
> 
> Review request for drill, Aman Sinha, Jacques Nadeau, and Jinfeng Ni.
> 
> 
> Bugs: DRILL-2139
> https://issues.apache.org/jira/browse/DRILL-2139
> 
> 
> Repository: drill-git
> 
> 
> Description
> ---
> 
> Expand * at the run time
> 
> 
> Diffs
> -
> 
>   
> exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/aggregate/AggregateUtils.java
>  PRE-CREATION 
>   
> exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/aggregate/HashAggBatch.java
>  a033a8e 
>   
> exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/aggregate/StreamingAggBatch.java
>  2ab1e66 
>   
> exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/xsort/ExternalSortBatch.java
>  49a64cf 
>   exec/java-exec/src/test/java/org/apache/drill/TestDistinctStar.java 
> PRE-CREATION 
>   
> exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/agg/TestHashAggr.java
>  3786bfd 
>   exec/java-exec/src/test/resources/store/text/data/repeatedRows.json 
> PRE-CREATION 
>   
> exec/java-exec/src/test/resources/testframework/testDistinctStar/testSelectDistinct.tsv
>  PRE-CREATION 
>   
> exec/java-exec/src/test/resources/testframework/testDistinctStar/testSelectDistinctExpression.tsv
>  PRE-CREATION 
>   
> exec/java-exec/src/test/resources/testframework/testDistinctStar/testSelectDistinctOverJoin.tsv
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/32248/diff/
> 
> 
> Testing
> ---
> 
> Unit and all QA tests passed.
> 
> 
> Thanks,
> 
> Sean Hsuan-Yi Chu
> 
>

[jira] [Created] (DRILL-3885) Column alias "`f.c`" rejected if number of regions is > 1 in HBase unit tests

[Daniel Barclay (Drill) (JIRA)]

Daniel Barclay (Drill) created DRILL-3885:
-

 Summary: Column alias "`f.c`" rejected if number of regions is > 1 
in HBase unit tests
 Key: DRILL-3885
 URL: https://issues.apache.org/jira/browse/DRILL-3885
 Project: Apache Drill
  Issue Type: Bug
Reporter: Daniel Barclay (Drill)


Drill rejects the column alias {{`f.c`}}, because of its period character, in 
this query:

{noformat}
SELECT
  row_key, convert_from(tableName.f.c, 'UTF8') `f.c`
FROM
  hbase.`TestTable3` tableName
WHERE
  row_key LIKE '08%0' OR row_key LIKE '%70'
{noformat}

in unit test {{TestHBaseFilterPushDown.testFilterPushDownRowKeyLike}} if the 
number of regions used in {{HBaseTestsSuite}} is set to something greater than 
one.

One problem seems to be that the validation check is inconsistent, happening 
only if the data structure containing that alias get serialized and 
deserialized.

The rejection of that alias seems like a problem (at least from the SQL level), 
although it seems that it might be reasonable given some nearby code, 
suggesting that maybe names/expressions/something aren't encoded enough to 
handle name segments with periods. 

The exception stack trace is:
{noformat}
org.apache.drill.exec.rpc.RpcException: 
org.apache.drill.common.exceptions.UserRemoteException: SYSTEM ERROR: 
UnsupportedOperationException: Field references must be singular names.

Fragment 1:1

[Error Id: 34475f52-6f22-43be-9011-c31a84469781 on dev-linux2:31010]
at 
org.apache.drill.exec.rpc.RpcException.mapException(RpcException.java:60)
at 
org.apache.drill.exec.client.DrillClient$ListHoldingResultsListener.getResults(DrillClient.java:386)
at 
org.apache.drill.exec.client.DrillClient.runQuery(DrillClient.java:291)
at 
org.apache.drill.BaseTestQuery.testRunAndReturn(BaseTestQuery.java:292)
at 
org.apache.drill.BaseTestQuery.testSqlWithResults(BaseTestQuery.java:279)
at 
org.apache.drill.hbase.BaseHBaseTest.runHBaseSQLlWithResults(BaseHBaseTest.java:86)
at 
org.apache.drill.hbase.BaseHBaseTest.runHBaseSQLVerifyCount(BaseHBaseTest.java:90)
at 
org.apache.drill.hbase.TestHBaseFilterPushDown.testFilterPushDownRowKeyLike(TestHBaseFilterPushDown.java:466)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.lang.reflect.Method.invoke(Method.java:606)
at java.lang.reflect.Method.invoke(Method.java:606)
Caused by: org.apache.drill.common.exceptions.UserRemoteException: SYSTEM 
ERROR: UnsupportedOperationException: Field references must be singular names.

Fragment 1:1

[Error Id: 34475f52-6f22-43be-9011-c31a84469781 on dev-linux2:31010]
at 
org.apache.drill.exec.rpc.user.QueryResultHandler.resultArrived(QueryResultHandler.java:118)
at 
org.apache.drill.exec.rpc.user.UserClient.handleReponse(UserClient.java:110)
at 
org.apache.drill.exec.rpc.BasicClientWithConnection.handle(BasicClientWithConnection.java:47)
at 
org.apache.drill.exec.rpc.BasicClientWithConnection.handle(BasicClientWithConnection.java:1)
at org.apache.drill.exec.rpc.RpcBus.handle(RpcBus.java:61)
at 
org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:233)
at org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:1)
at 
io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:89)
at 
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
at 
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
at 
io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
at 
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
at 
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
at 
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:242)
at 
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
at 
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
at 
io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86)
at 
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
at 
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
at 
io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:847)
at 
io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:131)
at 

[jira] [Created] (DRILL-3884) Hive native scan has lower parallelization leading to performance degradation

[Venki Korukanti (JIRA)]

Venki Korukanti created DRILL-3884:
--

 Summary: Hive native scan has lower parallelization leading to 
performance degradation
 Key: DRILL-3884
 URL: https://issues.apache.org/jira/browse/DRILL-3884
 Project: Apache Drill
  Issue Type: Bug
  Components: Query Planning & Optimization, Storage - Hive
Affects Versions: 1.2.0
Reporter: Venki Korukanti
Assignee: Venki Korukanti
Priority: Critical
 Fix For: 1.2.0


Currently {{HiveDrillNativeParquetScan.getScanStats()}} divides the rowCount 
got from {{HiveScan}} by a factor and returns that as cost. Problem is all cost 
calculations and parallelization depends on the rowCount. Value {{cpuCost}} is 
not taken into consideration in current cost calculations in {{ScanPrel}}. In 
order for the planner to choose {{HiveDrillNativeParquetScan}} over 
{{HiveScan}}, rowCount has to be lowered for the former, but this leads to 
lower parallelization and performance degradation.

Temporary fix for Drill 1.2 before DRILL-3856 fully resolves considering CPU 
cost in cost model:
1. Change ScanPrel to consider the CPU cost in given Stats from GroupScan
2. Have higher CPU cost for {{HiveScan}} (SerDe route)
3. Lower CPU cost for {{HiveDrillNativeParquetScan}}





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] drill pull request: DRILL-3884: Fix lower parallelization issues w...

[amansinha100]

Github user amansinha100 commented on a diff in the pull request:

https://github.com/apache/drill/pull/185#discussion_r40981727
  
--- Diff: 
contrib/storage-hive/core/src/main/java/org/apache/drill/exec/store/hive/HiveScan.java
 ---
@@ -319,16 +319,12 @@ public ScanStats getScanStats() {
 estRowCount = data/1024;
   }
 
-  // Hive's native reader is neither memory efficient nor fast. If the 
rowcount is below
-  // HIVE_SERDE_SCAN_OVERHEAD_FACTOR, make sure it is at least 
HIVE_SERDE_SCAN_OVERHEAD_FACTOR to enable the planner
-  // to choose HiveDrillNativeParquetScan. Due to the project on top 
of HiveDrillNativeParquetScan, we end up
-  // choosing the HiveScan instead of HiveDrillNativeParquetScan if 
the cost is too low.
-  if (estRowCount <= HIVE_SERDE_SCAN_OVERHEAD_FACTOR) {
-estRowCount = HIVE_SERDE_SCAN_OVERHEAD_FACTOR;
-  }
+  // Hive's native reader is neither memory efficient nor fast. 
Increase the CPU cost
+  // by a factor to let the planner choose HiveDrillNativeScan over 
HiveScan with SerDes.
+  float cpuCost = 1 * HIVE_SERDE_SCAN_OVERHEAD_FACTOR;
--- End diff --

I am thinking this cost should include the number of columns as 
well...suppose there are lots of columns(e.g 500), we want both types of plans 
to be equally penalized.  


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Timing for 1.2 release, ideas around things to include

[Parth Chandra]

What time do you want the hangout? I'm in favour of bumping if we cannot
get decided by tomorrow.



On Thu, Oct 1, 2015 at 2:05 PM, Jacques Nadeau  wrote:

> On Thu, Oct 1, 2015 at 11:17 AM, Abdel Hakim Deneche <
> adene...@maprtech.com>
>  wrote:
>
> > I can manage the release.
> >
>
> Great! I use a manual variation of these steps [1] typically. Julian Hyde
> made a much nicer set of steps (for Incubator) here: [2] if you want to use
> those.
>
> On Thu, Oct 1, 2015 at 2:32 PM, Parth Chandra  wrote:
>
> > +1 on the release. I'm all for starting the process ASAP.
> >
>
> I've moved all the non-critical, non-active jiras out of 1.2. However, that
> still leaves 63 issues open. Do you want to do a quick hangout to try to
> figure what we really want to include versus bump?
>
> J
>
> [1]
>
> https://gist.github.com/jacques-n/be0d2f584171d832aedb#file-release_commands
> [2]
>
> https://calcite.incubator.apache.org/docs/howto.html#making-a-release-for-calcite-committers
>
>
>
> >
> >
> >
> > On Thu, Oct 1, 2015 at 8:17 AM, Abdel Hakim Deneche <
> adene...@maprtech.com
> > >
> > wrote:
> >
> > > I can manage the release.
> > >
> > > On Wed, Sep 30, 2015 at 6:10 PM, Aman Sinha 
> wrote:
> > >
> > > > +1 for starting the process of finalizing the 1.2 release and 'bounce
> > > > everything not critical (and not in-progress or reviewable) off of
> this
> > > > release'.
> > > >
> > > > Aman
> > > >
> > > > On Wed, Sep 30, 2015 at 5:36 PM, Jacques Nadeau 
> > > > wrote:
> > > >
> > > > > Hey All,
> > > > >
> > > > > It seems like we should finalize on the 1.2 release pretty soon. I
> > > > > currently see something like 174 bugs currently open. Of those, ~20
> > are
> > > > > reviewable and ~20 are in progress. I'm inclined to bounce
> everything
> > > not
> > > > > critical (and not in-progress or reviewable) off of this release.
> Any
> > > > > thoughts on when everyone would like to do a release? Also, anyone
> > want
> > > > to
> > > > > be release manager?
> > > > >
> > > > > Jacques
> > > > >
> > > > > --
> > > > > Jacques Nadeau
> > > > > CTO and Co-Founder, Dremio
> > > > >
> > > > > On Tue, Aug 18, 2015 at 7:58 PM, Parth Chandra 
> > > > wrote:
> > > > >
> > > > > > A couple of reasons -
> > > > > >   a) We definitely need more QA time for the features that are
> > ready.
> > > > > >   b) Some features, in particular, Parquet metadata caching
> > (Steven)
> > > > and
> > > > > > Faster Hive reads (Jason) are close enough to completion that we
> > > really
> > > > > > should include them. Again the team could use the extra time to
> QA
> > > > these.
> > > > > >
> > > > > >
> > > > > > On Tue, Aug 18, 2015 at 4:32 PM, Chris Westin <
> > > chriswesti...@gmail.com
> > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Unfortunately, we've had to modify our plans, and are now
> > thinking
> > > of
> > > > > > > aiming for a vote on September 10th.
> > > > > > >
> > > > > > > On Thu, Aug 13, 2015 at 1:29 PM, Chris Westin <
> > > > chriswesti...@gmail.com
> > > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > > > We were thinking of targetting for a vote on August 26th.
> > > > > > > >
> > > > > > > > On Thu, Aug 13, 2015 at 1:19 PM, Jacques Nadeau <
> > > > jacq...@dremio.com>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > >> Following up since we haven't discussed the 1.2 release in a
> > > > while.
> > > > > > How
> > > > > > > >> are people feeling about trying to get a candidate out in
> the
> > > next
> > > > > > week
> > > > > > > or
> > > > > > > >> two?
> > > > > > > >>
> > > > > > > >> thanks,
> > > > > > > >> Jacques
> > > > > > > >>
> > > > > > > >> --
> > > > > > > >> Jacques Nadeau
> > > > > > > >> CTO and Co-Founder, Dremio
> > > > > > > >>
> > > > > > > >>
> > > > > > > >>
> > > > > > > >> > -- Forwarded message --
> > > > > > > >> > From: Parth Chandra 
> > > > > > > >> > Date: Thu, Jul 9, 2015 at 11:29 AM
> > > > > > > >> > Subject: Re: Timing for 1.2 release, ideas around things
> to
> > > > > include
> > > > > > > >> > To: dev@drill.apache.org
> > > > > > > >> >
> > > > > > > >> >
> > > > > > > >> > I just moved DRILL-3201 to 1.2. It's one of the candidates
> > for
> > > > > > > inclusion
> > > > > > > >> > into 1.2
> > > > > > > >> >
> > > > > > > >> > On Thu, Jul 9, 2015 at 7:04 AM, <
> michael.engl...@nomura.com
> > >
> > > > > wrote:
> > > > > > > >> >
> > > > > > > >> > > I'm not sure what the appetite is for pushing this into
> > 1.2
> > > vs
> > > > > the
> > > > > > > >> > planned
> > > > > > > >> > > 1.3, but Drill UI Authentication would be really great
> for
> > > > > > > >> > companies/users
> > > > > > > >> > > that run multi-tenant Hadoop clusters with Drill. At the
> > > > moment
> > > > > > any
> > > > > > > >> user
> > > > > > > >> > > can change storage plugins and cancel running queries,
> > even
> > > if
> > > > > > they
> > > 

Re: Timing for 1.2 release, ideas around things to include

[Jacques Nadeau]

How about now?



--
Jacques Nadeau
CTO and Co-Founder, Dremio

On Thu, Oct 1, 2015 at 5:38 PM, Parth Chandra  wrote:

> What time do you want the hangout? I'm in favour of bumping if we cannot
> get decided by tomorrow.
>
>
>
> On Thu, Oct 1, 2015 at 2:05 PM, Jacques Nadeau  wrote:
>
> > On Thu, Oct 1, 2015 at 11:17 AM, Abdel Hakim Deneche <
> > adene...@maprtech.com>
> >  wrote:
> >
> > > I can manage the release.
> > >
> >
> > Great! I use a manual variation of these steps [1] typically. Julian Hyde
> > made a much nicer set of steps (for Incubator) here: [2] if you want to
> use
> > those.
> >
> > On Thu, Oct 1, 2015 at 2:32 PM, Parth Chandra  wrote:
> >
> > > +1 on the release. I'm all for starting the process ASAP.
> > >
> >
> > I've moved all the non-critical, non-active jiras out of 1.2. However,
> that
> > still leaves 63 issues open. Do you want to do a quick hangout to try to
> > figure what we really want to include versus bump?
> >
> > J
> >
> > [1]
> >
> >
> https://gist.github.com/jacques-n/be0d2f584171d832aedb#file-release_commands
> > [2]
> >
> >
> https://calcite.incubator.apache.org/docs/howto.html#making-a-release-for-calcite-committers
> >
> >
> >
> > >
> > >
> > >
> > > On Thu, Oct 1, 2015 at 8:17 AM, Abdel Hakim Deneche <
> > adene...@maprtech.com
> > > >
> > > wrote:
> > >
> > > > I can manage the release.
> > > >
> > > > On Wed, Sep 30, 2015 at 6:10 PM, Aman Sinha 
> > wrote:
> > > >
> > > > > +1 for starting the process of finalizing the 1.2 release and
> 'bounce
> > > > > everything not critical (and not in-progress or reviewable) off of
> > this
> > > > > release'.
> > > > >
> > > > > Aman
> > > > >
> > > > > On Wed, Sep 30, 2015 at 5:36 PM, Jacques Nadeau <
> jacq...@dremio.com>
> > > > > wrote:
> > > > >
> > > > > > Hey All,
> > > > > >
> > > > > > It seems like we should finalize on the 1.2 release pretty soon.
> I
> > > > > > currently see something like 174 bugs currently open. Of those,
> ~20
> > > are
> > > > > > reviewable and ~20 are in progress. I'm inclined to bounce
> > everything
> > > > not
> > > > > > critical (and not in-progress or reviewable) off of this release.
> > Any
> > > > > > thoughts on when everyone would like to do a release? Also,
> anyone
> > > want
> > > > > to
> > > > > > be release manager?
> > > > > >
> > > > > > Jacques
> > > > > >
> > > > > > --
> > > > > > Jacques Nadeau
> > > > > > CTO and Co-Founder, Dremio
> > > > > >
> > > > > > On Tue, Aug 18, 2015 at 7:58 PM, Parth Chandra <
> par...@apache.org>
> > > > > wrote:
> > > > > >
> > > > > > > A couple of reasons -
> > > > > > >   a) We definitely need more QA time for the features that are
> > > ready.
> > > > > > >   b) Some features, in particular, Parquet metadata caching
> > > (Steven)
> > > > > and
> > > > > > > Faster Hive reads (Jason) are close enough to completion that
> we
> > > > really
> > > > > > > should include them. Again the team could use the extra time to
> > QA
> > > > > these.
> > > > > > >
> > > > > > >
> > > > > > > On Tue, Aug 18, 2015 at 4:32 PM, Chris Westin <
> > > > chriswesti...@gmail.com
> > > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Unfortunately, we've had to modify our plans, and are now
> > > thinking
> > > > of
> > > > > > > > aiming for a vote on September 10th.
> > > > > > > >
> > > > > > > > On Thu, Aug 13, 2015 at 1:29 PM, Chris Westin <
> > > > > chriswesti...@gmail.com
> > > > > > >
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > We were thinking of targetting for a vote on August 26th.
> > > > > > > > >
> > > > > > > > > On Thu, Aug 13, 2015 at 1:19 PM, Jacques Nadeau <
> > > > > jacq...@dremio.com>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > >> Following up since we haven't discussed the 1.2 release
> in a
> > > > > while.
> > > > > > > How
> > > > > > > > >> are people feeling about trying to get a candidate out in
> > the
> > > > next
> > > > > > > week
> > > > > > > > or
> > > > > > > > >> two?
> > > > > > > > >>
> > > > > > > > >> thanks,
> > > > > > > > >> Jacques
> > > > > > > > >>
> > > > > > > > >> --
> > > > > > > > >> Jacques Nadeau
> > > > > > > > >> CTO and Co-Founder, Dremio
> > > > > > > > >>
> > > > > > > > >>
> > > > > > > > >>
> > > > > > > > >> > -- Forwarded message --
> > > > > > > > >> > From: Parth Chandra 
> > > > > > > > >> > Date: Thu, Jul 9, 2015 at 11:29 AM
> > > > > > > > >> > Subject: Re: Timing for 1.2 release, ideas around things
> > to
> > > > > > include
> > > > > > > > >> > To: dev@drill.apache.org
> > > > > > > > >> >
> > > > > > > > >> >
> > > > > > > > >> > I just moved DRILL-3201 to 1.2. It's one of the
> candidates
> > > for
> > > > > > > > inclusion
> > > > > > > > >> > into 1.2
> > > > > > > > >> >
> > > > > > > > >> > On Thu, Jul 9, 2015 at 7:04 AM, <
> > michael.engl...@nomura.com
> > > >
> > > > > > wrote:
> > > > > > > > >> >
> > > > > > > > >> > > 

Re: Timing for 1.2 release, ideas around things to include

[Jacques Nadeau]

I'm on:

https://plus.google.com/hangouts/_/event/ci4rdiju8bv04a64efj5fedd0lc

--
Jacques Nadeau
CTO and Co-Founder, Dremio

On Thu, Oct 1, 2015 at 5:45 PM, Jacques Nadeau  wrote:

> How about now?
>
>
>
> --
> Jacques Nadeau
> CTO and Co-Founder, Dremio
>
> On Thu, Oct 1, 2015 at 5:38 PM, Parth Chandra  wrote:
>
>> What time do you want the hangout? I'm in favour of bumping if we cannot
>> get decided by tomorrow.
>>
>>
>>
>> On Thu, Oct 1, 2015 at 2:05 PM, Jacques Nadeau 
>> wrote:
>>
>> > On Thu, Oct 1, 2015 at 11:17 AM, Abdel Hakim Deneche <
>> > adene...@maprtech.com>
>> >  wrote:
>> >
>> > > I can manage the release.
>> > >
>> >
>> > Great! I use a manual variation of these steps [1] typically. Julian
>> Hyde
>> > made a much nicer set of steps (for Incubator) here: [2] if you want to
>> use
>> > those.
>> >
>> > On Thu, Oct 1, 2015 at 2:32 PM, Parth Chandra 
>> wrote:
>> >
>> > > +1 on the release. I'm all for starting the process ASAP.
>> > >
>> >
>> > I've moved all the non-critical, non-active jiras out of 1.2. However,
>> that
>> > still leaves 63 issues open. Do you want to do a quick hangout to try to
>> > figure what we really want to include versus bump?
>> >
>> > J
>> >
>> > [1]
>> >
>> >
>> https://gist.github.com/jacques-n/be0d2f584171d832aedb#file-release_commands
>> > [2]
>> >
>> >
>> https://calcite.incubator.apache.org/docs/howto.html#making-a-release-for-calcite-committers
>> >
>> >
>> >
>> > >
>> > >
>> > >
>> > > On Thu, Oct 1, 2015 at 8:17 AM, Abdel Hakim Deneche <
>> > adene...@maprtech.com
>> > > >
>> > > wrote:
>> > >
>> > > > I can manage the release.
>> > > >
>> > > > On Wed, Sep 30, 2015 at 6:10 PM, Aman Sinha 
>> > wrote:
>> > > >
>> > > > > +1 for starting the process of finalizing the 1.2 release and
>> 'bounce
>> > > > > everything not critical (and not in-progress or reviewable) off of
>> > this
>> > > > > release'.
>> > > > >
>> > > > > Aman
>> > > > >
>> > > > > On Wed, Sep 30, 2015 at 5:36 PM, Jacques Nadeau <
>> jacq...@dremio.com>
>> > > > > wrote:
>> > > > >
>> > > > > > Hey All,
>> > > > > >
>> > > > > > It seems like we should finalize on the 1.2 release pretty
>> soon. I
>> > > > > > currently see something like 174 bugs currently open. Of those,
>> ~20
>> > > are
>> > > > > > reviewable and ~20 are in progress. I'm inclined to bounce
>> > everything
>> > > > not
>> > > > > > critical (and not in-progress or reviewable) off of this
>> release.
>> > Any
>> > > > > > thoughts on when everyone would like to do a release? Also,
>> anyone
>> > > want
>> > > > > to
>> > > > > > be release manager?
>> > > > > >
>> > > > > > Jacques
>> > > > > >
>> > > > > > --
>> > > > > > Jacques Nadeau
>> > > > > > CTO and Co-Founder, Dremio
>> > > > > >
>> > > > > > On Tue, Aug 18, 2015 at 7:58 PM, Parth Chandra <
>> par...@apache.org>
>> > > > > wrote:
>> > > > > >
>> > > > > > > A couple of reasons -
>> > > > > > >   a) We definitely need more QA time for the features that are
>> > > ready.
>> > > > > > >   b) Some features, in particular, Parquet metadata caching
>> > > (Steven)
>> > > > > and
>> > > > > > > Faster Hive reads (Jason) are close enough to completion that
>> we
>> > > > really
>> > > > > > > should include them. Again the team could use the extra time
>> to
>> > QA
>> > > > > these.
>> > > > > > >
>> > > > > > >
>> > > > > > > On Tue, Aug 18, 2015 at 4:32 PM, Chris Westin <
>> > > > chriswesti...@gmail.com
>> > > > > >
>> > > > > > > wrote:
>> > > > > > >
>> > > > > > > > Unfortunately, we've had to modify our plans, and are now
>> > > thinking
>> > > > of
>> > > > > > > > aiming for a vote on September 10th.
>> > > > > > > >
>> > > > > > > > On Thu, Aug 13, 2015 at 1:29 PM, Chris Westin <
>> > > > > chriswesti...@gmail.com
>> > > > > > >
>> > > > > > > > wrote:
>> > > > > > > >
>> > > > > > > > > We were thinking of targetting for a vote on August 26th.
>> > > > > > > > >
>> > > > > > > > > On Thu, Aug 13, 2015 at 1:19 PM, Jacques Nadeau <
>> > > > > jacq...@dremio.com>
>> > > > > > > > > wrote:
>> > > > > > > > >
>> > > > > > > > >> Following up since we haven't discussed the 1.2 release
>> in a
>> > > > > while.
>> > > > > > > How
>> > > > > > > > >> are people feeling about trying to get a candidate out in
>> > the
>> > > > next
>> > > > > > > week
>> > > > > > > > or
>> > > > > > > > >> two?
>> > > > > > > > >>
>> > > > > > > > >> thanks,
>> > > > > > > > >> Jacques
>> > > > > > > > >>
>> > > > > > > > >> --
>> > > > > > > > >> Jacques Nadeau
>> > > > > > > > >> CTO and Co-Founder, Dremio
>> > > > > > > > >>
>> > > > > > > > >>
>> > > > > > > > >>
>> > > > > > > > >> > -- Forwarded message --
>> > > > > > > > >> > From: Parth Chandra 
>> > > > > > > > >> > Date: Thu, Jul 9, 2015 at 11:29 AM
>> > > > > > > > >> > Subject: Re: Timing for 1.2 release, ideas around
>> things
>> > to
>> > > > > > include
>> > > > > > > > >> > 

[jira] [Created] (DRILL-3886) Range query over HBase table from Drill returns no results

[Khurram Faraaz (JIRA)]

Khurram Faraaz created DRILL-3886:
-

 Summary: Range query over HBase table from Drill returns no results
 Key: DRILL-3886
 URL: https://issues.apache.org/jira/browse/DRILL-3886
 Project: Apache Drill
  Issue Type: Bug
  Components: Execution - Flow
Affects Versions: 1.2.0
 Environment: 4 node cluster CentOS
Reporter: Khurram Faraaz
Assignee: Smidth Panchamia


Range query over HBase table from Drill returns no results. Note that the 
filter is pushed down to the Scan, however query does not return any results. 
Drill master commit ID: b9afcf8f

case 1) without cast and without quotes around integer value

0: jdbc:drill:schema=dfs.tmp> explain plan for select 
convert_from(row_key,'INT_OBD') as rk from int_Tbl WHERE 
convert_from(row_key,'INT_OBD') > -536870912 AND 
convert_from(row_key,'INT_OBD') < 536870911;
+--+--+
| text | json |
+--+--+
| 00-00Screen
00-01  Project(rk=[CONVERT_FROMINT_OBD($0)])
00-02Scan(groupscan=[HBaseGroupScan [HBaseScanSpec=HBaseScanSpec 
[tableName=int_Tbl, startRow=\xD4`\x00\x00\x00\x00, 
stopRow=\xD4\x9F\xFF\xFF\xFF, filter=FilterList AND (2/2): [RowFilter (LESS, 
\xD4\x9F\xFF\xFF\xFF), RowFilter (GREATER, \xD4`\x00\x00\x00)]], 
columns=[`row_key`]]])

0: jdbc:drill:schema=dfs.tmp> select convert_from(row_key,'INT_OBD') as rk from 
int_Tbl WHERE convert_from(row_key,'INT_OBD') > -536870912 AND 
convert_from(row_key,'INT_OBD') < 536870911;
+--+
|  |
+--+
+--+
No rows selected (0.869 seconds)

---

case 2) with cast and without quotes around integer value

{code}
0: jdbc:drill:schema=dfs.tmp> select convert_from(row_key,'INT_OBD') as rk from 
int_Tbl WHERE convert_from(row_key,'INT_OBD') > cast(-536870912 as int) AND 
convert_from(row_key,'INT_OBD') < cast(536870911 as int);
+--+
|  |
+--+
+--+
No rows selected (0.905 seconds)

0: jdbc:drill:schema=dfs.tmp> explain plan for select 
convert_from(row_key,'INT_OBD') as rk from int_Tbl WHERE 
convert_from(row_key,'INT_OBD') > cast(-536870912 as int) AND 
convert_from(row_key,'INT_OBD') < cast(536870911 as int);
+--+--+
| text | json |
+--+--+
| 00-00Screen
00-01  Project(rk=[CONVERT_FROMINT_OBD($0)])
00-02Scan(groupscan=[HBaseGroupScan [HBaseScanSpec=HBaseScanSpec 
[tableName=int_Tbl, startRow=\xD4`\x00\x00\x00\x00, 
stopRow=\xD4\x9F\xFF\xFF\xFF, filter=FilterList AND (2/2): [RowFilter (LESS, 
\xD4\x9F\xFF\xFF\xFF), RowFilter (GREATER, \xD4`\x00\x00\x00)]], 
columns=[`row_key`]]])
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] drill pull request: DRILL-3874: flattening large JSON objects uses...

[cwestin]

Github user cwestin commented on a diff in the pull request:

https://github.com/apache/drill/pull/181#discussion_r40985372
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/vector/complex/BaseRepeatedValueVector.java
 ---
@@ -123,6 +123,15 @@ public int getBufferSize() {
   }
 
   @Override
+  public int getBufferSizeFor(int valueCount) {
+if (getAccessor().getValueCount() == 0) {
+  return 0;
+}
+
+return offsets.getBufferSizeFor(valueCount) + 
vector.getBufferSizeFor(valueCount);
--- End diff --

Based on setInitialValueCapacity(), I think you're right. Will do. (But I'm 
not going to go back and patch the EBF for that, because it'll just be off by 
one offsets value, or 4 bytes, which isn't going to kill the emergency fix.)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (DRILL-3887) Parquet metadata cache not being used

[Steven Phillips (JIRA)]

Steven Phillips created DRILL-3887:
--

 Summary: Parquet metadata cache not being used
 Key: DRILL-3887
 URL: https://issues.apache.org/jira/browse/DRILL-3887
 Project: Apache Drill
  Issue Type: Bug
Reporter: Steven Phillips
Priority: Critical


The fix for DRILL-3788 causes a directory to be expanded to its list of files 
early in the query, and this change causes the ParquetGroupScan to no longer 
use the parquet metadata file, even when it is there.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] drill pull request: DRILL-3874: flattening large JSON objects uses...

[cwestin]

Github user cwestin commented on the pull request:

https://github.com/apache/drill/pull/181#issuecomment-144898125
  
Parth:
Re ObjectVector: I don't know what that's for. I just followed the pattern: 
getBufferSize() already throws that exception.
Re OUTPUT_MEMORY_LIMIT: what do you think? I tend to avoid adding more 
knobs, but I can easily do that if you like (with the current 512MB as the 
default). Let me know soon, about to kick off testing on Jason's suggested 
replacement of getBufferSize() implementations with calls to 
getBufferSizeFor(). The problem I see here is that it will affect all 
flatten()s, whether they need it or not. And, this isn't really the long term 
solution, which is really to add projection capabilities so that we're not 
passing through the original record like this.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Timing for 1.2 release, ideas around things to include

[Jacques Nadeau]

We went through the list. I propose that we include only the following
issues in 1.2 and move the rest to 1.3

DRILL-1065 Provide a reset command to reset an option to its default value
DRILL-3884 Hive native scan has lower parallelization leading to
performance degradation
DRILL-2879 Add Remaining portion of "Drill extended json's support $oid"
DRILL-3887 Parquet metadata cache not being used (regression caused by
DRILL-3788)

What do other people think?


--
Jacques Nadeau
CTO and Co-Founder, Dremio

On Thu, Oct 1, 2015 at 5:46 PM, Jacques Nadeau  wrote:

> I'm on:
>
> https://plus.google.com/hangouts/_/event/ci4rdiju8bv04a64efj5fedd0lc
>
> --
> Jacques Nadeau
> CTO and Co-Founder, Dremio
>
> On Thu, Oct 1, 2015 at 5:45 PM, Jacques Nadeau  wrote:
>
>> How about now?
>>
>>
>>
>> --
>> Jacques Nadeau
>> CTO and Co-Founder, Dremio
>>
>> On Thu, Oct 1, 2015 at 5:38 PM, Parth Chandra  wrote:
>>
>>> What time do you want the hangout? I'm in favour of bumping if we cannot
>>> get decided by tomorrow.
>>>
>>>
>>>
>>> On Thu, Oct 1, 2015 at 2:05 PM, Jacques Nadeau 
>>> wrote:
>>>
>>> > On Thu, Oct 1, 2015 at 11:17 AM, Abdel Hakim Deneche <
>>> > adene...@maprtech.com>
>>> >  wrote:
>>> >
>>> > > I can manage the release.
>>> > >
>>> >
>>> > Great! I use a manual variation of these steps [1] typically. Julian
>>> Hyde
>>> > made a much nicer set of steps (for Incubator) here: [2] if you want
>>> to use
>>> > those.
>>> >
>>> > On Thu, Oct 1, 2015 at 2:32 PM, Parth Chandra 
>>> wrote:
>>> >
>>> > > +1 on the release. I'm all for starting the process ASAP.
>>> > >
>>> >
>>> > I've moved all the non-critical, non-active jiras out of 1.2. However,
>>> that
>>> > still leaves 63 issues open. Do you want to do a quick hangout to try
>>> to
>>> > figure what we really want to include versus bump?
>>> >
>>> > J
>>> >
>>> > [1]
>>> >
>>> >
>>> https://gist.github.com/jacques-n/be0d2f584171d832aedb#file-release_commands
>>> > [2]
>>> >
>>> >
>>> https://calcite.incubator.apache.org/docs/howto.html#making-a-release-for-calcite-committers
>>> >
>>> >
>>> >
>>> > >
>>> > >
>>> > >
>>> > > On Thu, Oct 1, 2015 at 8:17 AM, Abdel Hakim Deneche <
>>> > adene...@maprtech.com
>>> > > >
>>> > > wrote:
>>> > >
>>> > > > I can manage the release.
>>> > > >
>>> > > > On Wed, Sep 30, 2015 at 6:10 PM, Aman Sinha 
>>> > wrote:
>>> > > >
>>> > > > > +1 for starting the process of finalizing the 1.2 release and
>>> 'bounce
>>> > > > > everything not critical (and not in-progress or reviewable) off
>>> of
>>> > this
>>> > > > > release'.
>>> > > > >
>>> > > > > Aman
>>> > > > >
>>> > > > > On Wed, Sep 30, 2015 at 5:36 PM, Jacques Nadeau <
>>> jacq...@dremio.com>
>>> > > > > wrote:
>>> > > > >
>>> > > > > > Hey All,
>>> > > > > >
>>> > > > > > It seems like we should finalize on the 1.2 release pretty
>>> soon. I
>>> > > > > > currently see something like 174 bugs currently open. Of
>>> those, ~20
>>> > > are
>>> > > > > > reviewable and ~20 are in progress. I'm inclined to bounce
>>> > everything
>>> > > > not
>>> > > > > > critical (and not in-progress or reviewable) off of this
>>> release.
>>> > Any
>>> > > > > > thoughts on when everyone would like to do a release? Also,
>>> anyone
>>> > > want
>>> > > > > to
>>> > > > > > be release manager?
>>> > > > > >
>>> > > > > > Jacques
>>> > > > > >
>>> > > > > > --
>>> > > > > > Jacques Nadeau
>>> > > > > > CTO and Co-Founder, Dremio
>>> > > > > >
>>> > > > > > On Tue, Aug 18, 2015 at 7:58 PM, Parth Chandra <
>>> par...@apache.org>
>>> > > > > wrote:
>>> > > > > >
>>> > > > > > > A couple of reasons -
>>> > > > > > >   a) We definitely need more QA time for the features that
>>> are
>>> > > ready.
>>> > > > > > >   b) Some features, in particular, Parquet metadata caching
>>> > > (Steven)
>>> > > > > and
>>> > > > > > > Faster Hive reads (Jason) are close enough to completion
>>> that we
>>> > > > really
>>> > > > > > > should include them. Again the team could use the extra time
>>> to
>>> > QA
>>> > > > > these.
>>> > > > > > >
>>> > > > > > >
>>> > > > > > > On Tue, Aug 18, 2015 at 4:32 PM, Chris Westin <
>>> > > > chriswesti...@gmail.com
>>> > > > > >
>>> > > > > > > wrote:
>>> > > > > > >
>>> > > > > > > > Unfortunately, we've had to modify our plans, and are now
>>> > > thinking
>>> > > > of
>>> > > > > > > > aiming for a vote on September 10th.
>>> > > > > > > >
>>> > > > > > > > On Thu, Aug 13, 2015 at 1:29 PM, Chris Westin <
>>> > > > > chriswesti...@gmail.com
>>> > > > > > >
>>> > > > > > > > wrote:
>>> > > > > > > >
>>> > > > > > > > > We were thinking of targetting for a vote on August 26th.
>>> > > > > > > > >
>>> > > > > > > > > On Thu, Aug 13, 2015 at 1:19 PM, Jacques Nadeau <
>>> > > > > jacq...@dremio.com>
>>> > > > > > > > > wrote:
>>> > > > > > > > >
>>> > > > > > > > >> Following up since we haven't discussed the 1.2 release
>>> in a
>>> > > > > while.
>>> > 

[GitHub] drill pull request: DRILL-3874: flattening large JSON objects uses...

[parthchandra]

Github user parthchandra commented on the pull request:

https://github.com/apache/drill/pull/181#issuecomment-144897173
  
Looks good. +1.
Some minor comments in line


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill pull request: DRILL-3884: Fix lower parallelization issues w...

[vkorukanti]

Github user vkorukanti commented on a diff in the pull request:

https://github.com/apache/drill/pull/185#discussion_r40982402
  
--- Diff: 
contrib/storage-hive/core/src/main/java/org/apache/drill/exec/store/hive/HiveScan.java
 ---
@@ -319,16 +319,12 @@ public ScanStats getScanStats() {
 estRowCount = data/1024;
   }
 
-  // Hive's native reader is neither memory efficient nor fast. If the 
rowcount is below
-  // HIVE_SERDE_SCAN_OVERHEAD_FACTOR, make sure it is at least 
HIVE_SERDE_SCAN_OVERHEAD_FACTOR to enable the planner
-  // to choose HiveDrillNativeParquetScan. Due to the project on top 
of HiveDrillNativeParquetScan, we end up
-  // choosing the HiveScan instead of HiveDrillNativeParquetScan if 
the cost is too low.
-  if (estRowCount <= HIVE_SERDE_SCAN_OVERHEAD_FACTOR) {
-estRowCount = HIVE_SERDE_SCAN_OVERHEAD_FACTOR;
-  }
+  // Hive's native reader is neither memory efficient nor fast. 
Increase the CPU cost
+  // by a factor to let the planner choose HiveDrillNativeScan over 
HiveScan with SerDes.
+  float cpuCost = 1 * HIVE_SERDE_SCAN_OVERHEAD_FACTOR;
--- End diff --

Fixed in updated patch.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill pull request: DRILL-3884: Fix lower parallelization issues w...

[amansinha100]

Github user amansinha100 commented on the pull request:

https://github.com/apache/drill/pull/185#issuecomment-144884113
  
+1 LGTM. 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: Timing for 1.2 release, ideas around things to include

[Parth Chandra]

DRILL-3874: Flattening large JSON objects uses too much direct memory.
 (This one is nearly ready to merge)


On Thu, Oct 1, 2015 at 6:35 PM, Jacques Nadeau  wrote:

> We went through the list. I propose that we include only the following
> issues in 1.2 and move the rest to 1.3
>
> DRILL-1065 Provide a reset command to reset an option to its default value
> DRILL-3884 Hive native scan has lower parallelization leading to
> performance degradation
> DRILL-2879 Add Remaining portion of "Drill extended json's support $oid"
> DRILL-3887 Parquet metadata cache not being used (regression caused by
> DRILL-3788)
>
> What do other people think?
>
>
> --
> Jacques Nadeau
> CTO and Co-Founder, Dremio
>
> On Thu, Oct 1, 2015 at 5:46 PM, Jacques Nadeau  wrote:
>
> > I'm on:
> >
> > https://plus.google.com/hangouts/_/event/ci4rdiju8bv04a64efj5fedd0lc
> >
> > --
> > Jacques Nadeau
> > CTO and Co-Founder, Dremio
> >
> > On Thu, Oct 1, 2015 at 5:45 PM, Jacques Nadeau 
> wrote:
> >
> >> How about now?
> >>
> >>
> >>
> >> --
> >> Jacques Nadeau
> >> CTO and Co-Founder, Dremio
> >>
> >> On Thu, Oct 1, 2015 at 5:38 PM, Parth Chandra 
> wrote:
> >>
> >>> What time do you want the hangout? I'm in favour of bumping if we
> cannot
> >>> get decided by tomorrow.
> >>>
> >>>
> >>>
> >>> On Thu, Oct 1, 2015 at 2:05 PM, Jacques Nadeau 
> >>> wrote:
> >>>
> >>> > On Thu, Oct 1, 2015 at 11:17 AM, Abdel Hakim Deneche <
> >>> > adene...@maprtech.com>
> >>> >  wrote:
> >>> >
> >>> > > I can manage the release.
> >>> > >
> >>> >
> >>> > Great! I use a manual variation of these steps [1] typically. Julian
> >>> Hyde
> >>> > made a much nicer set of steps (for Incubator) here: [2] if you want
> >>> to use
> >>> > those.
> >>> >
> >>> > On Thu, Oct 1, 2015 at 2:32 PM, Parth Chandra 
> >>> wrote:
> >>> >
> >>> > > +1 on the release. I'm all for starting the process ASAP.
> >>> > >
> >>> >
> >>> > I've moved all the non-critical, non-active jiras out of 1.2.
> However,
> >>> that
> >>> > still leaves 63 issues open. Do you want to do a quick hangout to try
> >>> to
> >>> > figure what we really want to include versus bump?
> >>> >
> >>> > J
> >>> >
> >>> > [1]
> >>> >
> >>> >
> >>>
> https://gist.github.com/jacques-n/be0d2f584171d832aedb#file-release_commands
> >>> > [2]
> >>> >
> >>> >
> >>>
> https://calcite.incubator.apache.org/docs/howto.html#making-a-release-for-calcite-committers
> >>> >
> >>> >
> >>> >
> >>> > >
> >>> > >
> >>> > >
> >>> > > On Thu, Oct 1, 2015 at 8:17 AM, Abdel Hakim Deneche <
> >>> > adene...@maprtech.com
> >>> > > >
> >>> > > wrote:
> >>> > >
> >>> > > > I can manage the release.
> >>> > > >
> >>> > > > On Wed, Sep 30, 2015 at 6:10 PM, Aman Sinha  >
> >>> > wrote:
> >>> > > >
> >>> > > > > +1 for starting the process of finalizing the 1.2 release and
> >>> 'bounce
> >>> > > > > everything not critical (and not in-progress or reviewable) off
> >>> of
> >>> > this
> >>> > > > > release'.
> >>> > > > >
> >>> > > > > Aman
> >>> > > > >
> >>> > > > > On Wed, Sep 30, 2015 at 5:36 PM, Jacques Nadeau <
> >>> jacq...@dremio.com>
> >>> > > > > wrote:
> >>> > > > >
> >>> > > > > > Hey All,
> >>> > > > > >
> >>> > > > > > It seems like we should finalize on the 1.2 release pretty
> >>> soon. I
> >>> > > > > > currently see something like 174 bugs currently open. Of
> >>> those, ~20
> >>> > > are
> >>> > > > > > reviewable and ~20 are in progress. I'm inclined to bounce
> >>> > everything
> >>> > > > not
> >>> > > > > > critical (and not in-progress or reviewable) off of this
> >>> release.
> >>> > Any
> >>> > > > > > thoughts on when everyone would like to do a release? Also,
> >>> anyone
> >>> > > want
> >>> > > > > to
> >>> > > > > > be release manager?
> >>> > > > > >
> >>> > > > > > Jacques
> >>> > > > > >
> >>> > > > > > --
> >>> > > > > > Jacques Nadeau
> >>> > > > > > CTO and Co-Founder, Dremio
> >>> > > > > >
> >>> > > > > > On Tue, Aug 18, 2015 at 7:58 PM, Parth Chandra <
> >>> par...@apache.org>
> >>> > > > > wrote:
> >>> > > > > >
> >>> > > > > > > A couple of reasons -
> >>> > > > > > >   a) We definitely need more QA time for the features that
> >>> are
> >>> > > ready.
> >>> > > > > > >   b) Some features, in particular, Parquet metadata caching
> >>> > > (Steven)
> >>> > > > > and
> >>> > > > > > > Faster Hive reads (Jason) are close enough to completion
> >>> that we
> >>> > > > really
> >>> > > > > > > should include them. Again the team could use the extra
> time
> >>> to
> >>> > QA
> >>> > > > > these.
> >>> > > > > > >
> >>> > > > > > >
> >>> > > > > > > On Tue, Aug 18, 2015 at 4:32 PM, Chris Westin <
> >>> > > > chriswesti...@gmail.com
> >>> > > > > >
> >>> > > > > > > wrote:
> >>> > > > > > >
> >>> > > > > > > > Unfortunately, we've had to modify our plans, and are now
> >>> > > thinking
> >>> > > > of
> >>> > > > > > > > aiming for a vote on September 10th.
> >>> > > 

[GitHub] drill pull request: DRILL-1065: Support for ALTER ... RESET statem...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/drill/pull/159


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] drill pull request: Parquet meta

[StevenMPhillips]

GitHub user StevenMPhillips opened a pull request:

https://github.com/apache/drill/pull/186

Parquet meta



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/StevenMPhillips/drill parquet_meta

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/drill/pull/186.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #186


commit c3cf974db0fe437f893fa9c1fa907620d93c994f
Author: Steven Phillips 
Date:   2015-10-01T23:54:18Z

DRILL-3887: Fix bug where metadata file not being used

commit 96534e3a5502524171df569373cc47e9860de466
Author: Steven Phillips 
Date:   2015-10-02T03:36:15Z

DRILL-3867: Store relative paths in metadata file

commit 728d6e6b12eec11d52e9f106deecaf311ebd7958
Author: Steven Phillips 
Date:   2015-10-02T03:51:29Z

DRILL-3820: Set 700 permission on metadata file




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (DRILL-3877) drill version 1.0.0 is shown in drill shell for 1.1.0 during start up

[Devender Yadav (JIRA)]

Devender Yadav  created DRILL-3877:
--

 Summary: drill  version 1.0.0 is shown in drill shell for 1.1.0 
during start up 
 Key: DRILL-3877
 URL: https://issues.apache.org/jira/browse/DRILL-3877
 Project: Apache Drill
  Issue Type: Improvement
Reporter: Devender Yadav 
Priority: Trivial


When I started shell for drill 1.1.0:

impadmin@ubuntu:~/softwares/apache-drill-1.1.0/bin$ ./drill-embedded 
Oct 01, 2015 2:18:49 AM org.glassfish.jersey.server.ApplicationHandler 
initialize
INFO: Initiating Jersey application, version Jersey: 2.8 2014-04-29 01:25:26...
apache drill 1.0.0 
"a drill is a terrible thing to waste"
0: jdbc:drill:zk=local> 


apache drill 1.0.0  should be  replaced with apache drill 1.1.0





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)