[GitHub] drill pull request: DRILL-3888: Build test jars for all Drill Modu...

2015-10-06 Thread adityakishore
GitHub user adityakishore opened a pull request: https://github.com/apache/drill/pull/188 DRILL-3888: Build test jars for all Drill Modules This patch moves the test jar configuration to the root pom and remove it from individual module's pom. You can merge this pull request into a

Re: Partial aggregation in Drill-on-Phoenix

2015-10-06 Thread Maryann Xue
Yes, but the partially aggregated results will not contain any duplicate rowkeys, since they are also group-by keys. What we need is the aggregators and call aggregate for each row. We can write a new simpler ResultIterator to replace this, but for now it should work correctly. On Tue, Oct 6, 2015

Re: Partial aggregation in Drill-on-Phoenix

2015-10-06 Thread James Taylor
The results we get back from the server-side scan are already the partial aggregated values we need. GroupedAggregatingResultIterator will collapse adjacent Tuples together which happen to have the same row key. I'm not sure we want/need this to happen. Instead I think we just need to decode the ag

Re: Partial aggregation in Drill-on-Phoenix

2015-10-06 Thread Maryann Xue
Hi James, bq. A few questions for you: not sure I understand the changes you made to PhoenixRecordReader. Is it necessary to wrap the server-side scan results in a GroupedAggregatingResultIterator? Each server-side scan will produce results with a single tuple per group by key. In Phoenix, the Gro

Drill Hangout minutes - 2015-10-06 Re: Drill Hangout starting now

2015-10-06 Thread Parth Chandra
Drill Hangout 2015-10-06 Attendees: Aman, Andries, Daniel, Kris, Charlie, Julien, Jacques, Jason, Jinfeng, Matt, Parth, Sudheesh, Venki 1. Matt hitting issues with Information Schema queries against Hive. Will connect with Venki on Slack to resolve. 2. Julien reported that he's

Re: Partial aggregation in Drill-on-Phoenix

2015-10-06 Thread Jacques Nadeau
I'm not sure how to accomplish this cleanly. The concept of two-phased agg-key distributed aggregation (and exchanges in general) seems very much a physical concept. Since Phoenix can only do half this operation (in parallel), I'm having trouble figuring out what the logical plan would look like if

[jira] [Resolved] (DRILL-3906) Add documentation for LEAD, LAD, FIRST_VALUE, LAST_VALUE and NTILE

2015-10-06 Thread Bridget Bevens (JIRA)
[ https://issues.apache.org/jira/browse/DRILL-3906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bridget Bevens resolved DRILL-3906. --- Resolution: Fixed Docs created, edited, pushed to apache drill site. > Add documentation for

[jira] [Created] (DRILL-3906) Add documentation for LEAD, LAD, FIRST_VALUE, LAST_VALUE and NTILE

2015-10-06 Thread Bridget Bevens (JIRA)
Bridget Bevens created DRILL-3906: - Summary: Add documentation for LEAD, LAD, FIRST_VALUE, LAST_VALUE and NTILE Key: DRILL-3906 URL: https://issues.apache.org/jira/browse/DRILL-3906 Project: Apache Dr

[jira] [Created] (DRILL-3905) Document DROP TABLE support

2015-10-06 Thread Bridget Bevens (JIRA)
Bridget Bevens created DRILL-3905: - Summary: Document DROP TABLE support Key: DRILL-3905 URL: https://issues.apache.org/jira/browse/DRILL-3905 Project: Apache Drill Issue Type: Bug

[jira] [Resolved] (DRILL-3905) Document DROP TABLE support

2015-10-06 Thread Bridget Bevens (JIRA)
[ https://issues.apache.org/jira/browse/DRILL-3905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bridget Bevens resolved DRILL-3905. --- Resolution: Fixed Doc written, reviewed, added to Drill website > Document DROP TABLE support

[jira] [Resolved] (DRILL-3904) Document support for multiple window functions in query

2015-10-06 Thread Bridget Bevens (JIRA)
[ https://issues.apache.org/jira/browse/DRILL-3904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bridget Bevens resolved DRILL-3904. --- Resolution: Fixed Updated SQL window function intro on drill website to include the info > Do

[jira] [Created] (DRILL-3904) Document support for multiple window functions in query

2015-10-06 Thread Bridget Bevens (JIRA)
Bridget Bevens created DRILL-3904: - Summary: Document support for multiple window functions in query Key: DRILL-3904 URL: https://issues.apache.org/jira/browse/DRILL-3904 Project: Apache Drill

Re: Partial aggregation in Drill-on-Phoenix

2015-10-06 Thread James Taylor
Nice progress, Maryann. A few questions for you: not sure I understand the changes you made to PhoenixRecordReader. Is it necessary to wrap the server-side scan results in a GroupedAggregatingResultIterator? Each server-side scan will produce results with a single tuple per group by key. In Phoeni

[jira] [Created] (DRILL-3903) Querying empty directory yield internal index-out-of-bounds error

2015-10-06 Thread Daniel Barclay (Drill) (JIRA)
Daniel Barclay (Drill) created DRILL-3903: - Summary: Querying empty directory yield internal index-out-of-bounds error Key: DRILL-3903 URL: https://issues.apache.org/jira/browse/DRILL-3903 Pro

[jira] [Created] (DRILL-3902) Bad error message: core cause not included in text; maybe wrong kind

2015-10-06 Thread Daniel Barclay (Drill) (JIRA)
Daniel Barclay (Drill) created DRILL-3902: - Summary: Bad error message: core cause not included in text; maybe wrong kind Key: DRILL-3902 URL: https://issues.apache.org/jira/browse/DRILL-3902

Re: Partial aggregation in Drill-on-Phoenix

2015-10-06 Thread Maryann Xue
Added a few fixes in the pull request. Tested with two regions, turned out that half of the result is empty (count = 0). Not sure if there's anything wrong with https://github.com/maryannxue/drill/blob/phoenix_plugin/contrib/storage-phoenix/src/main/java/org/apache/drill/exec/store/phoenix/rel/Phoe

Re: [UDF] How do I return NULL

2015-10-06 Thread Steven Phillips
In addition, your UDF needs to have the attribute "nulls = NullHandling.INTERNAL" On Tue, Oct 6, 2015 at 8:32 AM, Abdel Hakim Deneche wrote: > Hi Tug, > > Let's say your UDF returns an int, your @output field will be defined like > this: > > @Output NullableIntHolder out; > > > To return a NULL

Re: [VOTE] Release Apache Drill 1.2.0 (rc0)

2015-10-06 Thread Aman Sinha
I have filed DRILL-3901 for a performance issue that we are trying to address. We can discuss whether to continue with the existing release candidate or wait for a fix. On Tue, Oct 6, 2015 at 9:38 AM, Edmon Begoli wrote: > Humbly, +1. > > On Tue, Oct 6, 2015 at 12:32 PM, Abdel Hakim Deneche < >

Re: Partial aggregation in Drill-on-Phoenix

2015-10-06 Thread Julian Hyde
Drill's current approach seems adequate for Drill alone but extending it to a heterogenous system that includes Phoenix seems like a hack. I think you should only create Prels for algebra nodes that you know for sure are going to run on the Drill engine. If there's a possibility that it would run

[jira] [Created] (DRILL-3901) Performance regression with doing Explain of COUNT(*) over 100K files

2015-10-06 Thread Aman Sinha (JIRA)
Aman Sinha created DRILL-3901: - Summary: Performance regression with doing Explain of COUNT(*) over 100K files Key: DRILL-3901 URL: https://issues.apache.org/jira/browse/DRILL-3901 Project: Apache Drill

Re: Partial aggregation in Drill-on-Phoenix

2015-10-06 Thread Maryann Xue
The partial aggregate seems to be working now, with one interface extension and one bug fix in the Phoenix project. Will do some code cleanup and create a pull request soon. Still there was a hack in the Drill project which I made to force 2-phase aggregation. I'll try to fix that. Jacques, I hav

[GitHub] drill pull request: DRILL-3876: Avoid an extra copy of the origina...

2015-10-06 Thread jaltekruse
GitHub user jaltekruse opened a pull request: https://github.com/apache/drill/pull/187 DRILL-3876: Avoid an extra copy of the original list when flattening This only fixes a basic case, a more complete refactoring of the rewrite rule could avoid copies in cases with multiple flatten

[jira] [Created] (DRILL-3900) OOM with Hive native scan enabled on TPCH-100 parquet, query 05.q

2015-10-06 Thread Chun Chang (JIRA)
Chun Chang created DRILL-3900: - Summary: OOM with Hive native scan enabled on TPCH-100 parquet, query 05.q Key: DRILL-3900 URL: https://issues.apache.org/jira/browse/DRILL-3900 Project: Apache Drill

[jira] [Created] (DRILL-3899) SplitUpComplexExpressions rule should be enhanced to avoid planning unnecessary copies of data

2015-10-06 Thread Jason Altekruse (JIRA)
Jason Altekruse created DRILL-3899: -- Summary: SplitUpComplexExpressions rule should be enhanced to avoid planning unnecessary copies of data Key: DRILL-3899 URL: https://issues.apache.org/jira/browse/DRILL-3899

[jira] [Created] (DRILL-3898) NPE during external sort when there is not enough space for spilling

2015-10-06 Thread Victoria Markman (JIRA)
Victoria Markman created DRILL-3898: --- Summary: NPE during external sort when there is not enough space for spilling Key: DRILL-3898 URL: https://issues.apache.org/jira/browse/DRILL-3898 Project: Apa

Re: Drill Hangout starting now

2015-10-06 Thread Parth Chandra
Join us here: > https://plus.google.com/hangouts/_/event/ci4rdiju8bv04a64efj5fedd0lc >

Re: [VOTE] Release Apache Drill 1.2.0 (rc0)

2015-10-06 Thread Edmon Begoli
Humbly, +1. On Tue, Oct 6, 2015 at 12:32 PM, Abdel Hakim Deneche wrote: > verified the artifacts checksums and that they are signed by my gpg key. > Built Drill from source in MacOS and CentOS and both builds were successful > and all unit tests passed. Run some window functions queries and ever

Re: [VOTE] Release Apache Drill 1.2.0 (rc0)

2015-10-06 Thread Abdel Hakim Deneche
verified the artifacts checksums and that they are signed by my gpg key. Built Drill from source in MacOS and CentOS and both builds were successful and all unit tests passed. Run some window functions queries and everything seems fine. +1 (binding) On Mon, Oct 5, 2015 at 1:59 PM, Abdel Hakim Den

Re: [UDF] How do I return NULL

2015-10-06 Thread Abdel Hakim Deneche
Hi Tug, Let's say your UDF returns an int, your @output field will be defined like this: @Output NullableIntHolder out; To return a NULL you just have to set: out.isSet = 0; Thanks On Tue, Oct 6, 2015 at 1:56 AM, Tugdual Grall wrote: > Hello Drillers, > > I am developing a custom function

[jira] [Created] (DRILL-3897) Partitions not being pruned

2015-10-06 Thread Nathaniel Auvil (JIRA)
Nathaniel Auvil created DRILL-3897: -- Summary: Partitions not being pruned Key: DRILL-3897 URL: https://issues.apache.org/jira/browse/DRILL-3897 Project: Apache Drill Issue Type: Bug

[UDF] How do I return NULL

2015-10-06 Thread Tugdual Grall
Hello Drillers, I am developing a custom function and I would like to return NULL (based on the value, for example if the varchar is '' I want my function to return NULL) I have not found the way to do it. Regards Tug @tgrall