date:20150824


 [ 
https://issues.apache.org/jira/browse/DRILL-3689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deneche A. Hakim updated DRILL-3689:

Component/s: (was: Execution - Flow)
 Execution - Relational Operators

 incorrect results : aggregate AVG returns wrong results over results returned 
 by LEAD function.
 ---

 Key: DRILL-3689
 URL: https://issues.apache.org/jira/browse/DRILL-3689
 Project: Apache Drill
  Issue Type: Bug
  Components: Execution - Relational Operators
Affects Versions: 1.2.0
 Environment:  private-branch 
 https://github.com/adeneche/incubator-drill/tree/new-window-funcs
Reporter: Khurram Faraaz
Assignee: Deneche A. Hakim
Priority: Critical
  Labels: window_function
 Fix For: 1.2.0


 Aggregate AVG returns wrong results over results returned by LEAD function.
 results returned by Drill
 {code}
 0: jdbc:drill:schema=dfs.tmp SELECT  avg(lead_col1) FROM (SELECT LEAD(col1) 
 OVER(PARTITION BY col7 ORDER BY col1) lead_col1 , col7 FROM FEWRWSPQQ_101) 
 sub_query;
 +-+
 | EXPR$0  |
 +-+
 | 2.35195986941647008E17  |
 +-+
 1 row selected (0.264 seconds)
 {code}
 Explain plan for above query from Drill
 {code}
 0: jdbc:drill:schema=dfs.tmp explain plan for SELECT  avg(lead_col1) FROM 
 (SELECT LEAD(col1) OVER(PARTITION BY col7 ORDER BY col1) lead_col1 , col7 
 FROM FEWRWSPQQ_101) sub_query;
 +--+--+
 | text | json |
 +--+--+
 | 00-00Screen
 00-01  Project(EXPR$0=[$0])
 00-02Project(EXPR$0=[CAST(/(CastHigh(CASE(=($1, 0), null, $0)), 
 $1)):ANY NOT NULL])
 00-03  StreamAgg(group=[{}], agg#0=[$SUM0($0)], agg#1=[COUNT($0)])
 00-04Project(w0$o0=[$3])
 00-05  Window(window#0=[window(partition {2} order by [1] range 
 between UNBOUNDED PRECEDING and CURRENT ROW aggs [LEAD($1)])])
 00-06SelectionVectorRemover
 00-07  Sort(sort0=[$2], sort1=[$1], dir0=[ASC], dir1=[ASC])
 00-08Project(T36¦¦*=[$0], col1=[$1], col7=[$2])
 00-09  Scan(groupscan=[ParquetGroupScan 
 [entries=[ReadEntryWithPath [path=maprfs:///tmp/FEWRWSPQQ_101]], 
 selectionRoot=maprfs:/tmp/FEWRWSPQQ_101, numFiles=1, columns=[`*`]]])
 {code}
 results returned by Postgres
 {code}
 postgres=# SELECT  avg(lead_col1) FROM (SELECT LEAD(col1) OVER(PARTITION BY 
 col7 ORDER BY col1) lead_col1 , col7 FROM FEWRWSPQQ_101) sub_query;
  avg 
 -
  1157533190627124568
 (1 row)
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (DRILL-3689) incorrect results : aggregate AVG returns wrong results over results returned by LEAD function.


 [ 
https://issues.apache.org/jira/browse/DRILL-3689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deneche A. Hakim updated DRILL-3689:

Fix Version/s: 1.2.0

 incorrect results : aggregate AVG returns wrong results over results returned 
 by LEAD function.
 ---

 Key: DRILL-3689
 URL: https://issues.apache.org/jira/browse/DRILL-3689
 Project: Apache Drill
  Issue Type: Bug
  Components: Execution - Relational Operators
Affects Versions: 1.2.0
 Environment:  private-branch 
 https://github.com/adeneche/incubator-drill/tree/new-window-funcs
Reporter: Khurram Faraaz
Assignee: Deneche A. Hakim
Priority: Critical
  Labels: window_function
 Fix For: 1.2.0


 Aggregate AVG returns wrong results over results returned by LEAD function.
 results returned by Drill
 {code}
 0: jdbc:drill:schema=dfs.tmp SELECT  avg(lead_col1) FROM (SELECT LEAD(col1) 
 OVER(PARTITION BY col7 ORDER BY col1) lead_col1 , col7 FROM FEWRWSPQQ_101) 
 sub_query;
 +-+
 | EXPR$0  |
 +-+
 | 2.35195986941647008E17  |
 +-+
 1 row selected (0.264 seconds)
 {code}
 Explain plan for above query from Drill
 {code}
 0: jdbc:drill:schema=dfs.tmp explain plan for SELECT  avg(lead_col1) FROM 
 (SELECT LEAD(col1) OVER(PARTITION BY col7 ORDER BY col1) lead_col1 , col7 
 FROM FEWRWSPQQ_101) sub_query;
 +--+--+
 | text | json |
 +--+--+
 | 00-00Screen
 00-01  Project(EXPR$0=[$0])
 00-02Project(EXPR$0=[CAST(/(CastHigh(CASE(=($1, 0), null, $0)), 
 $1)):ANY NOT NULL])
 00-03  StreamAgg(group=[{}], agg#0=[$SUM0($0)], agg#1=[COUNT($0)])
 00-04Project(w0$o0=[$3])
 00-05  Window(window#0=[window(partition {2} order by [1] range 
 between UNBOUNDED PRECEDING and CURRENT ROW aggs [LEAD($1)])])
 00-06SelectionVectorRemover
 00-07  Sort(sort0=[$2], sort1=[$1], dir0=[ASC], dir1=[ASC])
 00-08Project(T36¦¦*=[$0], col1=[$1], col7=[$2])
 00-09  Scan(groupscan=[ParquetGroupScan 
 [entries=[ReadEntryWithPath [path=maprfs:///tmp/FEWRWSPQQ_101]], 
 selectionRoot=maprfs:/tmp/FEWRWSPQQ_101, numFiles=1, columns=[`*`]]])
 {code}
 results returned by Postgres
 {code}
 postgres=# SELECT  avg(lead_col1) FROM (SELECT LEAD(col1) OVER(PARTITION BY 
 col7 ORDER BY col1) lead_col1 , col7 FROM FEWRWSPQQ_101) sub_query;
  avg 
 -
  1157533190627124568
 (1 row)
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-3616) Memory leak in a cleanup code after canceling queries with window functions spilling to disk

2015-08-24 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/DRILL-3616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14709425#comment-14709425
 ] 

ASF GitHub Bot commented on DRILL-3616:
---

Github user adeneche closed the pull request at:

https://github.com/apache/drill/pull/112


 Memory leak in a cleanup code after canceling queries with window functions 
 spilling to disk
 

 Key: DRILL-3616
 URL: https://issues.apache.org/jira/browse/DRILL-3616
 Project: Apache Drill
  Issue Type: Sub-task
  Components: Execution - Flow
Affects Versions: 1.2.0
 Environment: private-locking-allocator-branch
Reporter: Victoria Markman
Assignee: Deneche A. Hakim
 Fix For: 1.2.0

 Attachments: DRILL-3616.1.patch.txt


 Bunch of concurrent queries with window functions were cancelled.
 Got an error in drillbit.log that might indicate that we have a memory leak 
 in in cleanup code after cancellation.
 Assigning to myself for creation of a reproducible test case.
 {code}
 2015-08-05 22:43:56,475 [2a3d6e54-12c2-2519-3ea1-736cb1e39e2a:frag:0:0] INFO  
 o.a.d.e.w.fragment.FragmentExecutor - 
 2a3d6e54-12c2-2519-3ea1-736cb1e39e2a:0:0: State change requested from 
 CANCELLATION_REQUESTED -- FAILED
 2015-08-05 22:43:56,475 [2a3d6e54-12c2-2519-3ea1-736cb1e39e2a:frag:0:0] INFO  
 o.a.d.e.w.fragment.FragmentExecutor - 
 2a3d6e54-12c2-2519-3ea1-736cb1e39e2a:0:0: State change requested from FAILED 
 -- FAILED
 2015-08-05 22:43:56,475 [2a3d6e54-12c2-2519-3ea1-736cb1e39e2a:frag:0:0] INFO  
 o.a.d.e.w.fragment.FragmentExecutor - 
 2a3d6e54-12c2-2519-3ea1-736cb1e39e2a:0:0: State change requested from FAILED 
 -- FINISHED
 2015-08-05 22:43:56,476 [2a3d6e54-12c2-2519-3ea1-736cb1e39e2a:frag:0:0] ERROR 
 o.a.d.e.w.fragment.FragmentExecutor - SYSTEM ERROR: IllegalStateException: 
 Unaccounted for outstanding allocation (902492)
 Fragment 0:0
 [Error Id: 1b9714b9-5a39-48ec-80e7-c49c79825cda on atsqa4-133.qa.lab:31010]
 org.apache.drill.common.exceptions.UserException: SYSTEM ERROR: 
 IllegalStateException: Unaccounted for outstanding allocation (902492)
 Fragment 0:0
 [Error Id: 1b9714b9-5a39-48ec-80e7-c49c79825cda on atsqa4-133.qa.lab:31010]
 at 
 org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:523)
  ~[drill-common-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:323)
  [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:178)
  [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:292)
  [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
  [drill-common-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  [na:1.7.0_71]
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  [na:1.7.0_71]
 at java.lang.Thread.run(Thread.java:745) [na:1.7.0_71]
 Caused by: java.lang.RuntimeException: Exception while closing
 at 
 org.apache.drill.common.DrillAutoCloseables.closeNoChecked(DrillAutoCloseables.java:46)
  ~[drill-common-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.exec.ops.OperatorContextImpl.close(OperatorContextImpl.java:139)
  ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.exec.ops.FragmentContext.suppressingClose(FragmentContext.java:439)
  ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.exec.ops.FragmentContext.close(FragmentContext.java:424) 
 ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.exec.work.fragment.FragmentExecutor.closeOutResources(FragmentExecutor.java:352)
  [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:173)
  [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 ... 5 common frames omitted
 Caused by: java.lang.IllegalStateException: Unaccounted for outstanding 
 allocation (902492)
 at 
 org.apache.drill.exec.memory.BaseAllocator.close(BaseAllocator.java:1278) 
 ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.common.DrillAutoCloseables.closeNoChecked(DrillAutoCloseables.java:44)
  ~[drill-common-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 ... 10 common frames omitted
 2015-08-05 22:43:56,477 [2a3d6e54-f2c1-4682-9121-73c9110e3dd7:frag:0:0]

[jira] [Commented] (DRILL-3698) Expose Show Files Command As SQL for sorting/filtering

2015-08-24 Thread Paul Mogren (JIRA)


[ 
https://issues.apache.org/jira/browse/DRILL-3698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14709394#comment-14709394
 ] 

Paul Mogren commented on DRILL-3698:


+1 for providing table-like access via INFORMATION_SCHEMA as mentioned on the 
mailing list.

 Expose Show Files Command As SQL for sorting/filtering
 --

 Key: DRILL-3698
 URL: https://issues.apache.org/jira/browse/DRILL-3698
 Project: Apache Drill
  Issue Type: Improvement
  Components: SQL Parser
Affects Versions: Future
 Environment: All
Reporter: John Omernik
Assignee: Aman Sinha
  Labels: features
 Fix For: Future


 When using drill, I had a workspace setup, and I found myself using the show 
 files command often to find my directories etc. The thing is, the return of 
 show files is not ordered.  And when looking at file system data there are 
 many possible ways to order the results for efficiency as a user.  
 Consider the ls command in unix; the ability to specify different sorting is 
 built in there.  I checked out 
 http://drill.apache.org/docs/show-files-command/ as well as tried the 
 obvious show files order by name and that didn't work nor did I see how I 
 could in the documentation. 
 Based on a mailing list discussion there is no way to do that currently in 
 Drill, hence this JIRA I think just adding ORDER BY SQL methodology would be 
 perfect here, you have 8 fields (seen below) and ordering by any one of them, 
 or group of them, with ASC/DESC just like standard SQL order by would be a 
 huge win.  
 I suppose one could potentially ask for WHERE clause (filtering)too, and 
 maybe a select (which fieldsto display) however I am more concerned with the 
 order, but if I had to implement all there I could see examples below:
 (All Three, select, where, and order) (I.e. after Files if the token isn't 
 WHERE  or ORDER then check for the fields, if it's not a valid field list 
 error)
 SHOW FILES name, accessTime WHERE name like '%.csv' ORDER BY name;
 (Where clause and order, note the token after FILES is WHERE)
 SHOW FILES WHERE name like '%.csv' ORDER BY length ASC, name DESC;
 (Only Order, ORDER Is the first token after FILES)
 SHOW FILES ORDER BY length ASC, name DESC
 I don't think we have to grant full SQL functionality here (i.e. aggregates), 
 just the ability to display various fields, filter on criteria, and ordering. 
 If you wanted to get fancy, I suppose you could take the table and make it a 
 full on table, i.e. take the results make it a quick inmemory table and then 
 utilize the whole drill stack on it.  Lots of options.  I just wanted to get 
 this down in an email/JIRA as it was something I found myself wishing I had 
 over and over during data exploration. 
 Fields Currently Returned:
 |name| isDirectory|isFile|length|owner 
 group|permissions|accessTime|modificationTime|



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-3689) incorrect results : aggregate AVG returns wrong results over results returned by LEAD function.


[ 
https://issues.apache.org/jira/browse/DRILL-3689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14709463#comment-14709463
 ] 

Deneche A. Hakim commented on DRILL-3689:
-

Looking at the output of the inner query:
{noformat}
select col7, col1, lead(col1) over(partition by col7 order by col1) leadcol1 
from `3648.parquet` order by col7, col1 limit 20;
++--+--+
|  col7  | col1 |   leadcol1   |
++--+--+
| false  | -1   | 1|
| false  | 1| 17   |
| false  | 17   | 30   |
| false  | 30   | 200  |
| false  | 200  | 1000 |
| false  | 1000 | 1001 |
| false  | 1001 | 5000 |
| false  | 5000 | 65534|
| false  | 65534| 4611686018427387903  |
| false  | 4611686018427387903  | 9223372036854775807  |
| false  | 9223372036854775807  | null |
| true   | -65535   | 0|
| true   | 0| 13   |
| true   | 13   | 23   |
| true   | 23   | 25   |
| true   | 25   | 197  |
| true   | 197  | 3000 |
| true   | 3000 | 999  |
| true   | 999  | 1000 |
| true   | 1000 | 92233720385475807|
++--+--+
{noformat}

Because col1 is stored as an INT64, trying to compute the average of 
{{lead(col1)}} will cause an overflow. This explains why we have different 
results than Postgres.

[~khfaraaz] Can you confirm this by running the same query on a column with 
smaller numbers ?

 incorrect results : aggregate AVG returns wrong results over results returned 
 by LEAD function.
 ---

 Key: DRILL-3689
 URL: https://issues.apache.org/jira/browse/DRILL-3689
 Project: Apache Drill
  Issue Type: Bug
  Components: Execution - Relational Operators
Affects Versions: 1.2.0
 Environment:  private-branch 
 https://github.com/adeneche/incubator-drill/tree/new-window-funcs
Reporter: Khurram Faraaz
Assignee: Deneche A. Hakim
Priority: Critical
  Labels: window_function
 Fix For: 1.2.0


 Aggregate AVG returns wrong results over results returned by LEAD function.
 results returned by Drill
 {code}
 0: jdbc:drill:schema=dfs.tmp SELECT  avg(lead_col1) FROM (SELECT LEAD(col1) 
 OVER(PARTITION BY col7 ORDER BY col1) lead_col1 , col7 FROM FEWRWSPQQ_101) 
 sub_query;
 +-+
 | EXPR$0  |
 +-+
 | 2.35195986941647008E17  |
 +-+
 1 row selected (0.264 seconds)
 {code}
 Explain plan for above query from Drill
 {code}
 0: jdbc:drill:schema=dfs.tmp explain plan for SELECT  avg(lead_col1) FROM 
 (SELECT LEAD(col1) OVER(PARTITION BY col7 ORDER BY col1) lead_col1 , col7 
 FROM FEWRWSPQQ_101) sub_query;
 +--+--+
 | text | json |
 +--+--+
 | 00-00Screen
 00-01  Project(EXPR$0=[$0])
 00-02Project(EXPR$0=[CAST(/(CastHigh(CASE(=($1, 0), null, $0)), 
 $1)):ANY NOT NULL])
 00-03  StreamAgg(group=[{}], agg#0=[$SUM0($0)], agg#1=[COUNT($0)])
 00-04Project(w0$o0=[$3])
 00-05  Window(window#0=[window(partition {2} order by [1] range 
 between UNBOUNDED PRECEDING and CURRENT ROW aggs [LEAD($1)])])
 00-06SelectionVectorRemover
 00-07  Sort(sort0=[$2], sort1=[$1], dir0=[ASC], dir1=[ASC])
 00-08Project(T36¦¦*=[$0], col1=[$1], col7=[$2])
 00-09  Scan(groupscan=[ParquetGroupScan 
 [entries=[ReadEntryWithPath [path=maprfs:///tmp/FEWRWSPQQ_101]], 
 selectionRoot=maprfs:/tmp/FEWRWSPQQ_101, numFiles=1, columns=[`*`]]])
 {code}
 results returned by Postgres
 {code}
 postgres=# SELECT  avg(lead_col1) FROM (SELECT LEAD(col1) OVER(PARTITION BY 
 col7 ORDER BY col1) lead_col1 , col7 FROM FEWRWSPQQ_101) sub_query;
  avg 
 -
  1157533190627124568
 (1 row)
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (DRILL-3689) incorrect results : aggregate AVG returns wrong results over results returned by LEAD function.


 [ 
https://issues.apache.org/jira/browse/DRILL-3689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deneche A. Hakim updated DRILL-3689:

Assignee: Khurram Faraaz  (was: Deneche A. Hakim)

 incorrect results : aggregate AVG returns wrong results over results returned 
 by LEAD function.
 ---

 Key: DRILL-3689
 URL: https://issues.apache.org/jira/browse/DRILL-3689
 Project: Apache Drill
  Issue Type: Bug
  Components: Execution - Relational Operators
Affects Versions: 1.2.0
 Environment:  private-branch 
 https://github.com/adeneche/incubator-drill/tree/new-window-funcs
Reporter: Khurram Faraaz
Assignee: Khurram Faraaz
Priority: Critical
  Labels: window_function
 Fix For: 1.2.0


 Aggregate AVG returns wrong results over results returned by LEAD function.
 results returned by Drill
 {code}
 0: jdbc:drill:schema=dfs.tmp SELECT  avg(lead_col1) FROM (SELECT LEAD(col1) 
 OVER(PARTITION BY col7 ORDER BY col1) lead_col1 , col7 FROM FEWRWSPQQ_101) 
 sub_query;
 +-+
 | EXPR$0  |
 +-+
 | 2.35195986941647008E17  |
 +-+
 1 row selected (0.264 seconds)
 {code}
 Explain plan for above query from Drill
 {code}
 0: jdbc:drill:schema=dfs.tmp explain plan for SELECT  avg(lead_col1) FROM 
 (SELECT LEAD(col1) OVER(PARTITION BY col7 ORDER BY col1) lead_col1 , col7 
 FROM FEWRWSPQQ_101) sub_query;
 +--+--+
 | text | json |
 +--+--+
 | 00-00Screen
 00-01  Project(EXPR$0=[$0])
 00-02Project(EXPR$0=[CAST(/(CastHigh(CASE(=($1, 0), null, $0)), 
 $1)):ANY NOT NULL])
 00-03  StreamAgg(group=[{}], agg#0=[$SUM0($0)], agg#1=[COUNT($0)])
 00-04Project(w0$o0=[$3])
 00-05  Window(window#0=[window(partition {2} order by [1] range 
 between UNBOUNDED PRECEDING and CURRENT ROW aggs [LEAD($1)])])
 00-06SelectionVectorRemover
 00-07  Sort(sort0=[$2], sort1=[$1], dir0=[ASC], dir1=[ASC])
 00-08Project(T36¦¦*=[$0], col1=[$1], col7=[$2])
 00-09  Scan(groupscan=[ParquetGroupScan 
 [entries=[ReadEntryWithPath [path=maprfs:///tmp/FEWRWSPQQ_101]], 
 selectionRoot=maprfs:/tmp/FEWRWSPQQ_101, numFiles=1, columns=[`*`]]])
 {code}
 results returned by Postgres
 {code}
 postgres=# SELECT  avg(lead_col1) FROM (SELECT LEAD(col1) OVER(PARTITION BY 
 col7 ORDER BY col1) lead_col1 , col7 FROM FEWRWSPQQ_101) sub_query;
  avg 
 -
  1157533190627124568
 (1 row)
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (DRILL-3700) Exception in a query with multiple fist_value window functions with different partitions


 [ 
https://issues.apache.org/jira/browse/DRILL-3700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Victoria Markman updated DRILL-3700:

Description: 
{code}
alter session set `planner.slice_target` = 1;
select
first_value(c_integer) over(order by c_integer nulls first),
first_value(c_bigint) over(partition by c_time),
first_value(c_integer) over (partition by c_time),
first_value(c_bigint) over(partition by c_time order by c_date),
first_value(c_integer) over (partition by c_time order by c_date)
from
j6;
{code}

drillbit.log
{code}
2015-08-24 15:43:14,157 [2a24c474-67f1-2359-d25d-f4e9c7f9e98b:frag:3:0] ERROR 
o.a.d.e.w.fragment.FragmentExecutor - SYSTEM ERROR: 
OversizedAllocationException: Unable to expand the buffer. Max allowed buffer 
size is reached.

Fragment 3:0

[Error Id: b5e13008-b128-4c39-b25d-9e1529296099 on atsqa4-133.qa.lab:31010]
org.apache.drill.common.exceptions.UserException: SYSTEM ERROR: 
OversizedAllocationException: Unable to expand the buffer. Max allowed buffer 
size is reached.

Fragment 3:0

[Error Id: b5e13008-b128-4c39-b25d-9e1529296099 on atsqa4-133.qa.lab:31010]
at 
org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:523)
 ~[drill-common-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at 
org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:323)
 [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at 
org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:178)
 [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at 
org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:292)
 [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at 
org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38) 
[drill-common-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) 
[na:1.7.0_71]
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) 
[na:1.7.0_71]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_71]
Caused by: org.apache.drill.exec.exception.OversizedAllocationException: Unable 
to expand the buffer. Max allowed buffer size is reached.
at 
org.apache.drill.exec.vector.BigIntVector.reAlloc(BigIntVector.java:205) 
~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at 
org.apache.drill.exec.vector.BigIntVector$Mutator.setSafe(BigIntVector.java:403)
 ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at 
org.apache.drill.exec.vector.NullableBigIntVector$Mutator.setSafe(NullableBigIntVector.java:481)
 ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at 
org.apache.drill.exec.test.generated.WindowFramerGen141.copyFirstValueToInternal(DefaultFrameTemplate.java:243)
 ~[na:na]
at 
org.apache.drill.exec.test.generated.WindowFramerGen141.newPartition(DefaultFrameTemplate.java:140)
 ~[na:na]
at 
org.apache.drill.exec.test.generated.WindowFramerGen141.doWork(DefaultFrameTemplate.java:108)
 ~[na:na]
at 
org.apache.drill.exec.physical.impl.window.WindowFrameRecordBatch.innerNext(WindowFrameRecordBatch.java:157)
 ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at 
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:147)
 ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at 
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:105)
 ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at 
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:95)
 ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at 
org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:51)
 ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at 
org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:129)
 ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at 
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:147)
 ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at 
org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:83) 
~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at 
org.apache.drill.exec.physical.impl.SingleSenderCreator$SingleSenderRootExec.innerNext(SingleSenderCreator.java:95)
 ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at 
org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:73) 
~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at 
org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:258)
 ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at

[jira] [Commented] (DRILL-3679) IOB Exception : when window functions used in outer and inner query


[ 
https://issues.apache.org/jira/browse/DRILL-3679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14709493#comment-14709493
 ] 

Deneche A. Hakim commented on DRILL-3679:
-

Looking at the plan, this looks similar to DRILL-3680: the Project operator 
couldn't find {{w0$o00}} because both window functions output to the same field 
{{w0$o0}}

{noformat}
explain plan for select rnum, ntile(4) over(order by position_id) from (select 
position_id, row_number() over(order by position_id) as rnum from 
cp.`employee.json`);
00-00Screen
00-01  ProjectAllowDup(rnum=[$0], EXPR$1=[$1])
00-02Project(w0$o0=[$1], w0$o00=[$2])
00-03  Window(window#0=[window(partition {} order by [0] range between 
UNBOUNDED PRECEDING and CURRENT ROW aggs [NTILE($2)])])
00-04SelectionVectorRemover
00-05  Sort(sort0=[$0], dir0=[ASC])
00-06Project(position_id=[$1], w0$o0=[$2])
00-07  Window(window#0=[window(partition {} order by [1] range 
between UNBOUNDED PRECEDING and CURRENT ROW aggs [ROW_NUMBER()])])
00-08SelectionVectorRemover
00-09  Sort(sort0=[$1], dir0=[ASC])
00-10Project(T7¦¦*=[$0], position_id=[$1])
00-11  Scan(groupscan=[EasyGroupScan 
[selectionRoot=classpath:/employee.json, numFiles=1, columns=[`*`], 
files=[classpath:/employee.json]]])
...
  }, {
pop : window,
@id : 7,
child : 8,
aggregations : [ {
  ref : `w0$o0`,
  expr : row_number(1) 
} ],
orderings : [ {
  expr : `position_id`,
  order : ASC,
  nullDirection : UNSPECIFIED
} ],
start : -9223372036854775808,
end : -9223372036854775808,
initialAllocation : 100,
maxAllocation : 100,
withins : [ ],
cost : 463.0
  }, 
  ...
  }, {
pop : window,
@id : 3,
child : 4,
aggregations : [ {
  ref : `w0$o0`,
  expr : ntile(4) 
} ],
orderings : [ {
  expr : `position_id`,
  order : ASC,
  nullDirection : UNSPECIFIED
} ],
start : -9223372036854775808,
end : -9223372036854775808,
initialAllocation : 100,
maxAllocation : 100,
withins : [ ],
cost : 463.0
  }, {
pop : project,
@id : 2,
exprs : [ {
  ref : `w0$o0`,
  expr : `w0$o0`
}, {
  ref : `w0$o00`,
  expr : `w0$o0`
} ],
child : 3,
initialAllocation : 100,
maxAllocation : 100,
cost : 463.0
  }, {
  ...
+--+--+
{noformat}

 IOB Exception : when window functions used in outer and inner query
 ---

 Key: DRILL-3679
 URL: https://issues.apache.org/jira/browse/DRILL-3679
 Project: Apache Drill
  Issue Type: Bug
  Components: Query Planning  Optimization
Affects Versions: 1.2.0
 Environment: private-branch 
 https://github.com/adeneche/incubator-drill/tree/new-window-funcs
Reporter: Khurram Faraaz
Assignee: Deneche A. Hakim
  Labels: window_function
 Fix For: 1.2.0


 IOB Exception seen when two different window functions are used in inner and 
 outer queries.
 {code}
 0: jdbc:drill:schema=dfs.tmp select rnum, position_id, ntile(4) over(order 
 by position_id) from (select position_id, row_number() over(order by 
 position_id) as rnum from cp.`employee.json`);
 java.lang.RuntimeException: java.sql.SQLException: SYSTEM ERROR: 
 IndexOutOfBoundsException: index: 0, length: 4 (expected: range(0, 0))
 Fragment 0:0
 [Error Id: 8e0cbf82-842d-4fa7-ab0d-1d982a3d6c24 on centos-03.qa.lab:31010]
   at sqlline.IncrementalRows.hasNext(IncrementalRows.java:73)
   at 
 sqlline.TableOutputFormat$ResizingRowsProvider.next(TableOutputFormat.java:87)
   at sqlline.TableOutputFormat.print(TableOutputFormat.java:118)
   at sqlline.SqlLine.print(SqlLine.java:1583)
   at sqlline.Commands.execute(Commands.java:852)
   at sqlline.Commands.sql(Commands.java:751)
   at sqlline.SqlLine.dispatch(SqlLine.java:738)
   at sqlline.SqlLine.begin(SqlLine.java:612)
   at sqlline.SqlLine.start(SqlLine.java:366)
   at sqlline.SqlLine.main(SqlLine.java:259)
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (DRILL-3679) IOB Exception : when window functions used in outer and inner query


 [ 
https://issues.apache.org/jira/browse/DRILL-3679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deneche A. Hakim updated DRILL-3679:

Assignee: Jinfeng Ni  (was: Deneche A. Hakim)

 IOB Exception : when window functions used in outer and inner query
 ---

 Key: DRILL-3679
 URL: https://issues.apache.org/jira/browse/DRILL-3679
 Project: Apache Drill
  Issue Type: Bug
  Components: Query Planning  Optimization
Affects Versions: 1.2.0
 Environment: private-branch 
 https://github.com/adeneche/incubator-drill/tree/new-window-funcs
Reporter: Khurram Faraaz
Assignee: Jinfeng Ni
  Labels: window_function
 Fix For: 1.2.0


 IOB Exception seen when two different window functions are used in inner and 
 outer queries.
 {code}
 0: jdbc:drill:schema=dfs.tmp select rnum, position_id, ntile(4) over(order 
 by position_id) from (select position_id, row_number() over(order by 
 position_id) as rnum from cp.`employee.json`);
 java.lang.RuntimeException: java.sql.SQLException: SYSTEM ERROR: 
 IndexOutOfBoundsException: index: 0, length: 4 (expected: range(0, 0))
 Fragment 0:0
 [Error Id: 8e0cbf82-842d-4fa7-ab0d-1d982a3d6c24 on centos-03.qa.lab:31010]
   at sqlline.IncrementalRows.hasNext(IncrementalRows.java:73)
   at 
 sqlline.TableOutputFormat$ResizingRowsProvider.next(TableOutputFormat.java:87)
   at sqlline.TableOutputFormat.print(TableOutputFormat.java:118)
   at sqlline.SqlLine.print(SqlLine.java:1583)
   at sqlline.Commands.execute(Commands.java:852)
   at sqlline.Commands.sql(Commands.java:751)
   at sqlline.SqlLine.dispatch(SqlLine.java:738)
   at sqlline.SqlLine.begin(SqlLine.java:612)
   at sqlline.SqlLine.start(SqlLine.java:366)
   at sqlline.SqlLine.main(SqlLine.java:259)
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-3700) Exception in a query with multiple fist_value window functions with different partitions


[ 
https://issues.apache.org/jira/browse/DRILL-3700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14709520#comment-14709520
 ] 

Deneche A. Hakim commented on DRILL-3700:
-

it's most likely a bug in the Window functions operators

 Exception in a query with multiple fist_value window functions with different 
 partitions
 

 Key: DRILL-3700
 URL: https://issues.apache.org/jira/browse/DRILL-3700
 Project: Apache Drill
  Issue Type: Bug
  Components: Execution - Flow
Affects Versions: 1.2.0
Reporter: Victoria Markman
Assignee: Deneche A. Hakim
  Labels: window_function
 Fix For: 1.2.0

 Attachments: j6.tar


 {code}
 alter session set `planner.slice_target` = 1;
 select
 first_value(c_integer) over(order by c_integer nulls first),
 first_value(c_bigint) over(partition by c_time),
 first_value(c_integer) over (partition by c_time),
 first_value(c_bigint) over(partition by c_time order by c_date),
 first_value(c_integer) over (partition by c_time order by c_date)
 from
 j6;
 {code}
 drillbit.log
 {code}
 2015-08-24 15:43:14,157 [2a24c474-67f1-2359-d25d-f4e9c7f9e98b:frag:3:0] ERROR 
 o.a.d.e.w.fragment.FragmentExecutor - SYSTEM ERROR: 
 OversizedAllocationException: Unable to expand the buffer. Max allowed buffer 
 size is reached.
 Fragment 3:0
 [Error Id: b5e13008-b128-4c39-b25d-9e1529296099 on atsqa4-133.qa.lab:31010]
 org.apache.drill.common.exceptions.UserException: SYSTEM ERROR: 
 OversizedAllocationException: Unable to expand the buffer. Max allowed buffer 
 size is reached.
 Fragment 3:0
 [Error Id: b5e13008-b128-4c39-b25d-9e1529296099 on atsqa4-133.qa.lab:31010]
 at 
 org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:523)
  ~[drill-common-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:323)
  [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:178)
  [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:292)
  [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
  [drill-common-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  [na:1.7.0_71]
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  [na:1.7.0_71]
 at java.lang.Thread.run(Thread.java:745) [na:1.7.0_71]
 Caused by: org.apache.drill.exec.exception.OversizedAllocationException: 
 Unable to expand the buffer. Max allowed buffer size is reached.
 at 
 org.apache.drill.exec.vector.BigIntVector.reAlloc(BigIntVector.java:205) 
 ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.exec.vector.BigIntVector$Mutator.setSafe(BigIntVector.java:403)
  ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.exec.vector.NullableBigIntVector$Mutator.setSafe(NullableBigIntVector.java:481)
  ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.exec.test.generated.WindowFramerGen141.copyFirstValueToInternal(DefaultFrameTemplate.java:243)
  ~[na:na]
 at 
 org.apache.drill.exec.test.generated.WindowFramerGen141.newPartition(DefaultFrameTemplate.java:140)
  ~[na:na]
 at 
 org.apache.drill.exec.test.generated.WindowFramerGen141.doWork(DefaultFrameTemplate.java:108)
  ~[na:na]
 at 
 org.apache.drill.exec.physical.impl.window.WindowFrameRecordBatch.innerNext(WindowFrameRecordBatch.java:157)
  ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:147)
  ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:105)
  ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:95)
  ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:51)
  ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:129)
  ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at

[jira] [Updated] (DRILL-3700) Exception in a query with multiple fist_value window functions with different partitions


 [ 
https://issues.apache.org/jira/browse/DRILL-3700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Victoria Markman updated DRILL-3700:

Attachment: j6.tar

 Exception in a query with multiple fist_value window functions with different 
 partitions
 

 Key: DRILL-3700
 URL: https://issues.apache.org/jira/browse/DRILL-3700
 Project: Apache Drill
  Issue Type: Bug
  Components: Execution - Flow
Affects Versions: 1.2.0
Reporter: Victoria Markman
Assignee: Chris Westin
 Attachments: j6.tar


 {code}
 select
 first_value(c_integer) over(order by c_integer nulls first),
 first_value(c_bigint) over(partition by c_time),
 first_value(c_integer) over (partition by c_time),
 first_value(c_bigint) over(partition by c_time order by c_date),
 first_value(c_integer) over (partition by c_time order by c_date)
 from
 j6;
 {code}
 drillbit.log
 {code}
 2015-08-24 15:43:14,157 [2a24c474-67f1-2359-d25d-f4e9c7f9e98b:frag:3:0] ERROR 
 o.a.d.e.w.fragment.FragmentExecutor - SYSTEM ERROR: 
 OversizedAllocationException: Unable to expand the buffer. Max allowed buffer 
 size is reached.
 Fragment 3:0
 [Error Id: b5e13008-b128-4c39-b25d-9e1529296099 on atsqa4-133.qa.lab:31010]
 org.apache.drill.common.exceptions.UserException: SYSTEM ERROR: 
 OversizedAllocationException: Unable to expand the buffer. Max allowed buffer 
 size is reached.
 Fragment 3:0
 [Error Id: b5e13008-b128-4c39-b25d-9e1529296099 on atsqa4-133.qa.lab:31010]
 at 
 org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:523)
  ~[drill-common-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:323)
  [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:178)
  [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:292)
  [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
  [drill-common-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  [na:1.7.0_71]
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  [na:1.7.0_71]
 at java.lang.Thread.run(Thread.java:745) [na:1.7.0_71]
 Caused by: org.apache.drill.exec.exception.OversizedAllocationException: 
 Unable to expand the buffer. Max allowed buffer size is reached.
 at 
 org.apache.drill.exec.vector.BigIntVector.reAlloc(BigIntVector.java:205) 
 ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.exec.vector.BigIntVector$Mutator.setSafe(BigIntVector.java:403)
  ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.exec.vector.NullableBigIntVector$Mutator.setSafe(NullableBigIntVector.java:481)
  ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.exec.test.generated.WindowFramerGen141.copyFirstValueToInternal(DefaultFrameTemplate.java:243)
  ~[na:na]
 at 
 org.apache.drill.exec.test.generated.WindowFramerGen141.newPartition(DefaultFrameTemplate.java:140)
  ~[na:na]
 at 
 org.apache.drill.exec.test.generated.WindowFramerGen141.doWork(DefaultFrameTemplate.java:108)
  ~[na:na]
 at 
 org.apache.drill.exec.physical.impl.window.WindowFrameRecordBatch.innerNext(WindowFrameRecordBatch.java:157)
  ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:147)
  ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:105)
  ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:95)
  ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:51)
  ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:129)
  ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:147)
  ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:83)

[jira] [Updated] (DRILL-3700) Exception in a query with multiple fist_value window functions with different partitions


 [ 
https://issues.apache.org/jira/browse/DRILL-3700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deneche A. Hakim updated DRILL-3700:

Labels: window_function  (was: )

 Exception in a query with multiple fist_value window functions with different 
 partitions
 

 Key: DRILL-3700
 URL: https://issues.apache.org/jira/browse/DRILL-3700
 Project: Apache Drill
  Issue Type: Bug
  Components: Execution - Flow
Affects Versions: 1.2.0
Reporter: Victoria Markman
Assignee: Deneche A. Hakim
  Labels: window_function
 Fix For: 1.2.0

 Attachments: j6.tar


 {code}
 alter session set `planner.slice_target` = 1;
 select
 first_value(c_integer) over(order by c_integer nulls first),
 first_value(c_bigint) over(partition by c_time),
 first_value(c_integer) over (partition by c_time),
 first_value(c_bigint) over(partition by c_time order by c_date),
 first_value(c_integer) over (partition by c_time order by c_date)
 from
 j6;
 {code}
 drillbit.log
 {code}
 2015-08-24 15:43:14,157 [2a24c474-67f1-2359-d25d-f4e9c7f9e98b:frag:3:0] ERROR 
 o.a.d.e.w.fragment.FragmentExecutor - SYSTEM ERROR: 
 OversizedAllocationException: Unable to expand the buffer. Max allowed buffer 
 size is reached.
 Fragment 3:0
 [Error Id: b5e13008-b128-4c39-b25d-9e1529296099 on atsqa4-133.qa.lab:31010]
 org.apache.drill.common.exceptions.UserException: SYSTEM ERROR: 
 OversizedAllocationException: Unable to expand the buffer. Max allowed buffer 
 size is reached.
 Fragment 3:0
 [Error Id: b5e13008-b128-4c39-b25d-9e1529296099 on atsqa4-133.qa.lab:31010]
 at 
 org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:523)
  ~[drill-common-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:323)
  [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:178)
  [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:292)
  [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
  [drill-common-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  [na:1.7.0_71]
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  [na:1.7.0_71]
 at java.lang.Thread.run(Thread.java:745) [na:1.7.0_71]
 Caused by: org.apache.drill.exec.exception.OversizedAllocationException: 
 Unable to expand the buffer. Max allowed buffer size is reached.
 at 
 org.apache.drill.exec.vector.BigIntVector.reAlloc(BigIntVector.java:205) 
 ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.exec.vector.BigIntVector$Mutator.setSafe(BigIntVector.java:403)
  ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.exec.vector.NullableBigIntVector$Mutator.setSafe(NullableBigIntVector.java:481)
  ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.exec.test.generated.WindowFramerGen141.copyFirstValueToInternal(DefaultFrameTemplate.java:243)
  ~[na:na]
 at 
 org.apache.drill.exec.test.generated.WindowFramerGen141.newPartition(DefaultFrameTemplate.java:140)
  ~[na:na]
 at 
 org.apache.drill.exec.test.generated.WindowFramerGen141.doWork(DefaultFrameTemplate.java:108)
  ~[na:na]
 at 
 org.apache.drill.exec.physical.impl.window.WindowFrameRecordBatch.innerNext(WindowFrameRecordBatch.java:157)
  ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:147)
  ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:105)
  ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:95)
  ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:51)
  ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:129)
  ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:147)

[jira] [Assigned] (DRILL-3700) Exception in a query with multiple fist_value window functions with different partitions


 [ 
https://issues.apache.org/jira/browse/DRILL-3700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deneche A. Hakim reassigned DRILL-3700:
---

Assignee: Deneche A. Hakim  (was: Chris Westin)

 Exception in a query with multiple fist_value window functions with different 
 partitions
 

 Key: DRILL-3700
 URL: https://issues.apache.org/jira/browse/DRILL-3700
 Project: Apache Drill
  Issue Type: Bug
  Components: Execution - Flow
Affects Versions: 1.2.0
Reporter: Victoria Markman
Assignee: Deneche A. Hakim
  Labels: window_function
 Fix For: 1.2.0

 Attachments: j6.tar


 {code}
 alter session set `planner.slice_target` = 1;
 select
 first_value(c_integer) over(order by c_integer nulls first),
 first_value(c_bigint) over(partition by c_time),
 first_value(c_integer) over (partition by c_time),
 first_value(c_bigint) over(partition by c_time order by c_date),
 first_value(c_integer) over (partition by c_time order by c_date)
 from
 j6;
 {code}
 drillbit.log
 {code}
 2015-08-24 15:43:14,157 [2a24c474-67f1-2359-d25d-f4e9c7f9e98b:frag:3:0] ERROR 
 o.a.d.e.w.fragment.FragmentExecutor - SYSTEM ERROR: 
 OversizedAllocationException: Unable to expand the buffer. Max allowed buffer 
 size is reached.
 Fragment 3:0
 [Error Id: b5e13008-b128-4c39-b25d-9e1529296099 on atsqa4-133.qa.lab:31010]
 org.apache.drill.common.exceptions.UserException: SYSTEM ERROR: 
 OversizedAllocationException: Unable to expand the buffer. Max allowed buffer 
 size is reached.
 Fragment 3:0
 [Error Id: b5e13008-b128-4c39-b25d-9e1529296099 on atsqa4-133.qa.lab:31010]
 at 
 org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:523)
  ~[drill-common-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:323)
  [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:178)
  [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:292)
  [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
  [drill-common-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  [na:1.7.0_71]
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  [na:1.7.0_71]
 at java.lang.Thread.run(Thread.java:745) [na:1.7.0_71]
 Caused by: org.apache.drill.exec.exception.OversizedAllocationException: 
 Unable to expand the buffer. Max allowed buffer size is reached.
 at 
 org.apache.drill.exec.vector.BigIntVector.reAlloc(BigIntVector.java:205) 
 ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.exec.vector.BigIntVector$Mutator.setSafe(BigIntVector.java:403)
  ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.exec.vector.NullableBigIntVector$Mutator.setSafe(NullableBigIntVector.java:481)
  ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.exec.test.generated.WindowFramerGen141.copyFirstValueToInternal(DefaultFrameTemplate.java:243)
  ~[na:na]
 at 
 org.apache.drill.exec.test.generated.WindowFramerGen141.newPartition(DefaultFrameTemplate.java:140)
  ~[na:na]
 at 
 org.apache.drill.exec.test.generated.WindowFramerGen141.doWork(DefaultFrameTemplate.java:108)
  ~[na:na]
 at 
 org.apache.drill.exec.physical.impl.window.WindowFrameRecordBatch.innerNext(WindowFrameRecordBatch.java:157)
  ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:147)
  ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:105)
  ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:95)
  ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:51)
  ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:129)
  ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:147)

[jira] [Resolved] (DRILL-3685) Failure to execute query with NTILE and ROW_NUMBER window functions with different window definitions


 [ 
https://issues.apache.org/jira/browse/DRILL-3685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deneche A. Hakim resolved DRILL-3685.
-
Resolution: Cannot Reproduce

Couldn't reproduce on master

 Failure to execute query with NTILE and ROW_NUMBER window functions with 
 different window definitions
 -

 Key: DRILL-3685
 URL: https://issues.apache.org/jira/browse/DRILL-3685
 Project: Apache Drill
  Issue Type: Bug
  Components: Execution - Data Types
Affects Versions: 1.2.0
Reporter: Victoria Markman
Assignee: Deneche A. Hakim
  Labels: window_function
 Fix For: 1.2.0

 Attachments: drillbit.log.drill-3685, j2.tar


 {code}
 0: jdbc:drill:schema=dfs select
 . . . . . . . . . . . .  c_integer,
 . . . . . . . . . . . .  row_number()   over (order by c_date desc),
 . . . . . . . . . . . .  ntile(100) over (order by c_date, 
 c_timestamp )
 . . . . . . . . . . . .  from
 . . . . . . . . . . . .  j2
 . . . . . . . . . . . .  ;
 Error: SYSTEM ERROR: UnsupportedOperationException: Unable to get value 
 vector class for minor type [LATE] and mode [OPTIONAL]
 Fragment 0:0
 [Error Id: 1912dd81-dda4-4576-9bd9-420189ff57fe on atsqa4-133.qa.lab:31010] 
 (state=,code=0)
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (DRILL-3700) Exception in a query with multiple fist_value window functions with different partitions

Victoria Markman created DRILL-3700:
---

 Summary: Exception in a query with multiple fist_value window 
functions with different partitions
 Key: DRILL-3700
 URL: https://issues.apache.org/jira/browse/DRILL-3700
 Project: Apache Drill
  Issue Type: Bug
  Components: Execution - Flow
Affects Versions: 1.2.0
Reporter: Victoria Markman
Assignee: Chris Westin


{code}
select
first_value(c_integer) over(order by c_integer nulls first),
first_value(c_bigint) over(partition by c_time),
first_value(c_integer) over (partition by c_time),
first_value(c_bigint) over(partition by c_time order by c_date),
first_value(c_integer) over (partition by c_time order by c_date)
from
j6;
{code}

drillbit.log
{code}
2015-08-24 15:43:14,157 [2a24c474-67f1-2359-d25d-f4e9c7f9e98b:frag:3:0] ERROR 
o.a.d.e.w.fragment.FragmentExecutor - SYSTEM ERROR: 
OversizedAllocationException: Unable to expand the buffer. Max allowed buffer 
size is reached.

Fragment 3:0

[Error Id: b5e13008-b128-4c39-b25d-9e1529296099 on atsqa4-133.qa.lab:31010]
org.apache.drill.common.exceptions.UserException: SYSTEM ERROR: 
OversizedAllocationException: Unable to expand the buffer. Max allowed buffer 
size is reached.

Fragment 3:0

[Error Id: b5e13008-b128-4c39-b25d-9e1529296099 on atsqa4-133.qa.lab:31010]
at 
org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:523)
 ~[drill-common-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at 
org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:323)
 [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at 
org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:178)
 [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at 
org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:292)
 [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at 
org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38) 
[drill-common-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) 
[na:1.7.0_71]
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) 
[na:1.7.0_71]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_71]
Caused by: org.apache.drill.exec.exception.OversizedAllocationException: Unable 
to expand the buffer. Max allowed buffer size is reached.
at 
org.apache.drill.exec.vector.BigIntVector.reAlloc(BigIntVector.java:205) 
~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at 
org.apache.drill.exec.vector.BigIntVector$Mutator.setSafe(BigIntVector.java:403)
 ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at 
org.apache.drill.exec.vector.NullableBigIntVector$Mutator.setSafe(NullableBigIntVector.java:481)
 ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at 
org.apache.drill.exec.test.generated.WindowFramerGen141.copyFirstValueToInternal(DefaultFrameTemplate.java:243)
 ~[na:na]
at 
org.apache.drill.exec.test.generated.WindowFramerGen141.newPartition(DefaultFrameTemplate.java:140)
 ~[na:na]
at 
org.apache.drill.exec.test.generated.WindowFramerGen141.doWork(DefaultFrameTemplate.java:108)
 ~[na:na]
at 
org.apache.drill.exec.physical.impl.window.WindowFrameRecordBatch.innerNext(WindowFrameRecordBatch.java:157)
 ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at 
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:147)
 ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at 
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:105)
 ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at 
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:95)
 ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at 
org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:51)
 ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at 
org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:129)
 ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at 
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:147)
 ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at 
org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:83) 
~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at 
org.apache.drill.exec.physical.impl.SingleSenderCreator$SingleSenderRootExec.innerNext(SingleSenderCreator.java:95)
 ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
at 
org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:73)

[jira] [Updated] (DRILL-3679) IOB Exception : when window functions used in outer and inner query


 [ 
https://issues.apache.org/jira/browse/DRILL-3679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deneche A. Hakim updated DRILL-3679:

Component/s: (was: Execution - Relational Operators)
 Query Planning  Optimization

 IOB Exception : when window functions used in outer and inner query
 ---

 Key: DRILL-3679
 URL: https://issues.apache.org/jira/browse/DRILL-3679
 Project: Apache Drill
  Issue Type: Bug
  Components: Query Planning  Optimization
Affects Versions: 1.2.0
 Environment: private-branch 
 https://github.com/adeneche/incubator-drill/tree/new-window-funcs
Reporter: Khurram Faraaz
Assignee: Deneche A. Hakim
  Labels: window_function
 Fix For: 1.2.0


 IOB Exception seen when two different window functions are used in inner and 
 outer queries.
 {code}
 0: jdbc:drill:schema=dfs.tmp select rnum, position_id, ntile(4) over(order 
 by position_id) from (select position_id, row_number() over(order by 
 position_id) as rnum from cp.`employee.json`);
 java.lang.RuntimeException: java.sql.SQLException: SYSTEM ERROR: 
 IndexOutOfBoundsException: index: 0, length: 4 (expected: range(0, 0))
 Fragment 0:0
 [Error Id: 8e0cbf82-842d-4fa7-ab0d-1d982a3d6c24 on centos-03.qa.lab:31010]
   at sqlline.IncrementalRows.hasNext(IncrementalRows.java:73)
   at 
 sqlline.TableOutputFormat$ResizingRowsProvider.next(TableOutputFormat.java:87)
   at sqlline.TableOutputFormat.print(TableOutputFormat.java:118)
   at sqlline.SqlLine.print(SqlLine.java:1583)
   at sqlline.Commands.execute(Commands.java:852)
   at sqlline.Commands.sql(Commands.java:751)
   at sqlline.SqlLine.dispatch(SqlLine.java:738)
   at sqlline.SqlLine.begin(SqlLine.java:612)
   at sqlline.SqlLine.start(SqlLine.java:366)
   at sqlline.SqlLine.main(SqlLine.java:259)
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (DRILL-3700) Exception in a query with multiple fist_value window functions with different partitions


 [ 
https://issues.apache.org/jira/browse/DRILL-3700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deneche A. Hakim updated DRILL-3700:

Fix Version/s: 1.2.0

 Exception in a query with multiple fist_value window functions with different 
 partitions
 

 Key: DRILL-3700
 URL: https://issues.apache.org/jira/browse/DRILL-3700
 Project: Apache Drill
  Issue Type: Bug
  Components: Execution - Flow
Affects Versions: 1.2.0
Reporter: Victoria Markman
Assignee: Deneche A. Hakim
  Labels: window_function
 Fix For: 1.2.0

 Attachments: j6.tar


 {code}
 alter session set `planner.slice_target` = 1;
 select
 first_value(c_integer) over(order by c_integer nulls first),
 first_value(c_bigint) over(partition by c_time),
 first_value(c_integer) over (partition by c_time),
 first_value(c_bigint) over(partition by c_time order by c_date),
 first_value(c_integer) over (partition by c_time order by c_date)
 from
 j6;
 {code}
 drillbit.log
 {code}
 2015-08-24 15:43:14,157 [2a24c474-67f1-2359-d25d-f4e9c7f9e98b:frag:3:0] ERROR 
 o.a.d.e.w.fragment.FragmentExecutor - SYSTEM ERROR: 
 OversizedAllocationException: Unable to expand the buffer. Max allowed buffer 
 size is reached.
 Fragment 3:0
 [Error Id: b5e13008-b128-4c39-b25d-9e1529296099 on atsqa4-133.qa.lab:31010]
 org.apache.drill.common.exceptions.UserException: SYSTEM ERROR: 
 OversizedAllocationException: Unable to expand the buffer. Max allowed buffer 
 size is reached.
 Fragment 3:0
 [Error Id: b5e13008-b128-4c39-b25d-9e1529296099 on atsqa4-133.qa.lab:31010]
 at 
 org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:523)
  ~[drill-common-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:323)
  [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:178)
  [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:292)
  [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
  [drill-common-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  [na:1.7.0_71]
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  [na:1.7.0_71]
 at java.lang.Thread.run(Thread.java:745) [na:1.7.0_71]
 Caused by: org.apache.drill.exec.exception.OversizedAllocationException: 
 Unable to expand the buffer. Max allowed buffer size is reached.
 at 
 org.apache.drill.exec.vector.BigIntVector.reAlloc(BigIntVector.java:205) 
 ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.exec.vector.BigIntVector$Mutator.setSafe(BigIntVector.java:403)
  ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.exec.vector.NullableBigIntVector$Mutator.setSafe(NullableBigIntVector.java:481)
  ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.exec.test.generated.WindowFramerGen141.copyFirstValueToInternal(DefaultFrameTemplate.java:243)
  ~[na:na]
 at 
 org.apache.drill.exec.test.generated.WindowFramerGen141.newPartition(DefaultFrameTemplate.java:140)
  ~[na:na]
 at 
 org.apache.drill.exec.test.generated.WindowFramerGen141.doWork(DefaultFrameTemplate.java:108)
  ~[na:na]
 at 
 org.apache.drill.exec.physical.impl.window.WindowFrameRecordBatch.innerNext(WindowFrameRecordBatch.java:157)
  ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:147)
  ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:105)
  ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:95)
  ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:51)
  ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:129)
  ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
 at 
 org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:147)

[jira] [Resolved] (DRILL-3685) Failure to execute query with NTILE and ROW_NUMBER window functions with different window definitions


 [ 
https://issues.apache.org/jira/browse/DRILL-3685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deneche A. Hakim resolved DRILL-3685.
-
Resolution: Fixed

seems to be fixed in master

 Failure to execute query with NTILE and ROW_NUMBER window functions with 
 different window definitions
 -

 Key: DRILL-3685
 URL: https://issues.apache.org/jira/browse/DRILL-3685
 Project: Apache Drill
  Issue Type: Bug
  Components: Execution - Data Types
Affects Versions: 1.2.0
Reporter: Victoria Markman
Assignee: Deneche A. Hakim
  Labels: window_function
 Fix For: 1.2.0

 Attachments: drillbit.log.drill-3685, j2.tar


 {code}
 0: jdbc:drill:schema=dfs select
 . . . . . . . . . . . .  c_integer,
 . . . . . . . . . . . .  row_number()   over (order by c_date desc),
 . . . . . . . . . . . .  ntile(100) over (order by c_date, 
 c_timestamp )
 . . . . . . . . . . . .  from
 . . . . . . . . . . . .  j2
 . . . . . . . . . . . .  ;
 Error: SYSTEM ERROR: UnsupportedOperationException: Unable to get value 
 vector class for minor type [LATE] and mode [OPTIONAL]
 Fragment 0:0
 [Error Id: 1912dd81-dda4-4576-9bd9-420189ff57fe on atsqa4-133.qa.lab:31010] 
 (state=,code=0)
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Reopened] (DRILL-3685) Failure to execute query with NTILE and ROW_NUMBER window functions with different window definitions


 [ 
https://issues.apache.org/jira/browse/DRILL-3685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deneche A. Hakim reopened DRILL-3685:
-

 Failure to execute query with NTILE and ROW_NUMBER window functions with 
 different window definitions
 -

 Key: DRILL-3685
 URL: https://issues.apache.org/jira/browse/DRILL-3685
 Project: Apache Drill
  Issue Type: Bug
  Components: Execution - Data Types
Affects Versions: 1.2.0
Reporter: Victoria Markman
Assignee: Deneche A. Hakim
  Labels: window_function
 Fix For: 1.2.0

 Attachments: drillbit.log.drill-3685, j2.tar


 {code}
 0: jdbc:drill:schema=dfs select
 . . . . . . . . . . . .  c_integer,
 . . . . . . . . . . . .  row_number()   over (order by c_date desc),
 . . . . . . . . . . . .  ntile(100) over (order by c_date, 
 c_timestamp )
 . . . . . . . . . . . .  from
 . . . . . . . . . . . .  j2
 . . . . . . . . . . . .  ;
 Error: SYSTEM ERROR: UnsupportedOperationException: Unable to get value 
 vector class for minor type [LATE] and mode [OPTIONAL]
 Fragment 0:0
 [Error Id: 1912dd81-dda4-4576-9bd9-420189ff57fe on atsqa4-133.qa.lab:31010] 
 (state=,code=0)
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-3523) Drill client CLI does not reconnect to Drillbit after long period of inactivity


[ 
https://issues.apache.org/jira/browse/DRILL-3523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14709637#comment-14709637
 ] 

Daniel Barclay (Drill) commented on DRILL-3523:
---

Regarding the seeming timeout and disconnection:

I was not able to reproduce that symptom (neither with Drill 1.1 nor with the 
current state of 1.2, and neither in embedded mode nor local non-embedded mode).

If you see the behavior again, please check whether there's anything that looks 
relevant in the log file (especially when the last visible log activity was, 
relative to your last user-level interaction), and check what netstat reports 
about open TCP listener ports and open TCP connections (especially ports 8047 
and 31010).

 Drill client CLI does not reconnect to Drillbit after long period of 
 inactivity
 ---

 Key: DRILL-3523
 URL: https://issues.apache.org/jira/browse/DRILL-3523
 Project: Apache Drill
  Issue Type: Bug
  Components: Client - CLI
Affects Versions: 1.1.0
Reporter: Hari Sekhon
Assignee: Daniel Barclay (Drill)
 Fix For: 1.2.0


 After a period of inactivity in the Drill CLI client before issuing another 
 statement, it appears to time out the session/connection and the client 
 returns an error instead of reconnecting to Drill.
 It should instead just implicitly !reconnect itself since I can't think of 
 a good reason why I have to do this by hand every time.
 Unlike in DRILL-3514 the drillbit in this case was never restarted, although 
 the same fix could probably solve both issues.
 The error received when trying to issue a new query after some absence is:
 {code}Error: CONNECTION ERROR: Connection /x.x.x.x:54367 -- /x.x.x.x:31010 
 (user client) closed unexpectedly.
 [Error Id: 68714cc2-65f5-4cac-b062-44331f8f4c31 ] (state=,code=0)
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (DRILL-3701) question : getTime and getTimestamp compatibility from JDBC

2015-08-24 Thread Khurram Faraaz (JIRA)

Khurram Faraaz created DRILL-3701:
-

 Summary: question : getTime and getTimestamp compatibility from 
JDBC
 Key: DRILL-3701
 URL: https://issues.apache.org/jira/browse/DRILL-3701
 Project: Apache Drill
  Issue Type: Bug
  Components: Execution - Data Types
Affects Versions: 1.2.0
Reporter: Khurram Faraaz
Assignee: Daniel Barclay (Drill)



When we use getTime instead of getTimestamp function from JDBC program to get a 
timestamp type column from a parquet file, I see this Exception. Should we 
support this from JDBC and return only the TIME portion of the TIMESTAMP value 
when getTime is used over a column type timestamp ? (that is what Drill does 
today on sqlline prompt when we do an explicit cast to TIME)

org.apache.drill.exec.vector.accessor.InvalidAccessException: Requesting value 
of type Time for an object of type TIMESTAMP:OPTIONAL is not allowed.

Note from below query output that the sixth column (col5) is of type TIMESTAMP 
in the parquet file and holds timestamp data.

{code}
0: jdbc:drill:schema=dfs.tmp SELECT * FROM FEWRWSPQQ_101 limit 3;
+---+---+---+-+---+--+-++---+---+
| col0  |   col1|   col2|  col3   | col4  |   col5  
 |col6 |  col7  | col8  | col9  
|
+---+---+---+-+---+--+-++---+---+
| 1 | 65534 | 256.0 | 1234.9  | 20:26:18.580  | 2014-03-02 
00:28:02.338  | 1952-08-14  | false  | CA| 
AXCZ  |
| 2 | 1000  | -256.0| 11.0| 10:59:58.119  | 2014-01-02 
00:28:02.228  | 1981-03-14  | true   | WI| 
BXCD  |
| 3 | -1| 255.9993  | 0.0 | 22:49:49.300  | 2014-09-02 
00:28:02.616  | 2000-01-03  | false  | NY| 
CXCB  |
+---+---+---+-+---+--+-++---+---+
3 rows selected (0.185 seconds)
{code}

Stack trace reported on prompt from where JDBC program is executed.
{code}
...
field {
major_type {
  minor_type: TIMESTAMP
  mode: OPTIONAL
}
name_part {
  type: NAME
  name: col5
}
value_count: 22
buffer_length: 198
...
Requesting value of type Time for an object of type TIMESTAMP:OPTIONAL is not 
allowed.
org.apache.drill.exec.vector.accessor.InvalidAccessException: Requesting value 
of type Time for an object of type TIMESTAMP:OPTIONAL is not allowed.
at 
org.apache.drill.exec.vector.accessor.AbstractSqlAccessor.newInvalidAccessException(AbstractSqlAccessor.java:116)
at 
org.apache.drill.exec.vector.accessor.AbstractSqlAccessor.getTime(AbstractSqlAccessor.java:106)
at 
org.apache.drill.exec.vector.accessor.BoundCheckingAccessor.getTime(BoundCheckingAccessor.java:129)
at 
org.apache.drill.jdbc.impl.TypeConvertingSqlAccessor.getTime(TypeConvertingSqlAccessor.java:689)
at 
org.apache.drill.jdbc.impl.AvaticaDrillSqlAccessor.getTime(AvaticaDrillSqlAccessor.java:219)
at 
net.hydromatic.avatica.AvaticaResultSet.getTime(AvaticaResultSet.java:250)
at 
org.apache.drill.jdbc.impl.DrillResultSetImpl.getTime(DrillResultSetImpl.java:248)
at DataFromDrill.main(DataFromDrill.java:30)
{code}

JDBC snippet to execute the above SQL

{code}
import org.apache.log4j.Logger;

import java.sql.Connection;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Statement;
import java.sql.Types;
import java.sql.*;

public class DataFromDrill {
public static void main(String s[]) {

try {
final String URL_STRING = 
jdbc:drill:schema=dfs.tmp;drillbit=ip-address;
Class.forName(org.apache.drill.jdbc.Driver).newInstance();
Connection conn = 
DriverManager.getConnection(URL_STRING,root,mapr);
Statement stmt = conn.createStatement();

String query = select * from FEWRWSPQQ_101;

ResultSet rs = stmt.executeQuery(query);

while (rs.next()) {
System.out.println(rs.getTime(6));
}

if (rs != null)
rs.close();
conn.close();
} catch ( Exception e ) {
System.out.println(e.getMessage());
e.printStackTrace();
}
}
}
{code}

Explicit cast to TIME of the TIMESTAMP type column results in a successful cast 
from sqlline, and the TIME

[jira] [Updated] (DRILL-3356) RPC-level data is missing data needed for result set metadata


 [ 
https://issues.apache.org/jira/browse/DRILL-3356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Daniel Barclay (Drill) updated DRILL-3356:
--
Priority: Critical  (was: Major)

 RPC-level data is missing data needed for result set metadata 
 --

 Key: DRILL-3356
 URL: https://issues.apache.org/jira/browse/DRILL-3356
 Project: Apache Drill
  Issue Type: Bug
Reporter: Daniel Barclay (Drill)
Priority: Critical
 Fix For: Future


 The result set metadata (schema data) available on the client side is missing 
 data needed by some jdbc.sql.ResultSetMetaData methods.  See DRILL-3355.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (DRILL-2769) many(?) JDBC methods throw non-SQLException exceptions (e.g., UnsupportedOperationException, RuntimeException)


[ 
https://issues.apache.org/jira/browse/DRILL-2769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14709761#comment-14709761
 ] 

Daniel Barclay (Drill) edited comment on DRILL-2769 at 8/24/15 6:19 PM:


Notes on UnsupportedOperationExceptions (and related RuntimeExceptions) in 
current Avatica classes:
- 5 UnsupportedOperationException in ArrayImpl
- 29 UnsupportedOperationException  in AvaticaConnection
- 10 Helper.todo() (RuntimeException) in AvaticaDatabaseMetaData
- 21 UnsupportedOperationException  in AvaticaStatement
-  4 UnsupportedOperationException  in AvaticaPreparedStatement
- 103 UnsupportedOperationException  in AvaticaResultSet

(no UnsupportedOperationException or Helper.todo() in:  AvaticaFactory, 
AvaticaJdbc40Factory,  AvaticaJdbc41Factory, AvaticaParameter, 
AvaticaPrepareResult, AvaticaResultSetMetaData, BuiltInConnectionProperty, 
ByteString, Casing, ColumnMetaData, ConnectionConfig, ConnectionConfigImpl, 
ConnectionProperty, ConnectStringParser, Cursor, DriverVersion, Handler, 
HandlerImpl, Helper, InternalProperty, Meta, Quoting, UnregisteredDriver)

A few of the throw new UnsupportedOperationException() are not directly in 
JDBC-defined methods (although most are).



was (Author: dsbos):
Notes on UnsupportedOperationExceptions (and related RuntimeExceptions) in 
current Avatica classes:
- 5 UnsupportedOperationException in ArrayImpl
- 29 UnsupportedOperationException  in AvaticaConnection
- 10 Helper.todo() (RuntimeException) in AvaticaDatabaseMetaData
- 21 UnsupportedOperationException  in AvaticaStatement
-  4 UnsupportedOperationException  in AvaticaPreparedStatement
- 103 UnsupportedOperationException  in AvaticaResultSet

(no UnsupportedOperationException or Helper.todo() in:  AvaticaFactory, 
AvaticaJdbc40Factory,  AvaticaJdbc41Factory, AvaticaParameter, 
AvaticaPrepareResult, AvaticaResultSetMetaData, BuiltInConnectionProperty, 
ByteString, Casing, ColumnMetaData, ConnectionConfig, ConnectionConfigImpl, 
ConnectionProperty, ConnectStringParser, Cursor, DriverVersion, Handler, 
HandlerImpl, Helper, InternalProperty, Meta, Quoting, UnregisteredDriver)

 many(?) JDBC methods throw non-SQLException exceptions (e.g., 
 UnsupportedOperationException, RuntimeException)
 --

 Key: DRILL-2769
 URL: https://issues.apache.org/jira/browse/DRILL-2769
 Project: Apache Drill
  Issue Type: Bug
Reporter: Daniel Barclay (Drill)
Assignee: Daniel Barclay (Drill)
 Fix For: 1.2.0


 It seems that many JDBC methods throw exceptions of type 
 {{UnsupportedOperationException}} or {{RuntimeException}} to indicate that 
 they are not applicable (e.g., Drill's implementation of 
 {{Connection.commit()}}, since Drill isn't transactional) or not implemented 
 yet (and some throw other {{RuntimeException}}s to indicate other problems ).
 However, these methods should be throwing exceptions of type {{SQLException}} 
 (or subclasses thereof). 
 The JDBC pattern is to throw {{SQLException}}s, not {{RuntimeException}}s, so 
 JDBC client code is not likely to handle {{RuntimeException}}s well.
 (For example, it is suspected that {{Connection.commit()}}'s throwing of 
 {{UnsupportedOperationException}} is causing a hang in the JDBC client 
 Spotfire.)
 JDBC does provide a {{SQLFeatureNotSupportedException}}.  However, it is 
 specified to be for when the JDBC driver does does not support an optional 
 JDBC feature.  It's not clear how risky it would be to use this exception 
 when Drill does not support a _non_-optional JDBC feature. 
 (Possibly, some methods that can't really do what JDBC specifies might need 
 to just return silently without throwing any exception.)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-2769) many(?) JDBC methods throw non-SQLException exceptions (e.g., UnsupportedOperationException, RuntimeException)


[ 
https://issues.apache.org/jira/browse/DRILL-2769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14709761#comment-14709761
 ] 

Daniel Barclay (Drill) commented on DRILL-2769:
---

Notes on UnsupportedOperationExceptions (and related RuntimeExceptions) in 
current Avatica classes:
- 5 UnsupportedOperationException in ArrayImpl
- 29 UnsupportedOperationException  in AvaticaConnection
- 10 Helper.todo() (RuntimeException) in AvaticaDatabaseMetaData
- 21 UnsupportedOperationException  in AvaticaStatement
-  4 UnsupportedOperationException  in AvaticaPreparedStatement
- 103 UnsupportedOperationException  in AvaticaResultSet

(no UnsupportedOperationException or Helper.todo() in:  AvaticaFactory, 
AvaticaJdbc40Factory,  AvaticaJdbc41Factory, AvaticaParameter, 
AvaticaPrepareResult, AvaticaResultSetMetaData, BuiltInConnectionProperty, 
ByteString, Casing, ColumnMetaData, ConnectionConfig, ConnectionConfigImpl, 
ConnectionProperty, ConnectStringParser, Cursor, DriverVersion, Handler, 
HandlerImpl, Helper, InternalProperty, Meta, Quoting, UnregisteredDriver)

 many(?) JDBC methods throw non-SQLException exceptions (e.g., 
 UnsupportedOperationException, RuntimeException)
 --

 Key: DRILL-2769
 URL: https://issues.apache.org/jira/browse/DRILL-2769
 Project: Apache Drill
  Issue Type: Bug
Reporter: Daniel Barclay (Drill)
Assignee: Daniel Barclay (Drill)
 Fix For: 1.2.0


 It seems that many JDBC methods throw exceptions of type 
 {{UnsupportedOperationException}} or {{RuntimeException}} to indicate that 
 they are not applicable (e.g., Drill's implementation of 
 {{Connection.commit()}}, since Drill isn't transactional) or not implemented 
 yet (and some throw other {{RuntimeException}}s to indicate other problems ).
 However, these methods should be throwing exceptions of type {{SQLException}} 
 (or subclasses thereof). 
 The JDBC pattern is to throw {{SQLException}}s, not {{RuntimeException}}s, so 
 JDBC client code is not likely to handle {{RuntimeException}}s well.
 (For example, it is suspected that {{Connection.commit()}}'s throwing of 
 {{UnsupportedOperationException}} is causing a hang in the JDBC client 
 Spotfire.)
 JDBC does provide a {{SQLFeatureNotSupportedException}}.  However, it is 
 specified to be for when the JDBC driver does does not support an optional 
 JDBC feature.  It's not clear how risky it would be to use this exception 
 when Drill does not support a _non_-optional JDBC feature. 
 (Possibly, some methods that can't really do what JDBC specifies might need 
 to just return silently without throwing any exception.)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (DRILL-3669) fix missing direct dependency

2015-08-24 Thread Julien Le Dem (JIRA)


 [ 
https://issues.apache.org/jira/browse/DRILL-3669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Julien Le Dem updated DRILL-3669:
-
Assignee: Jason Altekruse  (was: Julien Le Dem)

 fix missing direct dependency
 -

 Key: DRILL-3669
 URL: https://issues.apache.org/jira/browse/DRILL-3669
 Project: Apache Drill
  Issue Type: Bug
Reporter: Julien Le Dem
Assignee: Jason Altekruse
 Attachments: DRILL-3669.1.patch.txt, DRILL-3669.2.patch.txt


 This prevents generating a compiling project with mvn eclipse:eclipse
 pull request here:
 https://github.com/apache/drill/pull/121/files



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-3643) NTILE(0) returns RuntimeException

2015-08-24 Thread Aman Sinha (JIRA)


[ 
https://issues.apache.org/jira/browse/DRILL-3643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14708872#comment-14708872
 ] 

Aman Sinha commented on DRILL-3643:
---

DRILL-3643:  Changes look good.  +1
DRILL-3654 and DRILL-3668:  I made some comments on github about the zero-ing 
of the value vectors.  Can we discuss that further ? 

 NTILE(0) returns RuntimeException
 -

 Key: DRILL-3643
 URL: https://issues.apache.org/jira/browse/DRILL-3643
 Project: Apache Drill
  Issue Type: Bug
  Components: Execution - Flow
Affects Versions: 1.2.0
 Environment: private-branch 
 https://github.com/adeneche/incubator-drill/tree/new-window-funcs
Reporter: Khurram Faraaz
Assignee: Aman Sinha
Priority: Minor
  Labels: window_function
 Fix For: 1.2.0


 NTILE(0) returns Divide by zero error, on developers private branch
 {code}
 0: jdbc:drill:schema=dfs.tmp select col7 , col9 , ntile(0) over(order by 
 col0) lastVal_col9 from FEWRWSPQQ_101;
 java.lang.RuntimeException: java.sql.SQLException: SYSTEM ERROR: 
 ArithmeticException: / by zero
 Fragment 0:0
 [Error Id: 18d44cb4-f4f3-46eb-a69d-8ace607ef8fe on centos-03.qa.lab:31010]
   at sqlline.IncrementalRows.hasNext(IncrementalRows.java:73)
   at 
 sqlline.TableOutputFormat$ResizingRowsProvider.next(TableOutputFormat.java:87)
   at sqlline.TableOutputFormat.print(TableOutputFormat.java:118)
   at sqlline.SqlLine.print(SqlLine.java:1583)
   at sqlline.Commands.execute(Commands.java:852)
   at sqlline.Commands.sql(Commands.java:751)
   at sqlline.SqlLine.dispatch(SqlLine.java:738)
   at sqlline.SqlLine.begin(SqlLine.java:612)
   at sqlline.SqlLine.start(SqlLine.java:366)
   at sqlline.SqlLine.main(SqlLine.java:259)
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (DRILL-3699) Avro file format should validate the query with schema before processing.

2015-08-24 Thread Bhallamudi Venkata Siva Kamesh (JIRA)

Bhallamudi Venkata Siva Kamesh created DRILL-3699:
-

 Summary: Avro file format should validate the query with schema 
before processing.
 Key: DRILL-3699
 URL: https://issues.apache.org/jira/browse/DRILL-3699
 Project: Apache Drill
  Issue Type: Improvement
  Components: Storage - Other
Reporter: Bhallamudi Venkata Siva Kamesh
Assignee: Jacques Nadeau


For avro data, as schema will be available with data, Drill can evaluate the 
query with the schema before processing. This will avoid unnecessary processing 
and gives users a meaningful information, if the field, a user is using in the 
query does not exist in the data file.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (DRILL-3702) PartitionPruning hit ClassCastException in Interpreter when the pruning filter expression is of non-nullable type.

2015-08-24 Thread Jinfeng Ni (JIRA)

Jinfeng Ni created DRILL-3702:
-

 Summary: PartitionPruning hit ClassCastException in Interpreter 
when the pruning filter expression is of non-nullable type.
 Key: DRILL-3702
 URL: https://issues.apache.org/jira/browse/DRILL-3702
 Project: Apache Drill
  Issue Type: Bug
  Components: Query Planning  Optimization
Reporter: Jinfeng Ni
Assignee: Jinfeng Ni
 Fix For: 1.2.0


I have the following parquet table, created using partition by clause:

{code}
create table mypart (id, name) partition by (id) as select cast(n_regionkey as 
varchar(20)), n_name from cp.`tpch/nation.parquet`;
{code}

The generated parquet table consists of 5 files, each representing a partition:

{code}
0_0_1.parquet 0_0_2.parquet 0_0_3.parquet 0_0_4.parquet 0_0_5.parquet
{code}

For the following query, partition pruning works as expected:

{code}
select id, name from mypart where id  = '0' ;

00-01  Project(id=[$1], name=[$0])
00-02Scan(groupscan=[ParquetGroupScan [entries=[ReadEntryWithPath 
[path=/tmp/mypart/0_0_1.parquet]], selectionRoot=file:/tmp/mypart, numFiles=1, 
columns=[`id`, `name`]]])

selectionRoot : file:/tmp/mypart,
fileSet : [ /tmp/mypart/0_0_1.parquet ],
cost : 5.0
{code}

However, the following query would hit ClassCastException when PruneScanRule 
calls interpreter to evaluate the filtering condition, which happens to be 
non-nullable.

{code}

select id, name from mypart where concat(id,'')  = '0' ;

00-05  Project(id=[$1], name=[$0])
00-06Scan(groupscan=[ParquetGroupScan 
[entries=[ReadEntryWithPath [path=file:/tmp/mypart]], 
selectionRoot=file:/tmp/mypart, numFiles=1, columns=[`id`, `name`]]])

selectionRoot : file:/tmp/mypart,
fileSet : [ /tmp/mypart/0_0_1.parquet, /tmp/mypart/0_0_4.parquet, 
/tmp/mypart/0_0_5.parquet, /tmp/mypart/0_0_2.parquet, 
/tmp/mypart/0_0_3.parquet ],
cost : 25.0
  },
{code}

Here is the error for the ClassCastException, raised in Interpreter:

{code}
java.lang.ClassCastException: org.apache.drill.exec.expr.holders.BitHolder 
cannot be cast to org.apache.drill.exec.expr.holders.NullableBitHolder
{code}

The cause of the problem is that PruneScanRule assumes the output type of a 
filter condition is NullableBit, while in this case the filter condition is Bit 
type, which leads to ClassCastException. 




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (DRILL-3703) TopLevelAllocator has a public static mutable field set as a side effect of its Constructor

2015-08-24 Thread Julien Le Dem (JIRA)

Julien Le Dem created DRILL-3703:


 Summary: TopLevelAllocator has a public static mutable field set 
as a side effect of its Constructor
 Key: DRILL-3703
 URL: https://issues.apache.org/jira/browse/DRILL-3703
 Project: Apache Drill
  Issue Type: Bug
  Components: Execution - Flow
Reporter: Julien Le Dem
Assignee: Chris Westin


https://github.com/apache/drill/blob/c5f1164351fc4b93ba2ef814ad7ab28a80e2/exec/java-exec/src/main/java/org/apache/drill/exec/memory/TopLevelAllocator.java#L44
{noformat}   public static long MAXIMUM_DIRECT_MEMORY;{noformat}
https://github.com/apache/drill/blob/c5f1164351fc4b93ba2ef814ad7ab28a80e2/exec/java-exec/src/main/java/org/apache/drill/exec/memory/TopLevelAllocator.java#L65
{noformat}
private TopLevelAllocator(DrillConfig config, long maximumAllocation, boolean 
errorOnLeak){
MAXIMUM_DIRECT_MEMORY = maximumAllocation;
...
}
{noformat}
It seems to be used only in MemoryIterator (memory system table):
https://github.com/apache/drill/blob/c5f1164351fc4b93ba2ef814ad7ab28a80e2/exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/MemoryIterator.java#L66
Which should not need a static variable to access this information.

https://github.com/apache/drill/search?utf8=%E2%9C%93q=MAXIMUM_DIRECT_MEMORY




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-2743) Parquet file metadata caching

2015-08-24 Thread Rahul Challapalli (JIRA)


[ 
https://issues.apache.org/jira/browse/DRILL-2743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14710026#comment-14710026
 ] 

Rahul Challapalli commented on DRILL-2743:
--

Steven,

I see block locations as part of the cached file. Are these Hdfs/MaprFs block 
locations? If so, do we handle the situation when the blocks get shuffled by 
the balancer?

- Rahul

 Parquet file metadata caching
 -

 Key: DRILL-2743
 URL: https://issues.apache.org/jira/browse/DRILL-2743
 Project: Apache Drill
  Issue Type: New Feature
  Components: Storage - Parquet
Reporter: Steven Phillips
Assignee: Steven Phillips
 Fix For: 1.2.0

 Attachments: DRILL-2743.patch, drill.parquet_metadata


 To run a query against parquet files, we have to first recursively search the 
 directory tree for all of the files, get the block locations for each file, 
 and read the footer from each file, and this is done during the planning 
 phase. When there are many files, this can result in a very large delay in 
 running the query, and it does not scale.
 However, there isn't really any need to read the footers during planning, if 
 we instead treat each parquet file as a single work unit, all we need to know 
 are the block locations for the file, the number of rows, and the columns. We 
 should store only the information which we need for planning in a file 
 located in the top directory for a given parquet table, and then we can delay 
 reading of the footers until execution time, which can be done in parallel.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-2743) Parquet file metadata caching

2015-08-24 Thread Steven Phillips (JIRA)


[ 
https://issues.apache.org/jira/browse/DRILL-2743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14710078#comment-14710078
 ] 

Steven Phillips commented on DRILL-2743:


No, this case is not dealt with, so it is possible for the locations to get out 
of date. This won't cause any wrong results, but could give non-optimal 
performance. The only work around is to manually rerun the refresh metadata 
command.

 Parquet file metadata caching
 -

 Key: DRILL-2743
 URL: https://issues.apache.org/jira/browse/DRILL-2743
 Project: Apache Drill
  Issue Type: New Feature
  Components: Storage - Parquet
Reporter: Steven Phillips
Assignee: Steven Phillips
 Fix For: 1.2.0

 Attachments: DRILL-2743.patch, drill.parquet_metadata


 To run a query against parquet files, we have to first recursively search the 
 directory tree for all of the files, get the block locations for each file, 
 and read the footer from each file, and this is done during the planning 
 phase. When there are many files, this can result in a very large delay in 
 running the query, and it does not scale.
 However, there isn't really any need to read the footers during planning, if 
 we instead treat each parquet file as a single work unit, all we need to know 
 are the block locations for the file, the number of rows, and the columns. We 
 should store only the information which we need for planning in a file 
 located in the top directory for a given parquet table, and then we can delay 
 reading of the footers until execution time, which can be done in parallel.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-3596) Allow only (expression) or (expression, 1) for LEAD and LAG window functions as input parameters

2015-08-24 Thread ASF GitHub Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/DRILL-3596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14710125#comment-14710125
 ] 

ASF GitHub Bot commented on DRILL-3596:
---

GitHub user adeneche opened a pull request:

https://github.com/apache/drill/pull/128

DRILL-3596: Allow only (expression) or (expression, 1) for LEAD a…

…nd LAG window functions as input parameters

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/adeneche/incubator-drill DRILL-3596

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/drill/pull/128.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #128


commit 87e411f38665c3c4cdf4313516862505a4cfdf0c
Author: adeneche adene...@gmail.com
Date:   2015-08-24T20:37:25Z

DRILL-3596: Allow only (expression) or (expression, 1) for LEAD and LAG 
window functions as input parameters




 Allow only (expression) or (expression, 1) for LEAD and LAG window 
 functions as input parameters
 

 Key: DRILL-3596
 URL: https://issues.apache.org/jira/browse/DRILL-3596
 Project: Apache Drill
  Issue Type: Bug
  Components: Execution - Relational Operators
Affects Versions: 1.2.0
Reporter: Khurram Faraaz
Assignee: Deneche A. Hakim
  Labels: window_function
 Fix For: 1.2.0


 We need to disable the third input parameter in LEAD and LAG window 
 functions. We should support this syntax.
 LEAD(expression) ,or LEAD(expression,1)
 LAG(expression), or LAG(expression,1)
 In both the functions above, the second parameter is the default offset 
 value, use of any other value must he handled and reported to user as error.
 Use of a third input parameter to both the functions must be disallowed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (DRILL-3596) Allow only (expression) or (expression, 1) for LEAD and LAG window functions as input parameters


 [ 
https://issues.apache.org/jira/browse/DRILL-3596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deneche A. Hakim updated DRILL-3596:

Assignee: Aman Sinha  (was: Deneche A. Hakim)

 Allow only (expression) or (expression, 1) for LEAD and LAG window 
 functions as input parameters
 

 Key: DRILL-3596
 URL: https://issues.apache.org/jira/browse/DRILL-3596
 Project: Apache Drill
  Issue Type: Bug
  Components: Execution - Relational Operators
Affects Versions: 1.2.0
Reporter: Khurram Faraaz
Assignee: Aman Sinha
  Labels: window_function
 Fix For: 1.2.0


 We need to disable the third input parameter in LEAD and LAG window 
 functions. We should support this syntax.
 LEAD(expression) ,or LEAD(expression,1)
 LAG(expression), or LAG(expression,1)
 In both the functions above, the second parameter is the default offset 
 value, use of any other value must he handled and reported to user as error.
 Use of a third input parameter to both the functions must be disallowed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-3493) Alter session command with an option to cancel query in a particular phase of execution

[
https://issues.apache.org/jira/browse/DRILL-3493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14710303#comment-14710303
]

Deneche A. Hakim commented on DRILL-3493:
-

The way I was thinking of implementing it doesn't involve exception/pause
injection, apart from maybe reusing the injection sites and the activation of
sites through an alter session command.
We want to see what happens when the Foreman receives a user cancellation
request when the code is in a specific state (i.e. the code reaches a
cancellation site), the cancellation site would basically call foreman.cancel()
directly (most likely through a separate cancellation thread to avoid any
concurrent issues) simulating a user cancellation.
This solution has it's limitations, for example it only works for the fragments
running on the Foreman's Drillbit, but it's a step in the right direction and
will at least provide some way to test cancellation in a reproducible manner.

Alter session command with an option to cancel query in a particular phase of
execution
---

Key: DRILL-3493
URL: https://issues.apache.org/jira/browse/DRILL-3493
Project: Apache Drill
Issue Type: New Feature
Components: Execution - Flow
Reporter: Victoria Markman
Assignee: Deneche A. Hakim
Priority: Blocker
Fix For: 1.2.0

This proposal is to implement:
alter session cancel execution phase = true|false.
where execution phase is an entry point for query cancellation, like say
sort or disk based sort , etc.
When this option is set, if query reaches this point, forman would issue
cancel.
This way: we can easily test correctness of query cancellation in different
types of queries.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (DRILL-3705) Query runs out of memory, reported as FAILED and leaves thread running

Victoria Markman created DRILL-3705:
---

 Summary: Query runs out of memory, reported as FAILED and leaves 
thread running 
 Key: DRILL-3705
 URL: https://issues.apache.org/jira/browse/DRILL-3705
 Project: Apache Drill
  Issue Type: Bug
  Components: Execution - Flow
Affects Versions: 1.2.0
Reporter: Victoria Markman
Assignee: Chris Westin


Single node drill installation
DRILL_MAX_DIRECT_MEMORY=2G
DRILL_HEAP=1G

Execute tpcds query 15 SF100 (parquet) with the settings above. Reproduces 2 
out of 3 times.
{code}
SELECT ca.ca_zip,
   Sum(cs.cs_sales_price)
FROM   catalog_salescs,
   customer c,
   customer_address ca,
   date_dim dd
WHERE  cs.cs_bill_customer_sk = c.c_customer_sk
   AND c.c_current_addr_sk = ca.ca_address_sk
   AND ( Substr(ca.ca_zip, 1, 5) IN ( '85669', '86197', '88274', '83405',
   '86475', '85392', '85460', '80348',
   '81792' )
  OR ca.ca_state IN ( 'CA', 'WA', 'GA' )
  OR cs.cs_sales_price  500 )
   AND cs.cs_sold_date_sk = dd.d_date_sk
   AND dd.d_qoy = 1
   AND dd.d_year = 1998
GROUP  BY ca.ca_zip
ORDER  BY ca.ca_zip
LIMIT 100;
{code}

Query runs out of memory, but leaves thread behind even though it is reported 
as FAILED (expected result)

Snippet from jstack:
{code}
2a2451ec-09d8-9f26-e856-5fd349ae72fd:frag:4:0 daemon prio=10 
tid=0x7f507414 nid=0x3000 waiting on condition [0x7f5055b66000]
   java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for  0xc012b038 (a 
java.util.concurrent.Semaphore$NonfairSync)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:994)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303)
at java.util.concurrent.Semaphore.acquire(Semaphore.java:472)
at 
org.apache.drill.exec.ops.SendingAccountor.waitForSendComplete(SendingAccountor.java:48)
- locked 0xc012b068 (a 
org.apache.drill.exec.ops.SendingAccountor)
at 
org.apache.drill.exec.ops.FragmentContext.waitForSendComplete(FragmentContext.java:436)
at 
org.apache.drill.exec.physical.impl.BaseRootExec.close(BaseRootExec.java:112)
at 
org.apache.drill.exec.work.fragment.FragmentExecutor.closeOutResources(FragmentExecutor.java:341)
at 
org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:173)
at 
org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:292)
at 
org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
{code}

NPE in drillbit.log:
{code}
2015-08-24 23:52:04,486 [BitServer-5] ERROR o.a.d.exec.rpc.RpcExceptionHandler 
- Exception in RPC communication.  Connection: /10.10.88.133:31012 -- 
/10.10.88.133:52417 (data server).  Closing connection.
io.netty.handler.codec.DecoderException: java.lang.NullPointerException
at 
io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:99)
 [netty-codec-4.0.27.Final.jar:4.0.27.Final]
at 
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
 [netty-transport-4.0.27.Final.jar:4.0.27.Final]
at 
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
 [netty-transport-4.0.27.Final.jar:4.0.27.Final]
at 
io.netty.handler.timeout.ReadTimeoutHandler.channelRead(ReadTimeoutHandler.java:150)
 [netty-handler-4.0.27.Final.jar:4.0.27.Final]
at 
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
 [netty-transport-4.0.27.Final.jar:4.0.27.Final]
at 
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
 [netty-transport-4.0.27.Final.jar:4.0.27.Final]
at 
io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
 [netty-codec-4.0.27.Final.jar:4.0.27.Final]
at 
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
 [netty-transport-4.0.27.Final.jar:4.0.27.Final]
at

[jira] [Commented] (DRILL-3493) Alter session command with an option to cancel query in a particular phase of execution

2015-08-24 Thread Chris Westin (JIRA)


[ 
https://issues.apache.org/jira/browse/DRILL-3493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14710313#comment-14710313
 ] 

Chris Westin commented on DRILL-3493:
-

[~adeneche], because of the EventProcessor, I think what you're proposing will 
have the same issue I described. We can talk it over outside this ticket before 
you proceed.

 Alter session command with an option to cancel query in a particular phase of 
 execution
 ---

 Key: DRILL-3493
 URL: https://issues.apache.org/jira/browse/DRILL-3493
 Project: Apache Drill
  Issue Type: New Feature
  Components: Execution - Flow
Reporter: Victoria Markman
Assignee: Deneche A. Hakim
Priority: Blocker
 Fix For: 1.2.0


 This proposal is to implement: 
 alter session cancel execution phase = true|false.
 where execution phase is an entry point for query cancellation, like say 
 sort or disk based sort , etc.
 When this option is set, if query reaches this point, forman would issue 
 cancel.
 This way: we can easily test correctness of query cancellation in different 
 types of queries.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (DRILL-3704) Document precedence of operators in Drill

2015-08-24 Thread Abhishek Girish (JIRA)


 [ 
https://issues.apache.org/jira/browse/DRILL-3704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Abhishek Girish updated DRILL-3704:
---
Description: 
I couldn't find documentation of precedence of SQL operators in Apache Drill. 
If they aren't added already, we should include them similar to

(1) https://docs.oracle.com/cd/E17952_01/refman-5.1-en/operator-precedence.html
(2) https://msdn.microsoft.com/en-us/library/ms190276.aspx
(3) https://dev.mysql.com/doc/refman/5.0/en/operator-precedence.html


  was:
I couldn't find documentation of SQL operators in Apache Drill. If they aren't 
added already, we should include them similar to

(1) https://docs.oracle.com/cd/E17952_01/refman-5.1-en/operator-precedence.html
(2) https://msdn.microsoft.com/en-us/library/ms190276.aspx
(3) https://dev.mysql.com/doc/refman/5.0/en/operator-precedence.html



 Document precedence of operators in Drill
 -

 Key: DRILL-3704
 URL: https://issues.apache.org/jira/browse/DRILL-3704
 Project: Apache Drill
  Issue Type: Improvement
  Components: Documentation
Reporter: Abhishek Girish
Assignee: Kristine Hahn

 I couldn't find documentation of precedence of SQL operators in Apache Drill. 
 If they aren't added already, we should include them similar to
 (1) 
 https://docs.oracle.com/cd/E17952_01/refman-5.1-en/operator-precedence.html
 (2) https://msdn.microsoft.com/en-us/library/ms190276.aspx
 (3) https://dev.mysql.com/doc/refman/5.0/en/operator-precedence.html



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (DRILL-3704) Document precedence of operators in Drill

2015-08-24 Thread Abhishek Girish (JIRA)

Abhishek Girish created DRILL-3704:
--

 Summary: Document precedence of operators in Drill
 Key: DRILL-3704
 URL: https://issues.apache.org/jira/browse/DRILL-3704
 Project: Apache Drill
  Issue Type: Improvement
  Components: Documentation
Reporter: Abhishek Girish
Assignee: Kristine Hahn


I couldn't find documentation of SQL operators in Apache Drill. If they aren't 
added already, we should include them similar to

(1) https://docs.oracle.com/cd/E17952_01/refman-5.1-en/operator-precedence.html
(2) https://msdn.microsoft.com/en-us/library/ms190276.aspx
(3) https://dev.mysql.com/doc/refman/5.0/en/operator-precedence.html




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-3493) Alter session command with an option to cancel query in a particular phase of execution

2015-08-24 Thread Chris Westin (JIRA)

[
https://issues.apache.org/jira/browse/DRILL-3493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14710253#comment-14710253
]

Chris Westin commented on DRILL-3493:
-

I talked this over with [~sudheeshkatkam], and we don't think it will be useful
if we implement this exactly as described. This is because of how the
exception/pause injection code (which this would use) works.

Specifically, delivery of events/messages to FragmentExecutors (under which all
this work takes place) is serialized. That severely reduces or eliminates the
utility of this, in this form. That's because when the execution phase is
detected, the cancellation message would be sent. But that happens under an
existing RPC message handler. Because of the serialization of message delivery
to the executor (for any particular query), the cancellation message won't be
delivered until the current message handler finishes. In other words, the
cancellation won't take place until whatever is currently executing finishes,
which is too late to be useful (in a single-node case, the query could be over).

We need some more time to think about a way this can be done usefully, which
might require support from the test framework itself.

Alter session command with an option to cancel query in a particular phase of
execution
---

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-3580) wrong plan for window function queries containing function(col1 + colb)

2015-08-24 Thread Sean Hsuan-Yi Chu (JIRA)


[ 
https://issues.apache.org/jira/browse/DRILL-3580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14710416#comment-14710416
 ] 

Sean Hsuan-Yi Chu commented on DRILL-3580:
--

[~amansinha100] I have an implementation which does not change the topological 
sort.
Can you help review?
https://github.com/hsuanyi/incubator-calcite/tree/CALCITE-841NEW

 wrong plan for window function queries containing function(col1 + colb)
 ---

 Key: DRILL-3580
 URL: https://issues.apache.org/jira/browse/DRILL-3580
 Project: Apache Drill
  Issue Type: Bug
  Components: Query Planning  Optimization
Affects Versions: 1.1.0
Reporter: Deneche A. Hakim
Assignee: Sean Hsuan-Yi Chu
Priority: Critical
  Labels: window_function
 Fix For: 1.2.0


 The following query has a wrong plan:
 {noformat}
 explain plan for select position_id, salary, sum(salary) over (partition by 
 position_id), sum(position_id + salary) over (partition by position_id) from 
 cp.`employee.json` limit 20;
 +--+--+
 | text | json |
 +--+--+
 | 00-00Screen
 00-01  ProjectAllowDup(position_id=[$0], salary=[$1], EXPR$2=[$2], 
 EXPR$3=[$3])
 00-02SelectionVectorRemover
 00-03  Limit(fetch=[20])
 00-04Project(position_id=[$0], salary=[$1], w0$o0=[$2], 
 w0$o00=[$4])
 00-05  Window(window#0=[window(partition {0} order by [] range 
 between UNBOUNDED PRECEDING and UNBOUNDED FOLLOWING aggs [SUM($3)])])
 00-06Project(position_id=[$1], salary=[$2], w0$o0=[$3], 
 $3=[+($1, $2)])
 00-07  Window(window#0=[window(partition {1} order by [] 
 range between UNBOUNDED PRECEDING and UNBOUNDED FOLLOWING aggs [SUM($2)])])
 00-08SelectionVectorRemover
 00-09  Sort(sort0=[$1], dir0=[ASC])
 00-10Project(T13¦¦*=[$0], position_id=[$1], 
 salary=[$2])
 00-11  Scan(groupscan=[EasyGroupScan 
 [selectionRoot=classpath:/employee.json, numFiles=1, columns=[`*`], 
 files=[classpath:/employee.json]]])
 {noformat}
 The plan contains 2 window operators which shouldn't be possible according to 
 DRILL-3196. 
 The results are also incorrect.
 Depending on which aggregation or window function used we get wrong results 
 or an IndexOutOfBounds exception



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-3596) Allow only (expression) or (expression, 1) for LEAD and LAG window functions as input parameters


[ 
https://issues.apache.org/jira/browse/DRILL-3596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14710157#comment-14710157
 ] 

Deneche A. Hakim commented on DRILL-3596:
-

[~amansinha100] can you please review ? thx

 Allow only (expression) or (expression, 1) for LEAD and LAG window 
 functions as input parameters
 

 Key: DRILL-3596
 URL: https://issues.apache.org/jira/browse/DRILL-3596
 Project: Apache Drill
  Issue Type: Bug
  Components: Execution - Relational Operators
Affects Versions: 1.2.0
Reporter: Khurram Faraaz
Assignee: Aman Sinha
  Labels: window_function
 Fix For: 1.2.0


 We need to disable the third input parameter in LEAD and LAG window 
 functions. We should support this syntax.
 LEAD(expression) ,or LEAD(expression,1)
 LAG(expression), or LAG(expression,1)
 In both the functions above, the second parameter is the default offset 
 value, use of any other value must he handled and reported to user as error.
 Use of a third input parameter to both the functions must be disallowed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (DRILL-3703) TopLevelAllocator has a public static mutable field set as a side effect of its Constructor

2015-08-24 Thread Jason Altekruse (JIRA)


 [ 
https://issues.apache.org/jira/browse/DRILL-3703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Altekruse updated DRILL-3703:
---
Assignee: Sudheesh Katkam  (was: Chris Westin)

 TopLevelAllocator has a public static mutable field set as a side effect of 
 its Constructor
 ---

 Key: DRILL-3703
 URL: https://issues.apache.org/jira/browse/DRILL-3703
 Project: Apache Drill
  Issue Type: Bug
  Components: Execution - Flow
Reporter: Julien Le Dem
Assignee: Sudheesh Katkam

 https://github.com/apache/drill/blob/c5f1164351fc4b93ba2ef814ad7ab28a80e2/exec/java-exec/src/main/java/org/apache/drill/exec/memory/TopLevelAllocator.java#L44
 {noformat}   public static long MAXIMUM_DIRECT_MEMORY;{noformat}
 https://github.com/apache/drill/blob/c5f1164351fc4b93ba2ef814ad7ab28a80e2/exec/java-exec/src/main/java/org/apache/drill/exec/memory/TopLevelAllocator.java#L65
 {noformat}
 private TopLevelAllocator(DrillConfig config, long maximumAllocation, boolean 
 errorOnLeak){
 MAXIMUM_DIRECT_MEMORY = maximumAllocation;
 ...
 }
 {noformat}
 It seems to be used only in MemoryIterator (memory system table):
 https://github.com/apache/drill/blob/c5f1164351fc4b93ba2ef814ad7ab28a80e2/exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/MemoryIterator.java#L66
 Which should not need a static variable to access this information.
 https://github.com/apache/drill/search?utf8=%E2%9C%93q=MAXIMUM_DIRECT_MEMORY



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-3703) TopLevelAllocator has a public static mutable field set as a side effect of its Constructor

2015-08-24 Thread Jason Altekruse (JIRA)


[ 
https://issues.apache.org/jira/browse/DRILL-3703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14710166#comment-14710166
 ] 

Jason Altekruse commented on DRILL-3703:


[~sudheeshkatkam] Can you try to refactor this? Julian has a good point about 
why this is not the right way to do this.

 TopLevelAllocator has a public static mutable field set as a side effect of 
 its Constructor
 ---

 Key: DRILL-3703
 URL: https://issues.apache.org/jira/browse/DRILL-3703
 Project: Apache Drill
  Issue Type: Bug
  Components: Execution - Flow
Reporter: Julien Le Dem
Assignee: Sudheesh Katkam

 https://github.com/apache/drill/blob/c5f1164351fc4b93ba2ef814ad7ab28a80e2/exec/java-exec/src/main/java/org/apache/drill/exec/memory/TopLevelAllocator.java#L44
 {noformat}   public static long MAXIMUM_DIRECT_MEMORY;{noformat}
 https://github.com/apache/drill/blob/c5f1164351fc4b93ba2ef814ad7ab28a80e2/exec/java-exec/src/main/java/org/apache/drill/exec/memory/TopLevelAllocator.java#L65
 {noformat}
 private TopLevelAllocator(DrillConfig config, long maximumAllocation, boolean 
 errorOnLeak){
 MAXIMUM_DIRECT_MEMORY = maximumAllocation;
 ...
 }
 {noformat}
 It seems to be used only in MemoryIterator (memory system table):
 https://github.com/apache/drill/blob/c5f1164351fc4b93ba2ef814ad7ab28a80e2/exec/java-exec/src/main/java/org/apache/drill/exec/store/sys/MemoryIterator.java#L66
 Which should not need a static variable to access this information.
 https://github.com/apache/drill/search?utf8=%E2%9C%93q=MAXIMUM_DIRECT_MEMORY



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (DRILL-3705) Query runs out of memory, reported as FAILED and leaves thread running


 [ 
https://issues.apache.org/jira/browse/DRILL-3705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Victoria Markman updated DRILL-3705:

Attachment: jstack.txt
2a2451ec-09d8-9f26-e856-5fd349ae72fd.sys.drill
drillbit.log

 Query runs out of memory, reported as FAILED and leaves thread running 
 ---

 Key: DRILL-3705
 URL: https://issues.apache.org/jira/browse/DRILL-3705
 Project: Apache Drill
  Issue Type: Bug
  Components: Execution - Flow
Affects Versions: 1.2.0
Reporter: Victoria Markman
Assignee: Chris Westin
 Attachments: 2a2451ec-09d8-9f26-e856-5fd349ae72fd.sys.drill, 
 drillbit.log, jstack.txt


 Single node drill installation
 DRILL_MAX_DIRECT_MEMORY=2G
 DRILL_HEAP=1G
 Execute tpcds query 15 SF100 (parquet) with the settings above. Reproduces 2 
 out of 3 times.
 {code}
 SELECT ca.ca_zip,
Sum(cs.cs_sales_price)
 FROM   catalog_salescs,
customer c,
customer_address ca,
date_dim dd
 WHERE  cs.cs_bill_customer_sk = c.c_customer_sk
AND c.c_current_addr_sk = ca.ca_address_sk
AND ( Substr(ca.ca_zip, 1, 5) IN ( '85669', '86197', '88274', '83405',
'86475', '85392', '85460', '80348',
'81792' )
   OR ca.ca_state IN ( 'CA', 'WA', 'GA' )
   OR cs.cs_sales_price  500 )
AND cs.cs_sold_date_sk = dd.d_date_sk
AND dd.d_qoy = 1
AND dd.d_year = 1998
 GROUP  BY ca.ca_zip
 ORDER  BY ca.ca_zip
 LIMIT 100;
 {code}
 Query runs out of memory, but leaves thread behind even though it is reported 
 as FAILED (expected result)
 Snippet from jstack:
 {code}
 2a2451ec-09d8-9f26-e856-5fd349ae72fd:frag:4:0 daemon prio=10 
 tid=0x7f507414 nid=0x3000 waiting on condition [0x7f5055b66000]
java.lang.Thread.State: WAITING (parking)
 at sun.misc.Unsafe.park(Native Method)
 - parking to wait for  0xc012b038 (a 
 java.util.concurrent.Semaphore$NonfairSync)
 at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
 at 
 java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
 at 
 java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:994)
 at 
 java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303)
 at java.util.concurrent.Semaphore.acquire(Semaphore.java:472)
 at 
 org.apache.drill.exec.ops.SendingAccountor.waitForSendComplete(SendingAccountor.java:48)
 - locked 0xc012b068 (a 
 org.apache.drill.exec.ops.SendingAccountor)
 at 
 org.apache.drill.exec.ops.FragmentContext.waitForSendComplete(FragmentContext.java:436)
 at 
 org.apache.drill.exec.physical.impl.BaseRootExec.close(BaseRootExec.java:112)
 at 
 org.apache.drill.exec.work.fragment.FragmentExecutor.closeOutResources(FragmentExecutor.java:341)
 at 
 org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:173)
 at 
 org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:292)
 at 
 org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
 at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
 at java.lang.Thread.run(Thread.java:745)
 {code}
 NPE in drillbit.log:
 {code}
 2015-08-24 23:52:04,486 [BitServer-5] ERROR 
 o.a.d.exec.rpc.RpcExceptionHandler - Exception in RPC communication.  
 Connection: /10.10.88.133:31012 -- /10.10.88.133:52417 (data server).  
 Closing connection.
 io.netty.handler.codec.DecoderException: java.lang.NullPointerException
 at 
 io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:99)
  [netty-codec-4.0.27.Final.jar:4.0.27.Final]
 at 
 io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
  [netty-transport-4.0.27.Final.jar:4.0.27.Final]
 at 
 io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
  [netty-transport-4.0.27.Final.jar:4.0.27.Final]
 at 
 io.netty.handler.timeout.ReadTimeoutHandler.channelRead(ReadTimeoutHandler.java:150)
  [netty-handler-4.0.27.Final.jar:4.0.27.Final]
 at 
 io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
  [netty-transport-4.0.27.Final.jar:4.0.27.Final]
 at

[jira] [Assigned] (DRILL-3654) FIRST_VALUE(char-column/varchar-column) returns IOB Exception


 [ 
https://issues.apache.org/jira/browse/DRILL-3654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deneche A. Hakim reassigned DRILL-3654:
---

Assignee: Deneche A. Hakim  (was: Aman Sinha)

 FIRST_VALUE(char-column/varchar-column) returns IOB Exception
 -

 Key: DRILL-3654
 URL: https://issues.apache.org/jira/browse/DRILL-3654
 Project: Apache Drill
  Issue Type: Bug
  Components: Execution - Relational Operators
Affects Versions: 1.2.0
 Environment: private-branch 
 https://github.com/adeneche/incubator-drill/tree/new-window-funcs
Reporter: Khurram Faraaz
Assignee: Deneche A. Hakim
Priority: Critical
  Labels: window_function
 Fix For: 1.2.0


 FIRST_VALUE(char-column/varchar-column) returns IOB Exception on 
 developers private branch.
 {code}
 0: jdbc:drill:schema=dfs.tmp select first_value(col8) over(partition by col7 
 order by col8) first_value_col8 from FEWRWSPQQ_101;
 java.lang.RuntimeException: java.sql.SQLException: SYSTEM ERROR: 
 IndexOutOfBoundsException: index: 0, length: 4 (expected: range(0, 0))
 Fragment 0:0
 [Error Id: a4799155-ba8a-4117-9381-45ec10e02aa6 on centos-04.qa.lab:31010]
   at sqlline.IncrementalRows.hasNext(IncrementalRows.java:73)
   at 
 sqlline.TableOutputFormat$ResizingRowsProvider.next(TableOutputFormat.java:87)
   at sqlline.TableOutputFormat.print(TableOutputFormat.java:118)
   at sqlline.SqlLine.print(SqlLine.java:1583)
   at sqlline.Commands.execute(Commands.java:852)
   at sqlline.Commands.sql(Commands.java:751)
   at sqlline.SqlLine.dispatch(SqlLine.java:738)
   at sqlline.SqlLine.begin(SqlLine.java:612)
   at sqlline.SqlLine.start(SqlLine.java:366)
   at sqlline.SqlLine.main(SqlLine.java:259)
 0: jdbc:drill:schema=dfs.tmp select first_value(col9) over(partition by col7 
 order by col9) first_value_col8 from FEWRWSPQQ_101;
 java.lang.RuntimeException: java.sql.SQLException: SYSTEM ERROR: 
 IndexOutOfBoundsException: index: 0, length: 4 (expected: range(0, 0))
 Fragment 0:0
 [Error Id: 30bcaf2b-98ce-4ec5-a9aa-4fa8fa1f3403 on centos-04.qa.lab:31010]
   at sqlline.IncrementalRows.hasNext(IncrementalRows.java:73)
   at 
 sqlline.TableOutputFormat$ResizingRowsProvider.next(TableOutputFormat.java:87)
   at sqlline.TableOutputFormat.print(TableOutputFormat.java:118)
   at sqlline.SqlLine.print(SqlLine.java:1583)
   at sqlline.Commands.execute(Commands.java:852)
   at sqlline.Commands.sql(Commands.java:751)
   at sqlline.SqlLine.dispatch(SqlLine.java:738)
   at sqlline.SqlLine.begin(SqlLine.java:612)
   at sqlline.SqlLine.start(SqlLine.java:366)
   at sqlline.SqlLine.main(SqlLine.java:259)
 0: jdbc:drill:schema=dfs.tmp 
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (DRILL-3643) NTILE(0) returns RuntimeException