[jira] [Commented] (DRILL-4134) Incorporate remaining patches from DRILL-1942 Allocator refactor

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15068223#comment-15068223
 ] 

ASF GitHub Bot commented on DRILL-4134:
---

Github user jacques-n commented on the pull request:

https://github.com/apache/drill/pull/283#issuecomment-166640458
  
Rebased and merged in 809f4620d7d82c72240212de13b993049550959d


> Incorporate remaining patches from DRILL-1942 Allocator refactor
> 
>
> Key: DRILL-4134
> URL: https://issues.apache.org/jira/browse/DRILL-4134
> Project: Apache Drill
>  Issue Type: Sub-task
>  Components: Execution - Flow
>Reporter: Jacques Nadeau
>Assignee: Jacques Nadeau
> Fix For: 1.5.0
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-4134) Incorporate remaining patches from DRILL-1942 Allocator refactor

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15068224#comment-15068224
 ] 

ASF GitHub Bot commented on DRILL-4134:
---

Github user jacques-n closed the pull request at:

https://github.com/apache/drill/pull/283


> Incorporate remaining patches from DRILL-1942 Allocator refactor
> 
>
> Key: DRILL-4134
> URL: https://issues.apache.org/jira/browse/DRILL-4134
> Project: Apache Drill
>  Issue Type: Sub-task
>  Components: Execution - Flow
>Reporter: Jacques Nadeau
>Assignee: Jacques Nadeau
> Fix For: 1.5.0
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-4020) The not-equal operator returns incorrect results when used on the HBase row key

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15068221#comment-15068221
 ] 

ASF GitHub Bot commented on DRILL-4020:
---

Github user jacques-n commented on the pull request:

https://github.com/apache/drill/pull/238#issuecomment-166640349
  
Shoot, I mistyped my close message. Should have been 283 instead of 238. 
@nagix, can you reopen this pull request? 


> The not-equal operator returns incorrect results when used on the HBase row 
> key
> ---
>
> Key: DRILL-4020
> URL: https://issues.apache.org/jira/browse/DRILL-4020
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Storage - HBase
>Affects Versions: 1.2.0
> Environment: Drill Sandbox
>Reporter: Akihiko Kusanagi
>Priority: Critical
> Attachments: DRILL-4020.1.patch.txt
>
>
> Create a test HBase table:
> {noformat}
> hbase> create 'table', 'f'
> hbase> put 'table', 'row1', 'f:c', 'value1'
> hbase> put 'table', 'row2', 'f:c', 'value2'
> hbase> put 'table', 'row3', 'f:c', 'value3'
> {noformat}
> The table looks like this:
> {noformat}
> 0: jdbc:drill:zk=maprdemo:5181> SELECT CONVERT_FROM(row_key, 'UTF8') FROM 
> hbase.`table`;
> +-+
> | EXPR$0  |
> +-+
> | row1|
> | row2|
> | row3|
> +-+
> 1 row selected (4.596 seconds)
> {noformat}
> However, this query returns incorrect results when a not-equal operator is 
> used on the row key:
> {noformat}
> 0: jdbc:drill:zk=maprdemo:5181> SELECT CONVERT_FROM(row_key, 'UTF8') FROM 
> hbase.`table` WHERE row_key <> 'row1';
> +-+
> | EXPR$0  |
> +-+
> | row1|
> | row2|
> | row3|
> +-+
> 1 row selected (0.573 seconds)
> {noformat}
> In the query plan, there is no RowFilter:
> {noformat}
> 00-00Screen
> 00-01  Project(EXPR$0=[CONVERT_FROMUTF8($0)])
> 00-02Scan(groupscan=[HBaseGroupScan [HBaseScanSpec=HBaseScanSpec 
> [tableName=table, startRow=, stopRow=, filter=null], columns=[`row_key`]]])
> {noformat}
> When the query has multiple not-equal operators, it works fine:
> {noformat}
> 0: jdbc:drill:zk=maprdemo:5181> SELECT CONVERT_FROM(row_key, 'UTF8') FROM 
> hbase.`table` WHERE row_key <> 'row1' AND row_key <> 'row2';
> +-+
> | EXPR$0  |
> +-+
> | row3|
> +-+
> 1 row selected (0.255 seconds)
> {noformat}
> In the query plan, a FilterList has two RowFilters with NOT_EQUAL operators:
> {noformat}
> 00-00Screen
> 00-01  Project(EXPR$0=[CONVERT_FROMUTF8($0)])
> 00-02Scan(groupscan=[HBaseGroupScan [HBaseScanSpec=HBaseScanSpec 
> [tableName=table, startRow=, stopRow=, filter=FilterList AND (2/2): 
> [RowFilter (NOT_EQUAL, row1), RowFilter (NOT_EQUAL, row2)]], 
> columns=[`row_key`]]])
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-4201) DrillPushFilterPastProject should allow partial filter pushdown.

[Jinfeng Ni (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15068717#comment-15068717
 ] 

Jinfeng Ni commented on DRILL-4201:
---

Fixed in commit:  1ea3d6c3f144614caf460648c1c27c6d0f5b06b8


> DrillPushFilterPastProject should allow partial filter pushdown. 
> -
>
> Key: DRILL-4201
> URL: https://issues.apache.org/jira/browse/DRILL-4201
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Query Planning & Optimization
>Reporter: Jinfeng Ni
>Assignee: Jinfeng Ni
> Fix For: 1.5.0
>
>
> Currently, DrillPushFilterPastProjectRule will stop pushing the filter down, 
> if the filter itself has ITEM or FLATTEN function, or its input reference is 
> referring to an ITEM or FLATTEN function. However, in case that the filter is 
> a conjunction of multiple sub-filters, some of them refer to ITEM  or FLATTEN 
> but the other not, then we should allow partial filter to be pushed down. For 
> instance,
> WHERE  partition_col > 10 and flatten_output_col = 'ABC'. 
> The "flatten_output_col" comes from the output of FLATTEN operator, and 
> therefore flatten_output_col = 'ABC' should not pushed past the project. But 
> partiion_col > 10 should be pushed down, such that we could trigger the 
> pruning rule to apply partition pruning.
> It would be improve Drill query performance, when the partially pushed filter 
> leads to partition pruning, or the partially pushed filter results in early 
> filtering in upstream operator. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-4201) DrillPushFilterPastProject should allow partial filter pushdown.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15068716#comment-15068716
 ] 

ASF GitHub Bot commented on DRILL-4201:
---

Github user asfgit closed the pull request at:

https://github.com/apache/drill/pull/305


> DrillPushFilterPastProject should allow partial filter pushdown. 
> -
>
> Key: DRILL-4201
> URL: https://issues.apache.org/jira/browse/DRILL-4201
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Query Planning & Optimization
>Reporter: Jinfeng Ni
>Assignee: Jinfeng Ni
> Fix For: 1.5.0
>
>
> Currently, DrillPushFilterPastProjectRule will stop pushing the filter down, 
> if the filter itself has ITEM or FLATTEN function, or its input reference is 
> referring to an ITEM or FLATTEN function. However, in case that the filter is 
> a conjunction of multiple sub-filters, some of them refer to ITEM  or FLATTEN 
> but the other not, then we should allow partial filter to be pushed down. For 
> instance,
> WHERE  partition_col > 10 and flatten_output_col = 'ABC'. 
> The "flatten_output_col" comes from the output of FLATTEN operator, and 
> therefore flatten_output_col = 'ABC' should not pushed past the project. But 
> partiion_col > 10 should be pushed down, such that we could trigger the 
> pruning rule to apply partition pruning.
> It would be improve Drill query performance, when the partially pushed filter 
> leads to partition pruning, or the partially pushed filter results in early 
> filtering in upstream operator. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-4187) Introduce a state to separate queries pending execution from those pending in the queue.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15068739#comment-15068739
 ] 

ASF GitHub Bot commented on DRILL-4187:
---

GitHub user hnfgns opened a pull request:

https://github.com/apache/drill/pull/310

DRILL-4187: introduce a new query state ENQUEUED and rename the state 
PENDING to STARTING



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/hnfgns/incubator-drill DRILL-4187

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/drill/pull/310.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #310


commit f108fcf65078d3173c0c4433494ca0d3f4f85de2
Author: Hanifi Gunes 
Date:   2015-12-16T01:24:27Z

DRILL-4187: introduce a new query state ENQUEUED and rename the state 
PENDING to STARTING




> Introduce a state to separate queries pending execution from those pending in 
> the queue.
> 
>
> Key: DRILL-4187
> URL: https://issues.apache.org/jira/browse/DRILL-4187
> Project: Apache Drill
>  Issue Type: Sub-task
>Reporter: Hanifi Gunes
>Assignee: Mehant Baid
> Fix For: 1.5.0
>
>
> Currently queries pending in the queue are not listed in the web UI besides 
> we use the state PENDING to mean pending executions. This issue proposes i) 
> to list enqueued queries in the web UI ii) to introduce a new state for 
> queries sitting at the queue, differentiating then from those pending 
> execution.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-4035) NPE seen on Functional test run using JDK 8

[Deneche A. Hakim (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15068524#comment-15068524
 ] 

Deneche A. Hakim commented on DRILL-4035:
-

[~khfaraaz] did some more tests on the latest master with JDK 8 and assertions 
disabled and we are no longer seeing the NPE

> NPE seen on Functional test run using JDK 8
> ---
>
> Key: DRILL-4035
> URL: https://issues.apache.org/jira/browse/DRILL-4035
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Execution - Flow
>Affects Versions: 1.3.0
> Environment: 4 node cluster CentOS
>Reporter: Khurram Faraaz
>Assignee: Deneche A. Hakim
>Priority: Critical
> Fix For: 1.5.0
>
>
> I am seeing an NPE in the Functional test run using JDK8 and Drill 1.3
> Failing test is : 
> Functional/partition_pruning/hive/parquet/dynamic_hier_intint/data/parquetCount1.q
> select count(*) from 
> hive.dynamic_partitions.lineitem_parquet_partitioned_hive_hier_intint;
> {code}
> Drill version was, git.commit.id=e4b94a78
> root@centos drill-1.3.0]# java -version
> openjdk version "1.8.0_65"
> OpenJDK Runtime Environment (build 1.8.0_65-b17)
> OpenJDK 64-Bit Server VM (build 25.65-b01, mixed mode)
> [root@centos drill-1.3.0]# javac -version
> javac 1.8.0_65
> {code}
> {code}
> 2015-11-05 01:37:45 INFO  DrillTestJdbc:76 - running test 
> /root/public_framework/drill-test-framework/framework/resources/Functional/window_functions/last_val/lastValFn_9.q
>  981260622
> 2015-11-05 01:37:45 INFO  DrillResultSetImpl$ResultsListener:1470 - [#137] 
> Query failed:
> oadd.org.apache.drill.common.exceptions.UserRemoteException: SYSTEM ERROR: 
> NullPointerException
> Fragment 0:0
> [Error Id: cefc7238-a646-4f9a-b4f2-0bd102efe393 on centos-01.qa.lab:31010]
> at 
> oadd.org.apache.drill.exec.rpc.user.QueryResultHandler.resultArrived(QueryResultHandler.java:118)
> at 
> oadd.org.apache.drill.exec.rpc.user.UserClient.handleReponse(UserClient.java:110)
> at 
> oadd.org.apache.drill.exec.rpc.BasicClientWithConnection.handle(BasicClientWithConnection.java:47)
> at 
> oadd.org.apache.drill.exec.rpc.BasicClientWithConnection.handle(BasicClientWithConnection.java:32)
> at oadd.org.apache.drill.exec.rpc.RpcBus.handle(RpcBus.java:61)
> at 
> oadd.org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:233)
> at 
> oadd.org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:205)
> at 
> oadd.io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:89)
> at 
> oadd.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
> at 
> oadd.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
> at 
> oadd.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:254)
> at 
> oadd.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
> at 
> oadd.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
> at 
> oadd.io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
> at 
> oadd.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
> at 
> oadd.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
> at 
> oadd.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:242)
> at 
> oadd.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
> at 
> oadd.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
> at 
> oadd.io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86)
> at 
> oadd.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
> at 
> oadd.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
> at 
> oadd.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:847)
> at 
> oadd.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:131)
> at 
> oadd.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511)
> at 
> oadd.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)
> at 
> oadd.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)
> 

[jira] [Created] (DRILL-4220) Drill not pushing down join with sql server

[nidhi (JIRA)]

nidhi created DRILL-4220:


 Summary: Drill not pushing down join with sql server 
 Key: DRILL-4220
 URL: https://issues.apache.org/jira/browse/DRILL-4220
 Project: Apache Drill
  Issue Type: Bug
  Components: Client - JDBC
Reporter: nidhi


Im using drill to query sql server 
Drill documentation says it "has powerful pushdown capabilities with RDBMS 
sources" but when I try to join 2 tables which are in separate DB on same sql 
server , drill doesn't push down that join to sql server 
Its trying to pull full table in and then join in drill 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-4187) Introduce a state to separate queries pending execution from those pending in the queue.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15068948#comment-15068948
 ] 

ASF GitHub Bot commented on DRILL-4187:
---

Github user asfgit closed the pull request at:

https://github.com/apache/drill/pull/310


> Introduce a state to separate queries pending execution from those pending in 
> the queue.
> 
>
> Key: DRILL-4187
> URL: https://issues.apache.org/jira/browse/DRILL-4187
> Project: Apache Drill
>  Issue Type: Sub-task
>Reporter: Hanifi Gunes
>Assignee: Mehant Baid
> Fix For: 1.5.0
>
>
> Currently queries pending in the queue are not listed in the web UI besides 
> we use the state PENDING to mean pending executions. This issue proposes i) 
> to list enqueued queries in the web UI ii) to introduce a new state for 
> queries sitting at the queue, differentiating then from those pending 
> execution.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-4187) Introduce a state to separate queries pending execution from those pending in the queue.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15068936#comment-15068936
 ] 

ASF GitHub Bot commented on DRILL-4187:
---

Github user sudheeshkatkam commented on the pull request:

https://github.com/apache/drill/pull/310#issuecomment-166765293
  
Yes. cancellation remains the same, but it would be nice to be able to 
cancel when the state is ENQUEUED (interrupt the thread waiting on the 
semaphore).

+ 1


> Introduce a state to separate queries pending execution from those pending in 
> the queue.
> 
>
> Key: DRILL-4187
> URL: https://issues.apache.org/jira/browse/DRILL-4187
> Project: Apache Drill
>  Issue Type: Sub-task
>Reporter: Hanifi Gunes
>Assignee: Mehant Baid
> Fix For: 1.5.0
>
>
> Currently queries pending in the queue are not listed in the web UI besides 
> we use the state PENDING to mean pending executions. This issue proposes i) 
> to list enqueued queries in the web UI ii) to introduce a new state for 
> queries sitting at the queue, differentiating then from those pending 
> execution.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (DRILL-4187) Introduce a state to separate queries pending execution from those pending in the queue.

[Hanifi Gunes (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-4187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hanifi Gunes reassigned DRILL-4187:
---

Assignee: Hanifi Gunes  (was: Mehant Baid)

> Introduce a state to separate queries pending execution from those pending in 
> the queue.
> 
>
> Key: DRILL-4187
> URL: https://issues.apache.org/jira/browse/DRILL-4187
> Project: Apache Drill
>  Issue Type: Sub-task
>Reporter: Hanifi Gunes
>Assignee: Hanifi Gunes
> Fix For: 1.5.0
>
>
> Currently queries pending in the queue are not listed in the web UI besides 
> we use the state PENDING to mean pending executions. This issue proposes i) 
> to list enqueued queries in the web UI ii) to introduce a new state for 
> queries sitting at the queue, differentiating then from those pending 
> execution.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (DRILL-4219) Im using drill to query data in sql server . killing query in drill doesnt kill query in sql server

[nidhi (JIRA)]

nidhi created DRILL-4219:


 Summary: Im using drill to query data in sql server . killing 
query in drill doesnt kill query in sql server 
 Key: DRILL-4219
 URL: https://issues.apache.org/jira/browse/DRILL-4219
 Project: Apache Drill
  Issue Type: Bug
  Components: Client - JDBC
Affects Versions: 1.3.0
Reporter: nidhi


Im using drill to query data in sql server . killing query in drill doesnt kill 
query in sql server .
The query keeps running on sql server side and needs to be manually killed in 
sql server 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-4216) Aggregate Window Function COUNT() With GROUP BY Clause expected: range(0, 32768)

[PIPELINE (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15069070#comment-15069070
 ] 

PIPELINE commented on DRILL-4216:
-

And 32768 is the MaxValue of Short type in Java.
I have see the source code of Drill somewhere, but found nothing...

> Aggregate Window Function COUNT() With GROUP BY Clause expected: range(0, 
> 32768)
> 
>
> Key: DRILL-4216
> URL: https://issues.apache.org/jira/browse/DRILL-4216
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Functions - Drill
>Affects Versions: 1.3.0
> Environment: Hadoop-2.5.2
> Hbase-0.9.15
> Java-1.7.0_85
>Reporter: PIPELINE
>Assignee: Deneche A. Hakim
>
> *When column is row_key,it work well !*
> 0: jdbc:drill:> select count(row_key) over() from hbase.web_initial_20151222 
> wi group by row_key limit 3;
> +-+
> | EXPR$0  |
> +-+
> | 102906  |
> | 102906  |
> | 102906  |
> +-+
> 3 rows selected (1.645 seconds)
> *When column is Hbase.Talbename.ColumnFamily.Qualify, and count(column) less 
> than 32768,it work well !*
> 0: jdbc:drill:> select count(wi.cf1.q5) over() from 
> hbase.web_initial_20151214 wi group by wi.cf1.q5 limit 3;
> +-+
> | EXPR$0  |
> +-+
> | 10383   |
> | 10383   |
> | 10383   |
> +-+
> 3 rows selected (1.044 seconds)
> {color:red}
> When column is Hbase.Talbename.ColumnFamily.Qualify, and count(column) 
> more than 32768,IndexOutOfBoundsException
> {color}
> 0: jdbc:drill:> select count(wi.cf1.q5) over() from 
> hbase.web_initial_20151222 wi group by wi.cf1.q5 limit 3;
> Error: SYSTEM ERROR: IndexOutOfBoundsException: index: 0, length: 62784 
> (expected: range(0, 32768))
> Fragment 0:0
> [Error Id: 77406a8a-8389-4f1b-af6c-d26d811379b7 on slave4.hadoop:31010] 
> (state=,code=0)
> java.sql.SQLException: SYSTEM ERROR: IndexOutOfBoundsException: index: 0, 
> length: 62784 (expected: range(0, 32768))
> Fragment 0:0
> [Error Id: 77406a8a-8389-4f1b-af6c-d26d811379b7 on slave4.hadoop:31010]
>   at 
> org.apache.drill.jdbc.impl.DrillCursor.nextRowInternally(DrillCursor.java:247)
>   at org.apache.drill.jdbc.impl.DrillCursor.next(DrillCursor.java:320)
>   at 
> net.hydromatic.avatica.AvaticaResultSet.next(AvaticaResultSet.java:187)
>   at 
> org.apache.drill.jdbc.impl.DrillResultSetImpl.next(DrillResultSetImpl.java:160)
>   at sqlline.IncrementalRows.hasNext(IncrementalRows.java:62)
>   at 
> sqlline.TableOutputFormat$ResizingRowsProvider.next(TableOutputFormat.java:87)
>   at sqlline.TableOutputFormat.print(TableOutputFormat.java:118)
>   at sqlline.SqlLine.print(SqlLine.java:1593)
>   at sqlline.Commands.execute(Commands.java:852)
>   at sqlline.Commands.sql(Commands.java:751)
>   at sqlline.SqlLine.dispatch(SqlLine.java:746)
>   at sqlline.SqlLine.begin(SqlLine.java:621)
>   at sqlline.SqlLine.start(SqlLine.java:375)
>   at sqlline.SqlLine.main(SqlLine.java:268)
> Caused by: org.apache.drill.common.exceptions.UserRemoteException: SYSTEM 
> ERROR: IndexOutOfBoundsException: index: 0, length: 62784 (expected: range(0, 
> 32768))
> Fragment 0:0
> [Error Id: 77406a8a-8389-4f1b-af6c-d26d811379b7 on slave4.hadoop:31010]
>   at 
> org.apache.drill.exec.rpc.user.QueryResultHandler.resultArrived(QueryResultHandler.java:118)
>   at 
> org.apache.drill.exec.rpc.user.UserClient.handleReponse(UserClient.java:112)
>   at 
> org.apache.drill.exec.rpc.BasicClientWithConnection.handle(BasicClientWithConnection.java:47)
>   at 
> org.apache.drill.exec.rpc.BasicClientWithConnection.handle(BasicClientWithConnection.java:32)
>   at org.apache.drill.exec.rpc.RpcBus.handle(RpcBus.java:69)
>   at org.apache.drill.exec.rpc.RpcBus$RequestEvent.run(RpcBus.java:400)
>   at 
> org.apache.drill.common.SerializedExecutor$RunnableProcessor.run(SerializedExecutor.java:105)
>   at 
> org.apache.drill.exec.rpc.RpcBus$SameExecutor.execute(RpcBus.java:264)
>   at 
> org.apache.drill.common.SerializedExecutor.execute(SerializedExecutor.java:142)
>   at 
> org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:298)
>   at 
> org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:269)
>   at 
> io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:89)
>   at 
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
>   at 
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
>   at 
> io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:254)
>   at 
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)

[jira] [Commented] (DRILL-4216) Aggregate Window Function COUNT() With GROUP BY Clause expected: range(0, 32768)

[PIPELINE (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15069057#comment-15069057
 ] 

PIPELINE commented on DRILL-4216:
-

I have already setted `exec.errors.verbose`=true,or it just show this(without 
java code Exception):

Error: SYSTEM ERROR: IndexOutOfBoundsException: index: 0, length: 62784 
(expected: range(0, 32768))

Fragment 0:0

[Error Id: 77406a8a-8389-4f1b-af6c-d26d811379b7 on slave4.hadoop:31010] 
(state=,code=0)
java.sql.SQLException: SYSTEM ERROR: IndexOutOfBoundsException: index: 0, 
length: 62784 (expected: range(0, 32768))

Fragment 0:0

[Error Id: 77406a8a-8389-4f1b-af6c-d26d811379b7 on slave4.hadoop:31010]

> Aggregate Window Function COUNT() With GROUP BY Clause expected: range(0, 
> 32768)
> 
>
> Key: DRILL-4216
> URL: https://issues.apache.org/jira/browse/DRILL-4216
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Functions - Drill
>Affects Versions: 1.3.0
> Environment: Hadoop-2.5.2
> Hbase-0.9.15
> Java-1.7.0_85
>Reporter: PIPELINE
>Assignee: Deneche A. Hakim
>
> *When column is row_key,it work well !*
> 0: jdbc:drill:> select count(row_key) over() from hbase.web_initial_20151222 
> wi group by row_key limit 3;
> +-+
> | EXPR$0  |
> +-+
> | 102906  |
> | 102906  |
> | 102906  |
> +-+
> 3 rows selected (1.645 seconds)
> *When column is Hbase.Talbename.ColumnFamily.Qualify, and count(column) less 
> than 32768,it work well !*
> 0: jdbc:drill:> select count(wi.cf1.q5) over() from 
> hbase.web_initial_20151214 wi group by wi.cf1.q5 limit 3;
> +-+
> | EXPR$0  |
> +-+
> | 10383   |
> | 10383   |
> | 10383   |
> +-+
> 3 rows selected (1.044 seconds)
> {color:red}
> When column is Hbase.Talbename.ColumnFamily.Qualify, and count(column) 
> more than 32768,IndexOutOfBoundsException
> {color}
> 0: jdbc:drill:> select count(wi.cf1.q5) over() from 
> hbase.web_initial_20151222 wi group by wi.cf1.q5 limit 3;
> Error: SYSTEM ERROR: IndexOutOfBoundsException: index: 0, length: 62784 
> (expected: range(0, 32768))
> Fragment 0:0
> [Error Id: 77406a8a-8389-4f1b-af6c-d26d811379b7 on slave4.hadoop:31010] 
> (state=,code=0)
> java.sql.SQLException: SYSTEM ERROR: IndexOutOfBoundsException: index: 0, 
> length: 62784 (expected: range(0, 32768))
> Fragment 0:0
> [Error Id: 77406a8a-8389-4f1b-af6c-d26d811379b7 on slave4.hadoop:31010]
>   at 
> org.apache.drill.jdbc.impl.DrillCursor.nextRowInternally(DrillCursor.java:247)
>   at org.apache.drill.jdbc.impl.DrillCursor.next(DrillCursor.java:320)
>   at 
> net.hydromatic.avatica.AvaticaResultSet.next(AvaticaResultSet.java:187)
>   at 
> org.apache.drill.jdbc.impl.DrillResultSetImpl.next(DrillResultSetImpl.java:160)
>   at sqlline.IncrementalRows.hasNext(IncrementalRows.java:62)
>   at 
> sqlline.TableOutputFormat$ResizingRowsProvider.next(TableOutputFormat.java:87)
>   at sqlline.TableOutputFormat.print(TableOutputFormat.java:118)
>   at sqlline.SqlLine.print(SqlLine.java:1593)
>   at sqlline.Commands.execute(Commands.java:852)
>   at sqlline.Commands.sql(Commands.java:751)
>   at sqlline.SqlLine.dispatch(SqlLine.java:746)
>   at sqlline.SqlLine.begin(SqlLine.java:621)
>   at sqlline.SqlLine.start(SqlLine.java:375)
>   at sqlline.SqlLine.main(SqlLine.java:268)
> Caused by: org.apache.drill.common.exceptions.UserRemoteException: SYSTEM 
> ERROR: IndexOutOfBoundsException: index: 0, length: 62784 (expected: range(0, 
> 32768))
> Fragment 0:0
> [Error Id: 77406a8a-8389-4f1b-af6c-d26d811379b7 on slave4.hadoop:31010]
>   at 
> org.apache.drill.exec.rpc.user.QueryResultHandler.resultArrived(QueryResultHandler.java:118)
>   at 
> org.apache.drill.exec.rpc.user.UserClient.handleReponse(UserClient.java:112)
>   at 
> org.apache.drill.exec.rpc.BasicClientWithConnection.handle(BasicClientWithConnection.java:47)
>   at 
> org.apache.drill.exec.rpc.BasicClientWithConnection.handle(BasicClientWithConnection.java:32)
>   at org.apache.drill.exec.rpc.RpcBus.handle(RpcBus.java:69)
>   at org.apache.drill.exec.rpc.RpcBus$RequestEvent.run(RpcBus.java:400)
>   at 
> org.apache.drill.common.SerializedExecutor$RunnableProcessor.run(SerializedExecutor.java:105)
>   at 
> org.apache.drill.exec.rpc.RpcBus$SameExecutor.execute(RpcBus.java:264)
>   at 
> org.apache.drill.common.SerializedExecutor.execute(SerializedExecutor.java:142)
>   at 
> org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:298)
>   at 
> org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:269)
>   at 
> io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:89)
>   at 
> 

[jira] [Commented] (DRILL-4216) Aggregate Window Function COUNT() With GROUP BY Clause expected: range(0, 32768)

[PIPELINE (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15069067#comment-15069067
 ] 

PIPELINE commented on DRILL-4216:
-

Year,in this query,I just wanna count how many groups.
And the complete statement may like this:
var page=1
var pageSize=10

"select count(qualify) over() as total,count() as `count`,... from tablename 
group by qualify order by `count` limit pageSize offset (page-1)*pageSize rows"

then can get the totalPage=total/pageSize, 
and in the next query,if((page-1)*pageSize+pageSize > totalPage) then do 
nothing.

The "total" in Hbase change with time,and I do not wanna use two queries.


> Aggregate Window Function COUNT() With GROUP BY Clause expected: range(0, 
> 32768)
> 
>
> Key: DRILL-4216
> URL: https://issues.apache.org/jira/browse/DRILL-4216
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Functions - Drill
>Affects Versions: 1.3.0
> Environment: Hadoop-2.5.2
> Hbase-0.9.15
> Java-1.7.0_85
>Reporter: PIPELINE
>Assignee: Deneche A. Hakim
>
> *When column is row_key,it work well !*
> 0: jdbc:drill:> select count(row_key) over() from hbase.web_initial_20151222 
> wi group by row_key limit 3;
> +-+
> | EXPR$0  |
> +-+
> | 102906  |
> | 102906  |
> | 102906  |
> +-+
> 3 rows selected (1.645 seconds)
> *When column is Hbase.Talbename.ColumnFamily.Qualify, and count(column) less 
> than 32768,it work well !*
> 0: jdbc:drill:> select count(wi.cf1.q5) over() from 
> hbase.web_initial_20151214 wi group by wi.cf1.q5 limit 3;
> +-+
> | EXPR$0  |
> +-+
> | 10383   |
> | 10383   |
> | 10383   |
> +-+
> 3 rows selected (1.044 seconds)
> {color:red}
> When column is Hbase.Talbename.ColumnFamily.Qualify, and count(column) 
> more than 32768,IndexOutOfBoundsException
> {color}
> 0: jdbc:drill:> select count(wi.cf1.q5) over() from 
> hbase.web_initial_20151222 wi group by wi.cf1.q5 limit 3;
> Error: SYSTEM ERROR: IndexOutOfBoundsException: index: 0, length: 62784 
> (expected: range(0, 32768))
> Fragment 0:0
> [Error Id: 77406a8a-8389-4f1b-af6c-d26d811379b7 on slave4.hadoop:31010] 
> (state=,code=0)
> java.sql.SQLException: SYSTEM ERROR: IndexOutOfBoundsException: index: 0, 
> length: 62784 (expected: range(0, 32768))
> Fragment 0:0
> [Error Id: 77406a8a-8389-4f1b-af6c-d26d811379b7 on slave4.hadoop:31010]
>   at 
> org.apache.drill.jdbc.impl.DrillCursor.nextRowInternally(DrillCursor.java:247)
>   at org.apache.drill.jdbc.impl.DrillCursor.next(DrillCursor.java:320)
>   at 
> net.hydromatic.avatica.AvaticaResultSet.next(AvaticaResultSet.java:187)
>   at 
> org.apache.drill.jdbc.impl.DrillResultSetImpl.next(DrillResultSetImpl.java:160)
>   at sqlline.IncrementalRows.hasNext(IncrementalRows.java:62)
>   at 
> sqlline.TableOutputFormat$ResizingRowsProvider.next(TableOutputFormat.java:87)
>   at sqlline.TableOutputFormat.print(TableOutputFormat.java:118)
>   at sqlline.SqlLine.print(SqlLine.java:1593)
>   at sqlline.Commands.execute(Commands.java:852)
>   at sqlline.Commands.sql(Commands.java:751)
>   at sqlline.SqlLine.dispatch(SqlLine.java:746)
>   at sqlline.SqlLine.begin(SqlLine.java:621)
>   at sqlline.SqlLine.start(SqlLine.java:375)
>   at sqlline.SqlLine.main(SqlLine.java:268)
> Caused by: org.apache.drill.common.exceptions.UserRemoteException: SYSTEM 
> ERROR: IndexOutOfBoundsException: index: 0, length: 62784 (expected: range(0, 
> 32768))
> Fragment 0:0
> [Error Id: 77406a8a-8389-4f1b-af6c-d26d811379b7 on slave4.hadoop:31010]
>   at 
> org.apache.drill.exec.rpc.user.QueryResultHandler.resultArrived(QueryResultHandler.java:118)
>   at 
> org.apache.drill.exec.rpc.user.UserClient.handleReponse(UserClient.java:112)
>   at 
> org.apache.drill.exec.rpc.BasicClientWithConnection.handle(BasicClientWithConnection.java:47)
>   at 
> org.apache.drill.exec.rpc.BasicClientWithConnection.handle(BasicClientWithConnection.java:32)
>   at org.apache.drill.exec.rpc.RpcBus.handle(RpcBus.java:69)
>   at org.apache.drill.exec.rpc.RpcBus$RequestEvent.run(RpcBus.java:400)
>   at 
> org.apache.drill.common.SerializedExecutor$RunnableProcessor.run(SerializedExecutor.java:105)
>   at 
> org.apache.drill.exec.rpc.RpcBus$SameExecutor.execute(RpcBus.java:264)
>   at 
> org.apache.drill.common.SerializedExecutor.execute(SerializedExecutor.java:142)
>   at 
> org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:298)
>   at 
> org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:269)
>   at 
> io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:89)
>   at 
> 

[jira] [Comment Edited] (DRILL-4216) Aggregate Window Function COUNT() With GROUP BY Clause expected: range(0, 32768)

[PIPELINE (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15069067#comment-15069067
 ] 

PIPELINE edited comment on DRILL-4216 at 12/23/15 3:04 AM:
---

Year,in this query,I just wanna count how many groups.
And the complete statement may like this:
var page=1
var pageSize=10

"select count(qualify) over() as total,count() as `count`,... from tablename 
group by qualify order by `count` desc limit pageSize offset (page-1)*pageSize 
rows"

then can get the totalPage=total/pageSize, 
and in the next query,if((page-1)*pageSize+pageSize > totalPage) then do 
nothing.

The "total" in Hbase change with time,and I do not wanna use two queries.



was (Author: pipe):
Year,in this query,I just wanna count how many groups.
And the complete statement may like this:
var page=1
var pageSize=10

"select count(qualify) over() as total,count() as `count`,... from tablename 
group by qualify order by `count` limit pageSize offset (page-1)*pageSize rows"

then can get the totalPage=total/pageSize, 
and in the next query,if((page-1)*pageSize+pageSize > totalPage) then do 
nothing.

The "total" in Hbase change with time,and I do not wanna use two queries.


> Aggregate Window Function COUNT() With GROUP BY Clause expected: range(0, 
> 32768)
> 
>
> Key: DRILL-4216
> URL: https://issues.apache.org/jira/browse/DRILL-4216
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Functions - Drill
>Affects Versions: 1.3.0
> Environment: Hadoop-2.5.2
> Hbase-0.9.15
> Java-1.7.0_85
>Reporter: PIPELINE
>Assignee: Deneche A. Hakim
>
> *When column is row_key,it work well !*
> 0: jdbc:drill:> select count(row_key) over() from hbase.web_initial_20151222 
> wi group by row_key limit 3;
> +-+
> | EXPR$0  |
> +-+
> | 102906  |
> | 102906  |
> | 102906  |
> +-+
> 3 rows selected (1.645 seconds)
> *When column is Hbase.Talbename.ColumnFamily.Qualify, and count(column) less 
> than 32768,it work well !*
> 0: jdbc:drill:> select count(wi.cf1.q5) over() from 
> hbase.web_initial_20151214 wi group by wi.cf1.q5 limit 3;
> +-+
> | EXPR$0  |
> +-+
> | 10383   |
> | 10383   |
> | 10383   |
> +-+
> 3 rows selected (1.044 seconds)
> {color:red}
> When column is Hbase.Talbename.ColumnFamily.Qualify, and count(column) 
> more than 32768,IndexOutOfBoundsException
> {color}
> 0: jdbc:drill:> select count(wi.cf1.q5) over() from 
> hbase.web_initial_20151222 wi group by wi.cf1.q5 limit 3;
> Error: SYSTEM ERROR: IndexOutOfBoundsException: index: 0, length: 62784 
> (expected: range(0, 32768))
> Fragment 0:0
> [Error Id: 77406a8a-8389-4f1b-af6c-d26d811379b7 on slave4.hadoop:31010] 
> (state=,code=0)
> java.sql.SQLException: SYSTEM ERROR: IndexOutOfBoundsException: index: 0, 
> length: 62784 (expected: range(0, 32768))
> Fragment 0:0
> [Error Id: 77406a8a-8389-4f1b-af6c-d26d811379b7 on slave4.hadoop:31010]
>   at 
> org.apache.drill.jdbc.impl.DrillCursor.nextRowInternally(DrillCursor.java:247)
>   at org.apache.drill.jdbc.impl.DrillCursor.next(DrillCursor.java:320)
>   at 
> net.hydromatic.avatica.AvaticaResultSet.next(AvaticaResultSet.java:187)
>   at 
> org.apache.drill.jdbc.impl.DrillResultSetImpl.next(DrillResultSetImpl.java:160)
>   at sqlline.IncrementalRows.hasNext(IncrementalRows.java:62)
>   at 
> sqlline.TableOutputFormat$ResizingRowsProvider.next(TableOutputFormat.java:87)
>   at sqlline.TableOutputFormat.print(TableOutputFormat.java:118)
>   at sqlline.SqlLine.print(SqlLine.java:1593)
>   at sqlline.Commands.execute(Commands.java:852)
>   at sqlline.Commands.sql(Commands.java:751)
>   at sqlline.SqlLine.dispatch(SqlLine.java:746)
>   at sqlline.SqlLine.begin(SqlLine.java:621)
>   at sqlline.SqlLine.start(SqlLine.java:375)
>   at sqlline.SqlLine.main(SqlLine.java:268)
> Caused by: org.apache.drill.common.exceptions.UserRemoteException: SYSTEM 
> ERROR: IndexOutOfBoundsException: index: 0, length: 62784 (expected: range(0, 
> 32768))
> Fragment 0:0
> [Error Id: 77406a8a-8389-4f1b-af6c-d26d811379b7 on slave4.hadoop:31010]
>   at 
> org.apache.drill.exec.rpc.user.QueryResultHandler.resultArrived(QueryResultHandler.java:118)
>   at 
> org.apache.drill.exec.rpc.user.UserClient.handleReponse(UserClient.java:112)
>   at 
> org.apache.drill.exec.rpc.BasicClientWithConnection.handle(BasicClientWithConnection.java:47)
>   at 
> org.apache.drill.exec.rpc.BasicClientWithConnection.handle(BasicClientWithConnection.java:32)
>   at org.apache.drill.exec.rpc.RpcBus.handle(RpcBus.java:69)
>   at org.apache.drill.exec.rpc.RpcBus$RequestEvent.run(RpcBus.java:400)
>   at 
> 

[jira] [Commented] (DRILL-4187) Introduce a state to separate queries pending execution from those pending in the queue.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15068859#comment-15068859
 ] 

ASF GitHub Bot commented on DRILL-4187:
---

Github user sudheeshkatkam commented on the pull request:

https://github.com/apache/drill/pull/310#issuecomment-166754159
  
How should cancellation be handled when the query is ENQUEUED?


> Introduce a state to separate queries pending execution from those pending in 
> the queue.
> 
>
> Key: DRILL-4187
> URL: https://issues.apache.org/jira/browse/DRILL-4187
> Project: Apache Drill
>  Issue Type: Sub-task
>Reporter: Hanifi Gunes
>Assignee: Mehant Baid
> Fix For: 1.5.0
>
>
> Currently queries pending in the queue are not listed in the web UI besides 
> we use the state PENDING to mean pending executions. This issue proposes i) 
> to list enqueued queries in the web UI ii) to introduce a new state for 
> queries sitting at the queue, differentiating then from those pending 
> execution.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-4187) Introduce a state to separate queries pending execution from those pending in the queue.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15068847#comment-15068847
 ] 

ASF GitHub Bot commented on DRILL-4187:
---

Github user sudheeshkatkam commented on a diff in the pull request:

https://github.com/apache/drill/pull/310#discussion_r48306322
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/QueryManager.java
 ---
@@ -287,6 +288,7 @@ QueryState updateEphemeralState(final QueryState 
queryState) {
   case CANCELED:
   case FAILED:
 try {
+  profilePStore.put(stringQueryId, getQueryProfile());
--- End diff --

Should be removed. Foreman.ForemanResult#close has the call to write the 
final profile.


> Introduce a state to separate queries pending execution from those pending in 
> the queue.
> 
>
> Key: DRILL-4187
> URL: https://issues.apache.org/jira/browse/DRILL-4187
> Project: Apache Drill
>  Issue Type: Sub-task
>Reporter: Hanifi Gunes
>Assignee: Mehant Baid
> Fix For: 1.5.0
>
>
> Currently queries pending in the queue are not listed in the web UI besides 
> we use the state PENDING to mean pending executions. This issue proposes i) 
> to list enqueued queries in the web UI ii) to introduce a new state for 
> queries sitting at the queue, differentiating then from those pending 
> execution.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-4187) Introduce a state to separate queries pending execution from those pending in the queue.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15068843#comment-15068843
 ] 

ASF GitHub Bot commented on DRILL-4187:
---

Github user sudheeshkatkam commented on a diff in the pull request:

https://github.com/apache/drill/pull/310#discussion_r48306302
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/Foreman.java ---
@@ -135,6 +135,8 @@
   private final ForemanResult foremanResult = new ForemanResult();
   private final ConnectionClosedListener closeListener = new 
ConnectionClosedListener();
   private final ChannelFuture closeFuture;
+  final boolean queuingEnabled;
--- End diff --

private


> Introduce a state to separate queries pending execution from those pending in 
> the queue.
> 
>
> Key: DRILL-4187
> URL: https://issues.apache.org/jira/browse/DRILL-4187
> Project: Apache Drill
>  Issue Type: Sub-task
>Reporter: Hanifi Gunes
>Assignee: Mehant Baid
> Fix For: 1.5.0
>
>
> Currently queries pending in the queue are not listed in the web UI besides 
> we use the state PENDING to mean pending executions. This issue proposes i) 
> to list enqueued queries in the web UI ii) to introduce a new state for 
> queries sitting at the queue, differentiating then from those pending 
> execution.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-4187) Introduce a state to separate queries pending execution from those pending in the queue.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15068845#comment-15068845
 ] 

ASF GitHub Bot commented on DRILL-4187:
---

Github user sudheeshkatkam commented on a diff in the pull request:

https://github.com/apache/drill/pull/310#discussion_r48306314
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/Foreman.java ---
@@ -458,49 +467,46 @@ private void setupSortMemoryAllocations(final 
PhysicalPlan plan) {
*/
   private void acquireQuerySemaphore(final PhysicalPlan plan) throws 
ForemanSetupException {
 final OptionManager optionManager = queryContext.getOptions();
-final boolean queuingEnabled = 
optionManager.getOption(ExecConstants.ENABLE_QUEUE);
-if (queuingEnabled) {
-  final long queueThreshold = 
optionManager.getOption(ExecConstants.QUEUE_THRESHOLD_SIZE);
-  double totalCost = 0;
-  for (final PhysicalOperator ops : plan.getSortedOperators()) {
-totalCost += ops.getCost();
-  }
-
-  final long queueTimeout = 
optionManager.getOption(ExecConstants.QUEUE_TIMEOUT);
-  final String queueName;
-
-  try {
-@SuppressWarnings("resource")
-final ClusterCoordinator clusterCoordinator = 
drillbitContext.getClusterCoordinator();
-final DistributedSemaphore distributedSemaphore;
-
-// get the appropriate semaphore
-if (totalCost > queueThreshold) {
-  final int largeQueue = (int) 
optionManager.getOption(ExecConstants.LARGE_QUEUE_SIZE);
-  distributedSemaphore = 
clusterCoordinator.getSemaphore("query.large", largeQueue);
-  queueName = "large";
-} else {
-  final int smallQueue = (int) 
optionManager.getOption(ExecConstants.SMALL_QUEUE_SIZE);
-  distributedSemaphore = 
clusterCoordinator.getSemaphore("query.small", smallQueue);
-  queueName = "small";
-}
+final long queueThreshold = 
optionManager.getOption(ExecConstants.QUEUE_THRESHOLD_SIZE);
+double totalCost = 0;
+for (final PhysicalOperator ops : plan.getSortedOperators()) {
+  totalCost += ops.getCost();
+}
 
+final long queueTimeout = 
optionManager.getOption(ExecConstants.QUEUE_TIMEOUT);
+final String queueName;
 
-lease = distributedSemaphore.acquire(queueTimeout, 
TimeUnit.MILLISECONDS);
-  } catch (final Exception e) {
-throw new ForemanSetupException("Unable to acquire slot for 
query.", e);
+try {
+  @SuppressWarnings("resource")
+  final ClusterCoordinator clusterCoordinator = 
drillbitContext.getClusterCoordinator();
+  final DistributedSemaphore distributedSemaphore;
+
+  // get the appropriate semaphore
+  if (totalCost > queueThreshold) {
+final int largeQueue = (int) 
optionManager.getOption(ExecConstants.LARGE_QUEUE_SIZE);
+distributedSemaphore = 
clusterCoordinator.getSemaphore("query.large", largeQueue);
+queueName = "large";
+  } else {
+final int smallQueue = (int) 
optionManager.getOption(ExecConstants.SMALL_QUEUE_SIZE);
+distributedSemaphore = 
clusterCoordinator.getSemaphore("query.small", smallQueue);
+queueName = "small";
   }
 
-  if (lease == null) {
-throw UserException
-.resourceError()
-.message(
-"Unable to acquire queue resources for query within 
timeout.  Timeout for %s queue was set at %d seconds.",
-queueName, queueTimeout / 1000)
-.build(logger);
-  }
+  recordNewState(QueryState.ENQUEUED);
--- End diff --

Since the current "state" is ENQUEUED, is this required?


> Introduce a state to separate queries pending execution from those pending in 
> the queue.
> 
>
> Key: DRILL-4187
> URL: https://issues.apache.org/jira/browse/DRILL-4187
> Project: Apache Drill
>  Issue Type: Sub-task
>Reporter: Hanifi Gunes
>Assignee: Mehant Baid
> Fix For: 1.5.0
>
>
> Currently queries pending in the queue are not listed in the web UI besides 
> we use the state PENDING to mean pending executions. This issue proposes i) 
> to list enqueued queries in the web UI ii) to introduce a new state for 
> queries sitting at the queue, differentiating then from those pending 
> execution.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-4187) Introduce a state to separate queries pending execution from those pending in the queue.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15068844#comment-15068844
 ] 

ASF GitHub Bot commented on DRILL-4187:
---

Github user sudheeshkatkam commented on a diff in the pull request:

https://github.com/apache/drill/pull/310#discussion_r48306306
  
--- Diff: 
exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/Foreman.java ---
@@ -391,7 +397,10 @@ private void parseAndRunPhysicalPlan(final String 
json) throws ExecutionSetupExc
   private void runPhysicalPlan(final PhysicalPlan plan) throws 
ExecutionSetupException {
 validatePlan(plan);
 setupSortMemoryAllocations(plan);
-acquireQuerySemaphore(plan);
+if (queuingEnabled) {
+  acquireQuerySemaphore(plan);
+}
+recordNewState(QueryState.STARTING);
--- End diff --

If the initialState is STARTING won't this fail (state transition from 
STARTING to STARTING is invalid given the fall-through)? The 
recordNewState(STARTING) should be inside the if condition, right?


> Introduce a state to separate queries pending execution from those pending in 
> the queue.
> 
>
> Key: DRILL-4187
> URL: https://issues.apache.org/jira/browse/DRILL-4187
> Project: Apache Drill
>  Issue Type: Sub-task
>Reporter: Hanifi Gunes
>Assignee: Mehant Baid
> Fix For: 1.5.0
>
>
> Currently queries pending in the queue are not listed in the web UI besides 
> we use the state PENDING to mean pending executions. This issue proposes i) 
> to list enqueued queries in the web UI ii) to introduce a new state for 
> queries sitting at the queue, differentiating then from those pending 
> execution.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-4192) Dir0 and Dir1 from drill-1.4 are messed up

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15068861#comment-15068861
 ] 

ASF GitHub Bot commented on DRILL-4192:
---

Github user asfgit closed the pull request at:

https://github.com/apache/drill/pull/307


> Dir0 and Dir1 from drill-1.4 are messed up
> --
>
> Key: DRILL-4192
> URL: https://issues.apache.org/jira/browse/DRILL-4192
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Execution - Flow
>Affects Versions: 1.4.0
>Reporter: Krystal
>Assignee: Aman Sinha
>Priority: Blocker
>
> I have the following directories:
> /drill/testdata/temp1/abc/dt=2014-12-30/lineitem.parquet
> /drill/testdata/temp1/abc/dt=2014-12-31/lineitem.parquet
> The following queries returned incorrect data.
> select dir0,dir1 from dfs.`/drill/testdata/temp1` limit 2;
> ++---+
> |  dir0  | dir1  |
> ++---+
> | dt=2014-12-30  | null  |
> | dt=2014-12-30  | null  |
> ++---+
> select dir0 from dfs.`/drill/testdata/temp1` limit 2;
> ++
> |  dir0  |
> ++
> | dt=2014-12-31  |
> | dt=2014-12-31  |
> ++



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-4187) Introduce a state to separate queries pending execution from those pending in the queue.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15068910#comment-15068910
 ] 

ASF GitHub Bot commented on DRILL-4187:
---

Github user hnfgns commented on the pull request:

https://github.com/apache/drill/pull/310#issuecomment-166762090
  
Cancellation is handled usual way after query is cancelled only after it 
has been started.


> Introduce a state to separate queries pending execution from those pending in 
> the queue.
> 
>
> Key: DRILL-4187
> URL: https://issues.apache.org/jira/browse/DRILL-4187
> Project: Apache Drill
>  Issue Type: Sub-task
>Reporter: Hanifi Gunes
>Assignee: Mehant Baid
> Fix For: 1.5.0
>
>
> Currently queries pending in the queue are not listed in the web UI besides 
> we use the state PENDING to mean pending executions. This issue proposes i) 
> to list enqueued queries in the web UI ii) to introduce a new state for 
> queries sitting at the queue, differentiating then from those pending 
> execution.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-4187) Introduce a state to separate queries pending execution from those pending in the queue.

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15068912#comment-15068912
 ] 

ASF GitHub Bot commented on DRILL-4187:
---

Github user hnfgns commented on the pull request:

https://github.com/apache/drill/pull/310#issuecomment-166762114
  
Addressed review comments.


> Introduce a state to separate queries pending execution from those pending in 
> the queue.
> 
>
> Key: DRILL-4187
> URL: https://issues.apache.org/jira/browse/DRILL-4187
> Project: Apache Drill
>  Issue Type: Sub-task
>Reporter: Hanifi Gunes
>Assignee: Mehant Baid
> Fix For: 1.5.0
>
>
> Currently queries pending in the queue are not listed in the web UI besides 
> we use the state PENDING to mean pending executions. This issue proposes i) 
> to list enqueued queries in the web UI ii) to introduce a new state for 
> queries sitting at the queue, differentiating then from those pending 
> execution.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-3336) to_date(to_timestamp) with group-by in hbase/maprdb table fails with "java.lang.UnsupportedOperationException"

[Abhishek Girish (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-3336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15068919#comment-15068919
 ] 

Abhishek Girish commented on DRILL-3336:


Verified this on Drill 1.5.0 snapshot (bc74629a546afe109252dcf4e3ef00ffc22e7a7a)
{code}
> select to_date(to_timestamp(cast(convert_from(esrtable.row_key,'UTF8') as 
> int))),count(*) from esr52 esrtable 
> group by to_date(to_timestamp(cast(convert_from(esrtable.row_key,'UTF8') as 
> int)));
+-+-+
|   EXPR$0| EXPR$1  |
+-+-+
| 2015-06-22  | 1   |
+-+-+
1 row selected (0.474 seconds)
{code}

> to_date(to_timestamp) with group-by in hbase/maprdb table fails with 
> "java.lang.UnsupportedOperationException"
> --
>
> Key: DRILL-3336
> URL: https://issues.apache.org/jira/browse/DRILL-3336
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Functions - Drill
>Affects Versions: 1.0.0
> Environment: 1.0 GA version
>Reporter: Hao Zhu
>Assignee: Mehant Baid
>Priority: Critical
> Fix For: 1.3.0
>
>
> 1. Create a hbase/maprdb table in hbase shell:
> {code}
> create '/tables/esr52','cf'
> put '/tables/esr52','1434998909','cf:c','abc'
> > scan '/tables/esr52'
> ROW  
> COLUMN+CELL
>  1434998909  
> column=cf:c, timestamp=1434998994785, value=abc
> {code}
> 2. Below SQLs work fine in Drill:
> {code}
> >  select * from maprdb.esr52;
> +--+---+
> |   row_key|  cf   |
> +--+---+
> | [B@5bafd971  | {"c":"YWJj"}  |
> +--+---+
> 1 row selected (0.095 seconds)
> > select to_date(to_timestamp(cast(convert_from(esrtable.row_key,'UTF8') as 
> > int))) from maprdb.esr52 esrtable;
> +-+
> |   EXPR$0|
> +-+
> | 2015-06-22  |
> +-+
> 1 row selected (0.127 seconds)
> {code}
> 3. However below SQL with group-by fails:
> {code}
> select to_date(to_timestamp(cast(convert_from(esrtable.row_key,'UTF8') as 
> int))),count(*) from maprdb.esr52 esrtable 
> group by to_date(to_timestamp(cast(convert_from(esrtable.row_key,'UTF8') as 
> int)));
> Error: SYSTEM ERROR: java.lang.UnsupportedOperationException: Failure finding 
> function that runtime code generation expected.  Signature: 
> compare_to_nulls_high( VAR16CHAR:OPTIONAL, VAR16CHAR:OPTIONAL ) returns 
> INT:REQUIRED
> Fragment 3:0
> [Error Id: 26003311-d40e-4a95-9d3c-68793459ad6d on h1.poc.com:31010]
>   (java.lang.UnsupportedOperationException) Failure finding function that 
> runtime code generation expected.  Signature: compare_to_nulls_high( 
> VAR16CHAR:OPTIONAL, VAR16CHAR:OPTIONAL ) returns INT:REQUIRED
> 
> org.apache.drill.exec.expr.fn.FunctionGenerationHelper.getFunctionExpression():109
> 
> org.apache.drill.exec.expr.fn.FunctionGenerationHelper.getOrderingComparator():62
> 
> org.apache.drill.exec.expr.fn.FunctionGenerationHelper.getOrderingComparatorNullsHigh():79
> 
> org.apache.drill.exec.physical.impl.common.ChainedHashTable.setupIsKeyMatchInternal():257
> 
> org.apache.drill.exec.physical.impl.common.ChainedHashTable.createAndSetupHashTable():206
> org.apache.drill.exec.test.generated.HashAggregatorGen1.setup():273
> 
> org.apache.drill.exec.physical.impl.aggregate.HashAggBatch.createAggregatorInternal():240
> 
> org.apache.drill.exec.physical.impl.aggregate.HashAggBatch.createAggregator():163
> 
> org.apache.drill.exec.physical.impl.aggregate.HashAggBatch.buildSchema():110
> org.apache.drill.exec.record.AbstractRecordBatch.next():127
> org.apache.drill.exec.record.AbstractRecordBatch.next():105
> org.apache.drill.exec.record.AbstractRecordBatch.next():95
> org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext():51
> 
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext():129
> org.apache.drill.exec.record.AbstractRecordBatch.next():146
> org.apache.drill.exec.physical.impl.BaseRootExec.next():83
> 
> org.apache.drill.exec.physical.impl.SingleSenderCreator$SingleSenderRootExec.innerNext():95
> org.apache.drill.exec.physical.impl.BaseRootExec.next():73
> org.apache.drill.exec.work.fragment.FragmentExecutor$1.run():259
> org.apache.drill.exec.work.fragment.FragmentExecutor$1.run():253
> java.security.AccessController.doPrivileged():-2
> javax.security.auth.Subject.doAs():422
> org.apache.hadoop.security.UserGroupInformation.doAs():1566
> org.apache.drill.exec.work.fragment.FragmentExecutor.run():253
> org.apache.drill.common.SelfCleaningRunnable.run():38
> 

[jira] [Closed] (DRILL-3336) to_date(to_timestamp) with group-by in hbase/maprdb table fails with "java.lang.UnsupportedOperationException"

[Abhishek Girish (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-3336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Abhishek Girish closed DRILL-3336.
--

> to_date(to_timestamp) with group-by in hbase/maprdb table fails with 
> "java.lang.UnsupportedOperationException"
> --
>
> Key: DRILL-3336
> URL: https://issues.apache.org/jira/browse/DRILL-3336
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Functions - Drill
>Affects Versions: 1.0.0
> Environment: 1.0 GA version
>Reporter: Hao Zhu
>Assignee: Mehant Baid
>Priority: Critical
> Fix For: 1.3.0
>
>
> 1. Create a hbase/maprdb table in hbase shell:
> {code}
> create '/tables/esr52','cf'
> put '/tables/esr52','1434998909','cf:c','abc'
> > scan '/tables/esr52'
> ROW  
> COLUMN+CELL
>  1434998909  
> column=cf:c, timestamp=1434998994785, value=abc
> {code}
> 2. Below SQLs work fine in Drill:
> {code}
> >  select * from maprdb.esr52;
> +--+---+
> |   row_key|  cf   |
> +--+---+
> | [B@5bafd971  | {"c":"YWJj"}  |
> +--+---+
> 1 row selected (0.095 seconds)
> > select to_date(to_timestamp(cast(convert_from(esrtable.row_key,'UTF8') as 
> > int))) from maprdb.esr52 esrtable;
> +-+
> |   EXPR$0|
> +-+
> | 2015-06-22  |
> +-+
> 1 row selected (0.127 seconds)
> {code}
> 3. However below SQL with group-by fails:
> {code}
> select to_date(to_timestamp(cast(convert_from(esrtable.row_key,'UTF8') as 
> int))),count(*) from maprdb.esr52 esrtable 
> group by to_date(to_timestamp(cast(convert_from(esrtable.row_key,'UTF8') as 
> int)));
> Error: SYSTEM ERROR: java.lang.UnsupportedOperationException: Failure finding 
> function that runtime code generation expected.  Signature: 
> compare_to_nulls_high( VAR16CHAR:OPTIONAL, VAR16CHAR:OPTIONAL ) returns 
> INT:REQUIRED
> Fragment 3:0
> [Error Id: 26003311-d40e-4a95-9d3c-68793459ad6d on h1.poc.com:31010]
>   (java.lang.UnsupportedOperationException) Failure finding function that 
> runtime code generation expected.  Signature: compare_to_nulls_high( 
> VAR16CHAR:OPTIONAL, VAR16CHAR:OPTIONAL ) returns INT:REQUIRED
> 
> org.apache.drill.exec.expr.fn.FunctionGenerationHelper.getFunctionExpression():109
> 
> org.apache.drill.exec.expr.fn.FunctionGenerationHelper.getOrderingComparator():62
> 
> org.apache.drill.exec.expr.fn.FunctionGenerationHelper.getOrderingComparatorNullsHigh():79
> 
> org.apache.drill.exec.physical.impl.common.ChainedHashTable.setupIsKeyMatchInternal():257
> 
> org.apache.drill.exec.physical.impl.common.ChainedHashTable.createAndSetupHashTable():206
> org.apache.drill.exec.test.generated.HashAggregatorGen1.setup():273
> 
> org.apache.drill.exec.physical.impl.aggregate.HashAggBatch.createAggregatorInternal():240
> 
> org.apache.drill.exec.physical.impl.aggregate.HashAggBatch.createAggregator():163
> 
> org.apache.drill.exec.physical.impl.aggregate.HashAggBatch.buildSchema():110
> org.apache.drill.exec.record.AbstractRecordBatch.next():127
> org.apache.drill.exec.record.AbstractRecordBatch.next():105
> org.apache.drill.exec.record.AbstractRecordBatch.next():95
> org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext():51
> 
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext():129
> org.apache.drill.exec.record.AbstractRecordBatch.next():146
> org.apache.drill.exec.physical.impl.BaseRootExec.next():83
> 
> org.apache.drill.exec.physical.impl.SingleSenderCreator$SingleSenderRootExec.innerNext():95
> org.apache.drill.exec.physical.impl.BaseRootExec.next():73
> org.apache.drill.exec.work.fragment.FragmentExecutor$1.run():259
> org.apache.drill.exec.work.fragment.FragmentExecutor$1.run():253
> java.security.AccessController.doPrivileged():-2
> javax.security.auth.Subject.doAs():422
> org.apache.hadoop.security.UserGroupInformation.doAs():1566
> org.apache.drill.exec.work.fragment.FragmentExecutor.run():253
> org.apache.drill.common.SelfCleaningRunnable.run():38
> java.util.concurrent.ThreadPoolExecutor.runWorker():1142
> java.util.concurrent.ThreadPoolExecutor$Worker.run():617
> java.lang.Thread.run():745 (state=,code=0)
> {code}
> 4. If we remove to_date, and only group-by to_timestamp, it works fine:
> {code}
> select to_timestamp(cast(convert_from(esrtable.row_key,'UTF8') as int)) from 
> maprdb.esr52 esrtable;
> ++
> | EXPR$0 |
> ++
> | 2015-06-22 18:48:29.0  |
> ++
> 1 

[jira] [Commented] (DRILL-4216) Aggregate Window Function COUNT() With GROUP BY Clause expected: range(0, 32768)

[PIPELINE (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15069177#comment-15069177
 ] 

PIPELINE commented on DRILL-4216:
-

*The drillbit.log*

2015-12-23 13:40:16,251 [2985cc3e-e1ee-4b0e-7400-3704bd613443:foreman] INFO  
o.a.drill.exec.work.foreman.Foreman - Query text for query id 
2985cc3e-e1ee-4b0e-7400-3704bd613443: select count(*) `count`,count(wi.cf1.q5) 
over() total from hbase.web_initial_20151223 wi group by wi.cf1.q5 limit 3
2015-12-23 13:40:16,702 [2985cc3e-e1ee-4b0e-7400-3704bd613443:foreman] INFO  
o.a.d.e.s.hbase.TableStatsCalculator - Region size calculation disabled.
2015-12-23 13:40:18,490 [2985cc3e-e1ee-4b0e-7400-3704bd613443:frag:0:0] INFO  
o.a.d.e.w.fragment.FragmentExecutor - 2985cc3e-e1ee-4b0e-7400-3704bd613443:0:0: 
State change requested AWAITING_ALLOCATION --> RUNNING
2015-12-23 13:40:18,491 [2985cc3e-e1ee-4b0e-7400-3704bd613443:frag:0:0] INFO  
o.a.d.e.w.f.FragmentStatusReporter - 2985cc3e-e1ee-4b0e-7400-3704bd613443:0:0: 
State to report: RUNNING
2015-12-23 13:40:21,976 [2985cc3e-e1ee-4b0e-7400-3704bd613443:frag:0:0] INFO  
o.a.d.e.w.fragment.FragmentExecutor - 2985cc3e-e1ee-4b0e-7400-3704bd613443:0:0: 
State change requested RUNNING --> FAILED
2015-12-23 13:40:21,977 [2985cc3e-e1ee-4b0e-7400-3704bd613443:frag:0:0] INFO  
o.a.d.e.w.fragment.FragmentExecutor - 2985cc3e-e1ee-4b0e-7400-3704bd613443:0:0: 
State change requested FAILED --> FAILED
2015-12-23 13:40:21,978 [2985cc3e-e1ee-4b0e-7400-3704bd613443:frag:0:0] INFO  
o.a.d.e.w.fragment.FragmentExecutor - 2985cc3e-e1ee-4b0e-7400-3704bd613443:0:0: 
State change requested FAILED --> FAILED
2015-12-23 13:40:21,978 [2985cc3e-e1ee-4b0e-7400-3704bd613443:frag:0:0] INFO  
o.a.d.e.w.fragment.FragmentExecutor - 2985cc3e-e1ee-4b0e-7400-3704bd613443:0:0: 
State change requested FAILED --> FINISHED
2015-12-23 13:40:21,981 [2985cc3e-e1ee-4b0e-7400-3704bd613443:frag:0:0] ERROR 
o.a.d.e.w.fragment.FragmentExecutor - SYSTEM ERROR: IndexOutOfBoundsException: 
index: 0, length: 90960 (expected: range(0, 32768))

Fragment 0:0

[Error Id: dec9a67a-5e60-4148-92b9-0c204e2be9b6 on slave4.hadoop:31010]
org.apache.drill.common.exceptions.UserException: SYSTEM ERROR: 
IndexOutOfBoundsException: index: 0, length: 90960 (expected: range(0, 32768))

Fragment 0:0

[Error Id: dec9a67a-5e60-4148-92b9-0c204e2be9b6 on slave4.hadoop:31010]
at 
org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:534)
 ~[drill-common-1.4.0.jar:1.4.0]
at 
org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:321)
 [drill-java-exec-1.4.0.jar:1.4.0]
at 
org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:184)
 [drill-java-exec-1.4.0.jar:1.4.0]
at 
org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:290)
 [drill-java-exec-1.4.0.jar:1.4.0]
at 
org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38) 
[drill-common-1.4.0.jar:1.4.0]
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) 
[na:1.7.0_80]
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) 
[na:1.7.0_80]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_80]
Caused by: java.lang.IndexOutOfBoundsException: index: 0, length: 90960 
(expected: range(0, 32768))
at 
io.netty.buffer.AbstractByteBuf.checkIndex(AbstractByteBuf.java:1134) 
~[netty-buffer-4.0.27.Final.jar:4.0.27.Final]
at 
io.netty.buffer.PooledUnsafeDirectByteBuf.setBytes(PooledUnsafeDirectByteBuf.java:250)
 ~[netty-buffer-4.0.27.Final.jar:4.0.27.Final]
at io.netty.buffer.WrappedByteBuf.setBytes(WrappedByteBuf.java:378) 
~[netty-buffer-4.0.27.Final.jar:4.0.27.Final]
at 
io.netty.buffer.UnsafeDirectLittleEndian.setBytes(UnsafeDirectLittleEndian.java:26)
 ~[drill-memory-base-1.4.0.jar:4.0.27.Final]
at io.netty.buffer.DrillBuf.setBytes(DrillBuf.java:685) 
~[drill-memory-base-1.4.0.jar:4.0.27.Final]
at 
org.apache.drill.exec.vector.UInt4Vector.reAlloc(UInt4Vector.java:219) 
~[vector-1.4.0.jar:1.4.0]
at 
org.apache.drill.exec.vector.UInt4Vector$Mutator.setSafe(UInt4Vector.java:412) 
~[vector-1.4.0.jar:1.4.0]
at 
org.apache.drill.exec.vector.VarBinaryVector$Mutator.setSafe(VarBinaryVector.java:519)
 ~[vector-1.4.0.jar:1.4.0]
at 
org.apache.drill.exec.vector.NullableVarBinaryVector$Mutator.fillEmpties(NullableVarBinaryVector.java:457)
 ~[vector-1.4.0.jar:1.4.0]
at 
org.apache.drill.exec.vector.NullableVarBinaryVector$Mutator.setValueCount(NullableVarBinaryVector.java:568)
 ~[vector-1.4.0.jar:1.4.0]
at 
org.apache.drill.exec.test.generated.WindowFramerGen20.doWork(DefaultFrameTemplate.java:137)
 ~[na:na]
at 

[jira] [Issue Comment Deleted] (DRILL-4216) Aggregate Window Function COUNT() With GROUP BY Clause expected: range(0, 32768)

[PIPELINE (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-4216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

PIPELINE updated DRILL-4216:

Comment: was deleted

(was: *The drillbit.log*

2015-12-23 13:40:16,251 [2985cc3e-e1ee-4b0e-7400-3704bd613443:foreman] INFO  
o.a.drill.exec.work.foreman.Foreman - Query text for query id 
2985cc3e-e1ee-4b0e-7400-3704bd613443: select count(*) `count`,count(wi.cf1.q5) 
over() total from hbase.web_initial_20151223 wi group by wi.cf1.q5 limit 3
2015-12-23 13:40:16,702 [2985cc3e-e1ee-4b0e-7400-3704bd613443:foreman] INFO  
o.a.d.e.s.hbase.TableStatsCalculator - Region size calculation disabled.
2015-12-23 13:40:18,490 [2985cc3e-e1ee-4b0e-7400-3704bd613443:frag:0:0] INFO  
o.a.d.e.w.fragment.FragmentExecutor - 2985cc3e-e1ee-4b0e-7400-3704bd613443:0:0: 
State change requested AWAITING_ALLOCATION --> RUNNING
2015-12-23 13:40:18,491 [2985cc3e-e1ee-4b0e-7400-3704bd613443:frag:0:0] INFO  
o.a.d.e.w.f.FragmentStatusReporter - 2985cc3e-e1ee-4b0e-7400-3704bd613443:0:0: 
State to report: RUNNING
2015-12-23 13:40:21,976 [2985cc3e-e1ee-4b0e-7400-3704bd613443:frag:0:0] INFO  
o.a.d.e.w.fragment.FragmentExecutor - 2985cc3e-e1ee-4b0e-7400-3704bd613443:0:0: 
State change requested RUNNING --> FAILED
2015-12-23 13:40:21,977 [2985cc3e-e1ee-4b0e-7400-3704bd613443:frag:0:0] INFO  
o.a.d.e.w.fragment.FragmentExecutor - 2985cc3e-e1ee-4b0e-7400-3704bd613443:0:0: 
State change requested FAILED --> FAILED
2015-12-23 13:40:21,978 [2985cc3e-e1ee-4b0e-7400-3704bd613443:frag:0:0] INFO  
o.a.d.e.w.fragment.FragmentExecutor - 2985cc3e-e1ee-4b0e-7400-3704bd613443:0:0: 
State change requested FAILED --> FAILED
2015-12-23 13:40:21,978 [2985cc3e-e1ee-4b0e-7400-3704bd613443:frag:0:0] INFO  
o.a.d.e.w.fragment.FragmentExecutor - 2985cc3e-e1ee-4b0e-7400-3704bd613443:0:0: 
State change requested FAILED --> FINISHED
2015-12-23 13:40:21,981 [2985cc3e-e1ee-4b0e-7400-3704bd613443:frag:0:0] ERROR 
o.a.d.e.w.fragment.FragmentExecutor - SYSTEM ERROR: IndexOutOfBoundsException: 
index: 0, length: 90960 (expected: range(0, 32768))

Fragment 0:0

[Error Id: dec9a67a-5e60-4148-92b9-0c204e2be9b6 on slave4.hadoop:31010]
org.apache.drill.common.exceptions.UserException: SYSTEM ERROR: 
IndexOutOfBoundsException: index: 0, length: 90960 (expected: range(0, 32768))

Fragment 0:0

[Error Id: dec9a67a-5e60-4148-92b9-0c204e2be9b6 on slave4.hadoop:31010]
at 
org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:534)
 ~[drill-common-1.4.0.jar:1.4.0]
at 
org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:321)
 [drill-java-exec-1.4.0.jar:1.4.0]
at 
org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:184)
 [drill-java-exec-1.4.0.jar:1.4.0]
at 
org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:290)
 [drill-java-exec-1.4.0.jar:1.4.0]
at 
org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38) 
[drill-common-1.4.0.jar:1.4.0]
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) 
[na:1.7.0_80]
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) 
[na:1.7.0_80]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_80]
Caused by: java.lang.IndexOutOfBoundsException: index: 0, length: 90960 
(expected: range(0, 32768))
at 
io.netty.buffer.AbstractByteBuf.checkIndex(AbstractByteBuf.java:1134) 
~[netty-buffer-4.0.27.Final.jar:4.0.27.Final]
at 
io.netty.buffer.PooledUnsafeDirectByteBuf.setBytes(PooledUnsafeDirectByteBuf.java:250)
 ~[netty-buffer-4.0.27.Final.jar:4.0.27.Final]
at io.netty.buffer.WrappedByteBuf.setBytes(WrappedByteBuf.java:378) 
~[netty-buffer-4.0.27.Final.jar:4.0.27.Final]
at 
io.netty.buffer.UnsafeDirectLittleEndian.setBytes(UnsafeDirectLittleEndian.java:26)
 ~[drill-memory-base-1.4.0.jar:4.0.27.Final]
at io.netty.buffer.DrillBuf.setBytes(DrillBuf.java:685) 
~[drill-memory-base-1.4.0.jar:4.0.27.Final]
at 
org.apache.drill.exec.vector.UInt4Vector.reAlloc(UInt4Vector.java:219) 
~[vector-1.4.0.jar:1.4.0]
at 
org.apache.drill.exec.vector.UInt4Vector$Mutator.setSafe(UInt4Vector.java:412) 
~[vector-1.4.0.jar:1.4.0]
at 
org.apache.drill.exec.vector.VarBinaryVector$Mutator.setSafe(VarBinaryVector.java:519)
 ~[vector-1.4.0.jar:1.4.0]
at 
org.apache.drill.exec.vector.NullableVarBinaryVector$Mutator.fillEmpties(NullableVarBinaryVector.java:457)
 ~[vector-1.4.0.jar:1.4.0]
at 
org.apache.drill.exec.vector.NullableVarBinaryVector$Mutator.setValueCount(NullableVarBinaryVector.java:568)
 ~[vector-1.4.0.jar:1.4.0]
at 
org.apache.drill.exec.test.generated.WindowFramerGen20.doWork(DefaultFrameTemplate.java:137)
 ~[na:na]
at 
org.apache.drill.exec.physical.impl.window.WindowFrameRecordBatch.innerNext(WindowFrameRecordBatch.java:177)
 

[jira] [Commented] (DRILL-4216) Aggregate Window Function COUNT() With GROUP BY Clause expected: range(0, 32768)

[PIPELINE (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15069253#comment-15069253
 ] 

PIPELINE commented on DRILL-4216:
-

I am so sorry!
I just "!set verbose true",not "set `exec.errors.verbose`=true",
After I set is, the Error like this:
{color:red}
0: jdbc:drill:> select count(*) `count`,count(wi.cf1.q5) over() total from 
hbase.web_initial_20151223 wi group by wi.cf1.q5 limit 3;
Error: SYSTEM ERROR: IndexOutOfBoundsException: index: 0, length: 102516 
(expected: range(0, 32768))

Fragment 0:0

[Error Id: f811b575-d63e-4f2a-ab84-666cf162f2f8 on slave1.hadoop:31010]

  (java.lang.IndexOutOfBoundsException) index: 0, length: 102516 (expected: 
range(0, 32768))
io.netty.buffer.AbstractByteBuf.checkIndex():1134
io.netty.buffer.PooledUnsafeDirectByteBuf.setBytes():250
io.netty.buffer.WrappedByteBuf.setBytes():378
io.netty.buffer.UnsafeDirectLittleEndian.setBytes():26
io.netty.buffer.DrillBuf.setBytes():685
org.apache.drill.exec.vector.UInt4Vector.reAlloc():219
org.apache.drill.exec.vector.UInt4Vector$Mutator.setSafe():412
org.apache.drill.exec.vector.VarBinaryVector$Mutator.setSafe():519

org.apache.drill.exec.vector.NullableVarBinaryVector$Mutator.fillEmpties():457

org.apache.drill.exec.vector.NullableVarBinaryVector$Mutator.setValueCount():568
org.apache.drill.exec.test.generated.WindowFramerGen61.doWork():137

org.apache.drill.exec.physical.impl.window.WindowFrameRecordBatch.innerNext():177
org.apache.drill.exec.record.AbstractRecordBatch.next():162
org.apache.drill.exec.record.AbstractRecordBatch.next():119
org.apache.drill.exec.record.AbstractRecordBatch.next():109
org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext():51

org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext():132
org.apache.drill.exec.record.AbstractRecordBatch.next():162
org.apache.drill.exec.record.AbstractRecordBatch.next():119
org.apache.drill.exec.record.AbstractRecordBatch.next():109
org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext():51
org.apache.drill.exec.physical.impl.limit.LimitRecordBatch.innerNext():115
org.apache.drill.exec.record.AbstractRecordBatch.next():162
org.apache.drill.exec.record.AbstractRecordBatch.next():119
org.apache.drill.exec.record.AbstractRecordBatch.next():109
org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext():51

org.apache.drill.exec.physical.impl.svremover.RemovingRecordBatch.innerNext():94
org.apache.drill.exec.record.AbstractRecordBatch.next():162
org.apache.drill.exec.record.AbstractRecordBatch.next():119
org.apache.drill.exec.record.AbstractRecordBatch.next():109
org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext():51

org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext():132
org.apache.drill.exec.record.AbstractRecordBatch.next():162
org.apache.drill.exec.physical.impl.BaseRootExec.next():104
org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext():81
org.apache.drill.exec.physical.impl.BaseRootExec.next():94
org.apache.drill.exec.work.fragment.FragmentExecutor$1.run():256
org.apache.drill.exec.work.fragment.FragmentExecutor$1.run():250
java.security.AccessController.doPrivileged():-2
javax.security.auth.Subject.doAs():415
org.apache.hadoop.security.UserGroupInformation.doAs():1657
org.apache.drill.exec.work.fragment.FragmentExecutor.run():250
org.apache.drill.common.SelfCleaningRunnable.run():38
java.util.concurrent.ThreadPoolExecutor.runWorker():1145
java.util.concurrent.ThreadPoolExecutor$Worker.run():615
java.lang.Thread.run():745 (state=,code=0)
{color}

> Aggregate Window Function COUNT() With GROUP BY Clause expected: range(0, 
> 32768)
> 
>
> Key: DRILL-4216
> URL: https://issues.apache.org/jira/browse/DRILL-4216
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Functions - Drill
>Affects Versions: 1.3.0
> Environment: Hadoop-2.5.2
> Hbase-0.9.15
> Java-1.7.0_85
>Reporter: PIPELINE
>Assignee: Deneche A. Hakim
>
> *When column is row_key,it work well !*
> 0: jdbc:drill:> select count(row_key) over() from hbase.web_initial_20151222 
> wi group by row_key limit 3;
> +-+
> | EXPR$0  |
> +-+
> | 102906  |
> | 102906  |
> | 102906  |
> +-+
> 3 rows selected (1.645 seconds)
> *When column is Hbase.Talbename.ColumnFamily.Qualify, and count(column) less 
> than 32768,it work well !*
> 0: jdbc:drill:> select count(wi.cf1.q5) over() from 
> hbase.web_initial_20151214 wi group by wi.cf1.q5 limit 3;
> +-+
> | EXPR$0  |
> +-+
> | 10383   |
> | 10383   |
> | 10383   |
> +-+
> 3 

[jira] [Commented] (DRILL-4216) Aggregate Window Function COUNT() With GROUP BY Clause expected: range(0, 32768)

[PIPELINE (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15069334#comment-15069334
 ] 

PIPELINE commented on DRILL-4216:
-

*drillbit.log*
2015-12-23 14:52:40,246 [2985bb47-1a0f-980d-b659-4d09abb877f7:foreman] INFO  
o.a.drill.exec.work.foreman.Foreman - Query text for query id 
2985bb47-1a0f-980d-b659-4d09abb877f7: select count(*) `count`,count(wi.cf1.q5) 
over() total from hbase.web_initial_20151223 wi group by wi.cf1.q5 limit 3
2015-12-23 14:52:40,578 [2985bb47-1a0f-980d-b659-4d09abb877f7:foreman] INFO  
o.a.d.e.s.hbase.TableStatsCalculator - Region size calculation disabled.
2015-12-23 14:52:41,076 [2985bb47-1a0f-980d-b659-4d09abb877f7:frag:3:0] INFO  
o.a.d.e.w.fragment.FragmentExecutor - 2985bb47-1a0f-980d-b659-4d09abb877f7:3:0: 
State change requested AWAITING_ALLOCATION --> RUNNING
2015-12-23 14:52:41,076 [2985bb47-1a0f-980d-b659-4d09abb877f7:frag:3:0] INFO  
o.a.d.e.w.f.FragmentStatusReporter - 2985bb47-1a0f-980d-b659-4d09abb877f7:3:0: 
State to report: RUNNING
2015-12-23 14:52:41,115 [2985bb47-1a0f-980d-b659-4d09abb877f7:frag:2:0] INFO  
o.a.d.e.p.i.p.PartitionSenderRootExec - Preliminary number of sending threads 
is: 1
2015-12-23 14:52:41,115 [2985bb47-1a0f-980d-b659-4d09abb877f7:frag:2:0] INFO  
o.a.d.e.w.fragment.FragmentExecutor - 2985bb47-1a0f-980d-b659-4d09abb877f7:2:0: 
State change requested AWAITING_ALLOCATION --> RUNNING
2015-12-23 14:52:41,115 [2985bb47-1a0f-980d-b659-4d09abb877f7:frag:2:0] INFO  
o.a.d.e.w.f.FragmentStatusReporter - 2985bb47-1a0f-980d-b659-4d09abb877f7:2:0: 
State to report: RUNNING
2015-12-23 14:52:41,123 [2985bb47-1a0f-980d-b659-4d09abb877f7:frag:1:0] INFO  
o.a.d.e.w.fragment.FragmentExecutor - 2985bb47-1a0f-980d-b659-4d09abb877f7:1:0: 
State change requested AWAITING_ALLOCATION --> RUNNING
2015-12-23 14:52:41,123 [2985bb47-1a0f-980d-b659-4d09abb877f7:frag:1:0] INFO  
o.a.d.e.w.f.FragmentStatusReporter - 2985bb47-1a0f-980d-b659-4d09abb877f7:1:0: 
State to report: RUNNING
2015-12-23 14:52:41,139 [2985bb47-1a0f-980d-b659-4d09abb877f7:frag:0:0] INFO  
o.a.d.e.w.fragment.FragmentExecutor - 2985bb47-1a0f-980d-b659-4d09abb877f7:0:0: 
State change requested AWAITING_ALLOCATION --> RUNNING
2015-12-23 14:52:41,140 [2985bb47-1a0f-980d-b659-4d09abb877f7:frag:0:0] INFO  
o.a.d.e.w.f.FragmentStatusReporter - 2985bb47-1a0f-980d-b659-4d09abb877f7:0:0: 
State to report: RUNNING
2015-12-23 14:52:42,420 [2985bb47-1a0f-980d-b659-4d09abb877f7:frag:3:0] INFO  
o.a.d.e.w.fragment.FragmentExecutor - 2985bb47-1a0f-980d-b659-4d09abb877f7:3:0: 
State change requested RUNNING --> FINISHED
2015-12-23 14:52:42,420 [2985bb47-1a0f-980d-b659-4d09abb877f7:frag:3:0] INFO  
o.a.d.e.w.f.FragmentStatusReporter - 2985bb47-1a0f-980d-b659-4d09abb877f7:3:0: 
State to report: FINISHED
2015-12-23 14:52:42,477 [2985bb47-1a0f-980d-b659-4d09abb877f7:frag:2:0] INFO  
o.a.d.e.w.fragment.FragmentExecutor - 2985bb47-1a0f-980d-b659-4d09abb877f7:2:0: 
State change requested RUNNING --> FINISHED
2015-12-23 14:52:42,478 [2985bb47-1a0f-980d-b659-4d09abb877f7:frag:2:0] INFO  
o.a.d.e.w.f.FragmentStatusReporter - 2985bb47-1a0f-980d-b659-4d09abb877f7:2:0: 
State to report: FINISHED
2015-12-23 14:52:42,481 [2985bb47-1a0f-980d-b659-4d09abb877f7:frag:1:0] INFO  
o.a.d.e.w.fragment.FragmentExecutor - 2985bb47-1a0f-980d-b659-4d09abb877f7:1:0: 
State change requested RUNNING --> FINISHED
2015-12-23 14:52:42,481 [2985bb47-1a0f-980d-b659-4d09abb877f7:frag:1:0] INFO  
o.a.d.e.w.f.FragmentStatusReporter - 2985bb47-1a0f-980d-b659-4d09abb877f7:1:0: 
State to report: FINISHED
2015-12-23 14:52:42,513 [2985bb47-1a0f-980d-b659-4d09abb877f7:frag:0:0] INFO  
o.a.d.e.w.fragment.FragmentExecutor - 2985bb47-1a0f-980d-b659-4d09abb877f7:0:0: 
State change requested RUNNING --> FAILED
2015-12-23 14:52:42,513 [2985bb47-1a0f-980d-b659-4d09abb877f7:frag:0:0] INFO  
o.a.d.e.w.fragment.FragmentExecutor - 2985bb47-1a0f-980d-b659-4d09abb877f7:0:0: 
State change requested FAILED --> FAILED
2015-12-23 14:52:42,514 [2985bb47-1a0f-980d-b659-4d09abb877f7:frag:0:0] INFO  
o.a.d.e.w.fragment.FragmentExecutor - 2985bb47-1a0f-980d-b659-4d09abb877f7:0:0: 
State change requested FAILED --> FAILED
2015-12-23 14:52:42,514 [2985bb47-1a0f-980d-b659-4d09abb877f7:frag:0:0] INFO  
o.a.d.e.w.fragment.FragmentExecutor - 2985bb47-1a0f-980d-b659-4d09abb877f7:0:0: 
State change requested FAILED --> FINISHED
2015-12-23 14:52:42,516 [2985bb47-1a0f-980d-b659-4d09abb877f7:frag:0:0] ERROR 
o.a.d.e.w.fragment.FragmentExecutor - SYSTEM ERROR: IndexOutOfBoundsException: 
index: 0, length: 102516 (expected: range(0, 32768))

Fragment 0:0

[Error Id: f811b575-d63e-4f2a-ab84-666cf162f2f8 on slave1.hadoop:31010]
org.apache.drill.common.exceptions.UserException: SYSTEM ERROR: 
IndexOutOfBoundsException: index: 0, length: 102516 (expected: range(0, 32768))

Fragment 0:0

[Error Id: f811b575-d63e-4f2a-ab84-666cf162f2f8 on slave1.hadoop:31010]
at 

[jira] [Commented] (DRILL-4216) Aggregate Window Function COUNT() With GROUP BY Clause expected: range(0, 32768)

[PIPELINE (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15069144#comment-15069144
 ] 

PIPELINE commented on DRILL-4216:
-

Just now,I update my cluster's Drill version from 1.3.0 to 1.4.0, also see the 
same exception!

> Aggregate Window Function COUNT() With GROUP BY Clause expected: range(0, 
> 32768)
> 
>
> Key: DRILL-4216
> URL: https://issues.apache.org/jira/browse/DRILL-4216
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Functions - Drill
>Affects Versions: 1.3.0
> Environment: Hadoop-2.5.2
> Hbase-0.9.15
> Java-1.7.0_85
>Reporter: PIPELINE
>Assignee: Deneche A. Hakim
>
> *When column is row_key,it work well !*
> 0: jdbc:drill:> select count(row_key) over() from hbase.web_initial_20151222 
> wi group by row_key limit 3;
> +-+
> | EXPR$0  |
> +-+
> | 102906  |
> | 102906  |
> | 102906  |
> +-+
> 3 rows selected (1.645 seconds)
> *When column is Hbase.Talbename.ColumnFamily.Qualify, and count(column) less 
> than 32768,it work well !*
> 0: jdbc:drill:> select count(wi.cf1.q5) over() from 
> hbase.web_initial_20151214 wi group by wi.cf1.q5 limit 3;
> +-+
> | EXPR$0  |
> +-+
> | 10383   |
> | 10383   |
> | 10383   |
> +-+
> 3 rows selected (1.044 seconds)
> {color:red}
> When column is Hbase.Talbename.ColumnFamily.Qualify, and count(column) 
> more than 32768,IndexOutOfBoundsException
> {color}
> 0: jdbc:drill:> select count(wi.cf1.q5) over() from 
> hbase.web_initial_20151222 wi group by wi.cf1.q5 limit 3;
> Error: SYSTEM ERROR: IndexOutOfBoundsException: index: 0, length: 62784 
> (expected: range(0, 32768))
> Fragment 0:0
> [Error Id: 77406a8a-8389-4f1b-af6c-d26d811379b7 on slave4.hadoop:31010] 
> (state=,code=0)
> java.sql.SQLException: SYSTEM ERROR: IndexOutOfBoundsException: index: 0, 
> length: 62784 (expected: range(0, 32768))
> Fragment 0:0
> [Error Id: 77406a8a-8389-4f1b-af6c-d26d811379b7 on slave4.hadoop:31010]
>   at 
> org.apache.drill.jdbc.impl.DrillCursor.nextRowInternally(DrillCursor.java:247)
>   at org.apache.drill.jdbc.impl.DrillCursor.next(DrillCursor.java:320)
>   at 
> net.hydromatic.avatica.AvaticaResultSet.next(AvaticaResultSet.java:187)
>   at 
> org.apache.drill.jdbc.impl.DrillResultSetImpl.next(DrillResultSetImpl.java:160)
>   at sqlline.IncrementalRows.hasNext(IncrementalRows.java:62)
>   at 
> sqlline.TableOutputFormat$ResizingRowsProvider.next(TableOutputFormat.java:87)
>   at sqlline.TableOutputFormat.print(TableOutputFormat.java:118)
>   at sqlline.SqlLine.print(SqlLine.java:1593)
>   at sqlline.Commands.execute(Commands.java:852)
>   at sqlline.Commands.sql(Commands.java:751)
>   at sqlline.SqlLine.dispatch(SqlLine.java:746)
>   at sqlline.SqlLine.begin(SqlLine.java:621)
>   at sqlline.SqlLine.start(SqlLine.java:375)
>   at sqlline.SqlLine.main(SqlLine.java:268)
> Caused by: org.apache.drill.common.exceptions.UserRemoteException: SYSTEM 
> ERROR: IndexOutOfBoundsException: index: 0, length: 62784 (expected: range(0, 
> 32768))
> Fragment 0:0
> [Error Id: 77406a8a-8389-4f1b-af6c-d26d811379b7 on slave4.hadoop:31010]
>   at 
> org.apache.drill.exec.rpc.user.QueryResultHandler.resultArrived(QueryResultHandler.java:118)
>   at 
> org.apache.drill.exec.rpc.user.UserClient.handleReponse(UserClient.java:112)
>   at 
> org.apache.drill.exec.rpc.BasicClientWithConnection.handle(BasicClientWithConnection.java:47)
>   at 
> org.apache.drill.exec.rpc.BasicClientWithConnection.handle(BasicClientWithConnection.java:32)
>   at org.apache.drill.exec.rpc.RpcBus.handle(RpcBus.java:69)
>   at org.apache.drill.exec.rpc.RpcBus$RequestEvent.run(RpcBus.java:400)
>   at 
> org.apache.drill.common.SerializedExecutor$RunnableProcessor.run(SerializedExecutor.java:105)
>   at 
> org.apache.drill.exec.rpc.RpcBus$SameExecutor.execute(RpcBus.java:264)
>   at 
> org.apache.drill.common.SerializedExecutor.execute(SerializedExecutor.java:142)
>   at 
> org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:298)
>   at 
> org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:269)
>   at 
> io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:89)
>   at 
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
>   at 
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
>   at 
> io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:254)
>   at 
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
>   at 
> 

[jira] [Commented] (DRILL-4216) Aggregate Window Function COUNT() With GROUP BY Clause expected: range(0, 32768)

[Deneche A. Hakim (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15069155#comment-15069155
 ] 

Deneche A. Hakim commented on DRILL-4216:
-

Curiously, the error message doesn't show the stack trace on the server side.
The error message also shows an error-id, can you search for the error id in 
drill logs and share the corresponding error message from the logs (hopefully 
it will contain the full stack trace)

> Aggregate Window Function COUNT() With GROUP BY Clause expected: range(0, 
> 32768)
> 
>
> Key: DRILL-4216
> URL: https://issues.apache.org/jira/browse/DRILL-4216
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Functions - Drill
>Affects Versions: 1.3.0
> Environment: Hadoop-2.5.2
> Hbase-0.9.15
> Java-1.7.0_85
>Reporter: PIPELINE
>Assignee: Deneche A. Hakim
>
> *When column is row_key,it work well !*
> 0: jdbc:drill:> select count(row_key) over() from hbase.web_initial_20151222 
> wi group by row_key limit 3;
> +-+
> | EXPR$0  |
> +-+
> | 102906  |
> | 102906  |
> | 102906  |
> +-+
> 3 rows selected (1.645 seconds)
> *When column is Hbase.Talbename.ColumnFamily.Qualify, and count(column) less 
> than 32768,it work well !*
> 0: jdbc:drill:> select count(wi.cf1.q5) over() from 
> hbase.web_initial_20151214 wi group by wi.cf1.q5 limit 3;
> +-+
> | EXPR$0  |
> +-+
> | 10383   |
> | 10383   |
> | 10383   |
> +-+
> 3 rows selected (1.044 seconds)
> {color:red}
> When column is Hbase.Talbename.ColumnFamily.Qualify, and count(column) 
> more than 32768,IndexOutOfBoundsException
> {color}
> 0: jdbc:drill:> select count(wi.cf1.q5) over() from 
> hbase.web_initial_20151222 wi group by wi.cf1.q5 limit 3;
> Error: SYSTEM ERROR: IndexOutOfBoundsException: index: 0, length: 62784 
> (expected: range(0, 32768))
> Fragment 0:0
> [Error Id: 77406a8a-8389-4f1b-af6c-d26d811379b7 on slave4.hadoop:31010] 
> (state=,code=0)
> java.sql.SQLException: SYSTEM ERROR: IndexOutOfBoundsException: index: 0, 
> length: 62784 (expected: range(0, 32768))
> Fragment 0:0
> [Error Id: 77406a8a-8389-4f1b-af6c-d26d811379b7 on slave4.hadoop:31010]
>   at 
> org.apache.drill.jdbc.impl.DrillCursor.nextRowInternally(DrillCursor.java:247)
>   at org.apache.drill.jdbc.impl.DrillCursor.next(DrillCursor.java:320)
>   at 
> net.hydromatic.avatica.AvaticaResultSet.next(AvaticaResultSet.java:187)
>   at 
> org.apache.drill.jdbc.impl.DrillResultSetImpl.next(DrillResultSetImpl.java:160)
>   at sqlline.IncrementalRows.hasNext(IncrementalRows.java:62)
>   at 
> sqlline.TableOutputFormat$ResizingRowsProvider.next(TableOutputFormat.java:87)
>   at sqlline.TableOutputFormat.print(TableOutputFormat.java:118)
>   at sqlline.SqlLine.print(SqlLine.java:1593)
>   at sqlline.Commands.execute(Commands.java:852)
>   at sqlline.Commands.sql(Commands.java:751)
>   at sqlline.SqlLine.dispatch(SqlLine.java:746)
>   at sqlline.SqlLine.begin(SqlLine.java:621)
>   at sqlline.SqlLine.start(SqlLine.java:375)
>   at sqlline.SqlLine.main(SqlLine.java:268)
> Caused by: org.apache.drill.common.exceptions.UserRemoteException: SYSTEM 
> ERROR: IndexOutOfBoundsException: index: 0, length: 62784 (expected: range(0, 
> 32768))
> Fragment 0:0
> [Error Id: 77406a8a-8389-4f1b-af6c-d26d811379b7 on slave4.hadoop:31010]
>   at 
> org.apache.drill.exec.rpc.user.QueryResultHandler.resultArrived(QueryResultHandler.java:118)
>   at 
> org.apache.drill.exec.rpc.user.UserClient.handleReponse(UserClient.java:112)
>   at 
> org.apache.drill.exec.rpc.BasicClientWithConnection.handle(BasicClientWithConnection.java:47)
>   at 
> org.apache.drill.exec.rpc.BasicClientWithConnection.handle(BasicClientWithConnection.java:32)
>   at org.apache.drill.exec.rpc.RpcBus.handle(RpcBus.java:69)
>   at org.apache.drill.exec.rpc.RpcBus$RequestEvent.run(RpcBus.java:400)
>   at 
> org.apache.drill.common.SerializedExecutor$RunnableProcessor.run(SerializedExecutor.java:105)
>   at 
> org.apache.drill.exec.rpc.RpcBus$SameExecutor.execute(RpcBus.java:264)
>   at 
> org.apache.drill.common.SerializedExecutor.execute(SerializedExecutor.java:142)
>   at 
> org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:298)
>   at 
> org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:269)
>   at 
> io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:89)
>   at 
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
>   at 
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
>   at 
> 

[jira] [Commented] (DRILL-4161) Make Hive Metastore client caching user configurable.

[Zelaine Fong (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15068378#comment-15068378
 ] 

Zelaine Fong commented on DRILL-4161:
-

If Fishbowl does not update their hive metastore very frequently, it is highly 
recommended that they change the following setting in their HIVE storage plugin 
configuration. The following example shows to change caching policy to 600 
seconds after every access. 
Here's an example showing the parameters that need to be changed in the Hive 
storage plugin configuration to increase the cache invalidation time to 600 
seconds, after the last time a cached object was accessed.

"configProps": {
"hive.metastore.uris": "thrift://hive.ip.address.com:9083",
... // other setting, 
 
"hive.metastore.cache-ttl-seconds": "600",
"hive.metastore.cache-expire-after": "access"
  }



> Make Hive Metastore client caching user configurable.
> -
>
> Key: DRILL-4161
> URL: https://issues.apache.org/jira/browse/DRILL-4161
> Project: Apache Drill
>  Issue Type: Bug
>Reporter: Jinfeng Ni
> Fix For: 1.5.0
>
>
> Drill leverages LoadingCache in hive metastore client, in order to avoid the 
> long access time to hive metastore server. However, there is a tradeoff 
> between caching stale data and the possibility of cache hit. 
> For instance, DRILL-3893 changes cache invalidation policy to "1 minute after 
> last write", to avoid the chances of hitting stale data. However, it also 
> implies that the cached data would be only valid for 1 minute after 
> loading/write.
> It's desirable to allow user to configure the caching policy, per their 
> individual use case requirement. In particular, we probably should allow user 
> to specify:
> 1) caching invalidation policy : expire after last access, or expire after 
> last write.
> 2) cache TTL.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-4020) The not-equal operator returns incorrect results when used on the HBase row key

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15068379#comment-15068379
 ] 

ASF GitHub Bot commented on DRILL-4020:
---

GitHub user nagix opened a pull request:

https://github.com/apache/drill/pull/309

DRILL-4020: The not-equal operator returns incorrect results when used on 
the HBase row key

- Added a condition that checks if the filter to the scan specification 
doesn't have NOT_EQUAL operator
- Added testFilterPushDownRowKeyNotEqual() to TestHBaseFilterPushDown

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/nagix/drill drill-4020

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/drill/pull/309.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #309


commit edac8b0815377d19978cf4e352206bcabf63d12c
Author: Akihiko Kusanagi 
Date:   2015-11-04T12:33:06Z

DRILL-4020: The not-equal operator returns incorrect results when used on 
the HBase row key

- Added a condition that checks if the filter to the scan specification 
doesn't have NOT_EQUAL operator
- Added testFilterPushDownRowKeyNotEqual() to TestHBaseFilterPushDown




> The not-equal operator returns incorrect results when used on the HBase row 
> key
> ---
>
> Key: DRILL-4020
> URL: https://issues.apache.org/jira/browse/DRILL-4020
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Storage - HBase
>Affects Versions: 1.2.0
> Environment: Drill Sandbox
>Reporter: Akihiko Kusanagi
>Priority: Critical
> Attachments: DRILL-4020.1.patch.txt
>
>
> Create a test HBase table:
> {noformat}
> hbase> create 'table', 'f'
> hbase> put 'table', 'row1', 'f:c', 'value1'
> hbase> put 'table', 'row2', 'f:c', 'value2'
> hbase> put 'table', 'row3', 'f:c', 'value3'
> {noformat}
> The table looks like this:
> {noformat}
> 0: jdbc:drill:zk=maprdemo:5181> SELECT CONVERT_FROM(row_key, 'UTF8') FROM 
> hbase.`table`;
> +-+
> | EXPR$0  |
> +-+
> | row1|
> | row2|
> | row3|
> +-+
> 1 row selected (4.596 seconds)
> {noformat}
> However, this query returns incorrect results when a not-equal operator is 
> used on the row key:
> {noformat}
> 0: jdbc:drill:zk=maprdemo:5181> SELECT CONVERT_FROM(row_key, 'UTF8') FROM 
> hbase.`table` WHERE row_key <> 'row1';
> +-+
> | EXPR$0  |
> +-+
> | row1|
> | row2|
> | row3|
> +-+
> 1 row selected (0.573 seconds)
> {noformat}
> In the query plan, there is no RowFilter:
> {noformat}
> 00-00Screen
> 00-01  Project(EXPR$0=[CONVERT_FROMUTF8($0)])
> 00-02Scan(groupscan=[HBaseGroupScan [HBaseScanSpec=HBaseScanSpec 
> [tableName=table, startRow=, stopRow=, filter=null], columns=[`row_key`]]])
> {noformat}
> When the query has multiple not-equal operators, it works fine:
> {noformat}
> 0: jdbc:drill:zk=maprdemo:5181> SELECT CONVERT_FROM(row_key, 'UTF8') FROM 
> hbase.`table` WHERE row_key <> 'row1' AND row_key <> 'row2';
> +-+
> | EXPR$0  |
> +-+
> | row3|
> +-+
> 1 row selected (0.255 seconds)
> {noformat}
> In the query plan, a FilterList has two RowFilters with NOT_EQUAL operators:
> {noformat}
> 00-00Screen
> 00-01  Project(EXPR$0=[CONVERT_FROMUTF8($0)])
> 00-02Scan(groupscan=[HBaseGroupScan [HBaseScanSpec=HBaseScanSpec 
> [tableName=table, startRow=, stopRow=, filter=FilterList AND (2/2): 
> [RowFilter (NOT_EQUAL, row1), RowFilter (NOT_EQUAL, row2)]], 
> columns=[`row_key`]]])
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-4020) The not-equal operator returns incorrect results when used on the HBase row key

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15068309#comment-15068309
 ] 

ASF GitHub Bot commented on DRILL-4020:
---

Github user jacques-n commented on the pull request:

https://github.com/apache/drill/pull/238#issuecomment-166658431
  
Maybe just submit new request. I can't reopen either.
On Dec 22, 2015 7:30 AM, "Akihiko Kusanagi" 
wrote:

> @jacques-n , I don't believe I can re-open
> this because I didn't close it. Can you do it or do l need to open a new
> request?
>
> —
> Reply to this email directly or view it on GitHub
> .
>



> The not-equal operator returns incorrect results when used on the HBase row 
> key
> ---
>
> Key: DRILL-4020
> URL: https://issues.apache.org/jira/browse/DRILL-4020
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Storage - HBase
>Affects Versions: 1.2.0
> Environment: Drill Sandbox
>Reporter: Akihiko Kusanagi
>Priority: Critical
> Attachments: DRILL-4020.1.patch.txt
>
>
> Create a test HBase table:
> {noformat}
> hbase> create 'table', 'f'
> hbase> put 'table', 'row1', 'f:c', 'value1'
> hbase> put 'table', 'row2', 'f:c', 'value2'
> hbase> put 'table', 'row3', 'f:c', 'value3'
> {noformat}
> The table looks like this:
> {noformat}
> 0: jdbc:drill:zk=maprdemo:5181> SELECT CONVERT_FROM(row_key, 'UTF8') FROM 
> hbase.`table`;
> +-+
> | EXPR$0  |
> +-+
> | row1|
> | row2|
> | row3|
> +-+
> 1 row selected (4.596 seconds)
> {noformat}
> However, this query returns incorrect results when a not-equal operator is 
> used on the row key:
> {noformat}
> 0: jdbc:drill:zk=maprdemo:5181> SELECT CONVERT_FROM(row_key, 'UTF8') FROM 
> hbase.`table` WHERE row_key <> 'row1';
> +-+
> | EXPR$0  |
> +-+
> | row1|
> | row2|
> | row3|
> +-+
> 1 row selected (0.573 seconds)
> {noformat}
> In the query plan, there is no RowFilter:
> {noformat}
> 00-00Screen
> 00-01  Project(EXPR$0=[CONVERT_FROMUTF8($0)])
> 00-02Scan(groupscan=[HBaseGroupScan [HBaseScanSpec=HBaseScanSpec 
> [tableName=table, startRow=, stopRow=, filter=null], columns=[`row_key`]]])
> {noformat}
> When the query has multiple not-equal operators, it works fine:
> {noformat}
> 0: jdbc:drill:zk=maprdemo:5181> SELECT CONVERT_FROM(row_key, 'UTF8') FROM 
> hbase.`table` WHERE row_key <> 'row1' AND row_key <> 'row2';
> +-+
> | EXPR$0  |
> +-+
> | row3|
> +-+
> 1 row selected (0.255 seconds)
> {noformat}
> In the query plan, a FilterList has two RowFilters with NOT_EQUAL operators:
> {noformat}
> 00-00Screen
> 00-01  Project(EXPR$0=[CONVERT_FROMUTF8($0)])
> 00-02Scan(groupscan=[HBaseGroupScan [HBaseScanSpec=HBaseScanSpec 
> [tableName=table, startRow=, stopRow=, filter=FilterList AND (2/2): 
> [RowFilter (NOT_EQUAL, row1), RowFilter (NOT_EQUAL, row2)]], 
> columns=[`row_key`]]])
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (DRILL-4161) Make Hive Metastore client caching user configurable.

[Zelaine Fong (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15068378#comment-15068378
 ] 

Zelaine Fong edited comment on DRILL-4161 at 12/22/15 4:51 PM:
---

The following example shows to change caching policy to 600 seconds after every 
access. 
Here's an example showing the parameters that need to be changed in the Hive 
storage plugin configuration to increase the cache invalidation time to 600 
seconds, after the last time a cached object was accessed.

"configProps": {
"hive.metastore.uris": "thrift://hive.ip.address.com:9083",
... // other setting, 
 
"hive.metastore.cache-ttl-seconds": "600",
"hive.metastore.cache-expire-after": "access"
  }




was (Author: zfong):
If Fishbowl does not update their hive metastore very frequently, it is highly 
recommended that they change the following setting in their HIVE storage plugin 
configuration. The following example shows to change caching policy to 600 
seconds after every access. 
Here's an example showing the parameters that need to be changed in the Hive 
storage plugin configuration to increase the cache invalidation time to 600 
seconds, after the last time a cached object was accessed.

"configProps": {
"hive.metastore.uris": "thrift://hive.ip.address.com:9083",
... // other setting, 
 
"hive.metastore.cache-ttl-seconds": "600",
"hive.metastore.cache-expire-after": "access"
  }



> Make Hive Metastore client caching user configurable.
> -
>
> Key: DRILL-4161
> URL: https://issues.apache.org/jira/browse/DRILL-4161
> Project: Apache Drill
>  Issue Type: Bug
>Reporter: Jinfeng Ni
> Fix For: 1.5.0
>
>
> Drill leverages LoadingCache in hive metastore client, in order to avoid the 
> long access time to hive metastore server. However, there is a tradeoff 
> between caching stale data and the possibility of cache hit. 
> For instance, DRILL-3893 changes cache invalidation policy to "1 minute after 
> last write", to avoid the chances of hitting stale data. However, it also 
> implies that the cached data would be only valid for 1 minute after 
> loading/write.
> It's desirable to allow user to configure the caching policy, per their 
> individual use case requirement. In particular, we probably should allow user 
> to specify:
> 1) caching invalidation policy : expire after last access, or expire after 
> last write.
> 2) cache TTL.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (DRILL-4161) Make Hive Metastore client caching user configurable.

[Zelaine Fong (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15068378#comment-15068378
 ] 

Zelaine Fong edited comment on DRILL-4161 at 12/22/15 4:56 PM:
---

The following example shows how to change caching policy to 600 seconds after 
every access. 
Here's an example showing the parameters that need to be changed in the Hive 
storage plugin configuration to increase the cache invalidation time to 600 
seconds, after the last time a cached object was accessed.
{code}
"configProps": {
"hive.metastore.uris": "thrift://hive.ip.address.com:9083",
... // other setting, 
 
"hive.metastore.cache-ttl-seconds": "600",
"hive.metastore.cache-expire-after": "access"
  }
{code}



was (Author: zfong):
The following example shows how to change caching policy to 600 seconds after 
every access. 
Here's an example showing the parameters that need to be changed in the Hive 
storage plugin configuration to increase the cache invalidation time to 600 
seconds, after the last time a cached object was accessed.


"configProps": {
"hive.metastore.uris": "thrift://hive.ip.address.com:9083",
... // other setting, 
 
"hive.metastore.cache-ttl-seconds": "600",
"hive.metastore.cache-expire-after": "access"
  }




> Make Hive Metastore client caching user configurable.
> -
>
> Key: DRILL-4161
> URL: https://issues.apache.org/jira/browse/DRILL-4161
> Project: Apache Drill
>  Issue Type: Bug
>Reporter: Jinfeng Ni
> Fix For: 1.5.0
>
>
> Drill leverages LoadingCache in hive metastore client, in order to avoid the 
> long access time to hive metastore server. However, there is a tradeoff 
> between caching stale data and the possibility of cache hit. 
> For instance, DRILL-3893 changes cache invalidation policy to "1 minute after 
> last write", to avoid the chances of hitting stale data. However, it also 
> implies that the cached data would be only valid for 1 minute after 
> loading/write.
> It's desirable to allow user to configure the caching policy, per their 
> individual use case requirement. In particular, we probably should allow user 
> to specify:
> 1) caching invalidation policy : expire after last access, or expire after 
> last write.
> 2) cache TTL.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-4020) The not-equal operator returns incorrect results when used on the HBase row key

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15068385#comment-15068385
 ] 

ASF GitHub Bot commented on DRILL-4020:
---

Github user nagix commented on the pull request:

https://github.com/apache/drill/pull/238#issuecomment-166670829
  
Submitted a new request #309 


> The not-equal operator returns incorrect results when used on the HBase row 
> key
> ---
>
> Key: DRILL-4020
> URL: https://issues.apache.org/jira/browse/DRILL-4020
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Storage - HBase
>Affects Versions: 1.2.0
> Environment: Drill Sandbox
>Reporter: Akihiko Kusanagi
>Priority: Critical
> Attachments: DRILL-4020.1.patch.txt
>
>
> Create a test HBase table:
> {noformat}
> hbase> create 'table', 'f'
> hbase> put 'table', 'row1', 'f:c', 'value1'
> hbase> put 'table', 'row2', 'f:c', 'value2'
> hbase> put 'table', 'row3', 'f:c', 'value3'
> {noformat}
> The table looks like this:
> {noformat}
> 0: jdbc:drill:zk=maprdemo:5181> SELECT CONVERT_FROM(row_key, 'UTF8') FROM 
> hbase.`table`;
> +-+
> | EXPR$0  |
> +-+
> | row1|
> | row2|
> | row3|
> +-+
> 1 row selected (4.596 seconds)
> {noformat}
> However, this query returns incorrect results when a not-equal operator is 
> used on the row key:
> {noformat}
> 0: jdbc:drill:zk=maprdemo:5181> SELECT CONVERT_FROM(row_key, 'UTF8') FROM 
> hbase.`table` WHERE row_key <> 'row1';
> +-+
> | EXPR$0  |
> +-+
> | row1|
> | row2|
> | row3|
> +-+
> 1 row selected (0.573 seconds)
> {noformat}
> In the query plan, there is no RowFilter:
> {noformat}
> 00-00Screen
> 00-01  Project(EXPR$0=[CONVERT_FROMUTF8($0)])
> 00-02Scan(groupscan=[HBaseGroupScan [HBaseScanSpec=HBaseScanSpec 
> [tableName=table, startRow=, stopRow=, filter=null], columns=[`row_key`]]])
> {noformat}
> When the query has multiple not-equal operators, it works fine:
> {noformat}
> 0: jdbc:drill:zk=maprdemo:5181> SELECT CONVERT_FROM(row_key, 'UTF8') FROM 
> hbase.`table` WHERE row_key <> 'row1' AND row_key <> 'row2';
> +-+
> | EXPR$0  |
> +-+
> | row3|
> +-+
> 1 row selected (0.255 seconds)
> {noformat}
> In the query plan, a FilterList has two RowFilters with NOT_EQUAL operators:
> {noformat}
> 00-00Screen
> 00-01  Project(EXPR$0=[CONVERT_FROMUTF8($0)])
> 00-02Scan(groupscan=[HBaseGroupScan [HBaseScanSpec=HBaseScanSpec 
> [tableName=table, startRow=, stopRow=, filter=FilterList AND (2/2): 
> [RowFilter (NOT_EQUAL, row1), RowFilter (NOT_EQUAL, row2)]], 
> columns=[`row_key`]]])
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (DRILL-4126) Adding HiveMetaStore caching when impersonation is enabled.

[Zelaine Fong (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15068386#comment-15068386
 ] 

Zelaine Fong commented on DRILL-4126:
-

See DRILL-4217.  The changes there may also be needed to ensure that cache 
objects are kept in the cache long enough to be beneficial.  The prior existing 
hard-coded invalidation period of 1 minute may result in not seeing benefits 
from caching.

> Adding HiveMetaStore caching when impersonation is enabled. 
> 
>
> Key: DRILL-4126
> URL: https://issues.apache.org/jira/browse/DRILL-4126
> Project: Apache Drill
>  Issue Type: Bug
>Reporter: Jinfeng Ni
>Assignee: Jinfeng Ni
> Fix For: 1.5.0
>
>
> Currently, HiveMetastore caching is used only when impersonation is disabled, 
> such that all the hivemetastore call goes through 
> NonCloseableHiveClientWithCaching [1]. However, if impersonation is enabled, 
> caching is not used for HiveMetastore access.
> This could significantly increase the planning time when hive storage plugin 
> is enabled, or when running a query against INFORMATION_SCHEMA. Depending on 
> the # of databases/tables in Hive storage plugin, the planning time or 
> INFORMATION_SCHEMA query could become unacceptable. This becomes even worse 
> if the hive metastore is running on a different node from drillbit, making 
> the access of hivemetastore even slower.
> We are seeing that it could takes 30~60 seconds for planning time, or 
> execution time for INFORMATION_SCHEMA query.  The long planning or execution 
> time for INFORMATION_SCHEMA query prevents Drill from acting "interactively" 
> for such queries. 
> We should enable caching when impersonation is used. As long as the 
> authorizer verifies the user has the access to databases/tables, we should 
> get the data from caching. By doing that, we should see reduced number of api 
> call to HiveMetaStore.
> [1] 
> https://github.com/apache/drill/blob/master/contrib/storage-hive/core/src/main/java/org/apache/drill/exec/store/hive/DrillHiveMetaStoreClient.java#L299



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (DRILL-4161) Make Hive Metastore client caching user configurable.

[Zelaine Fong (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15068378#comment-15068378
 ] 

Zelaine Fong edited comment on DRILL-4161 at 12/22/15 4:51 PM:
---

The following example shows how to change caching policy to 600 seconds after 
every access. 
Here's an example showing the parameters that need to be changed in the Hive 
storage plugin configuration to increase the cache invalidation time to 600 
seconds, after the last time a cached object was accessed.

"configProps": {
"hive.metastore.uris": "thrift://hive.ip.address.com:9083",
... // other setting, 
 
"hive.metastore.cache-ttl-seconds": "600",
"hive.metastore.cache-expire-after": "access"
  }




was (Author: zfong):
The following example shows to change caching policy to 600 seconds after every 
access. 
Here's an example showing the parameters that need to be changed in the Hive 
storage plugin configuration to increase the cache invalidation time to 600 
seconds, after the last time a cached object was accessed.

"configProps": {
"hive.metastore.uris": "thrift://hive.ip.address.com:9083",
... // other setting, 
 
"hive.metastore.cache-ttl-seconds": "600",
"hive.metastore.cache-expire-after": "access"
  }



> Make Hive Metastore client caching user configurable.
> -
>
> Key: DRILL-4161
> URL: https://issues.apache.org/jira/browse/DRILL-4161
> Project: Apache Drill
>  Issue Type: Bug
>Reporter: Jinfeng Ni
> Fix For: 1.5.0
>
>
> Drill leverages LoadingCache in hive metastore client, in order to avoid the 
> long access time to hive metastore server. However, there is a tradeoff 
> between caching stale data and the possibility of cache hit. 
> For instance, DRILL-3893 changes cache invalidation policy to "1 minute after 
> last write", to avoid the chances of hitting stale data. However, it also 
> implies that the cached data would be only valid for 1 minute after 
> loading/write.
> It's desirable to allow user to configure the caching policy, per their 
> individual use case requirement. In particular, we probably should allow user 
> to specify:
> 1) caching invalidation policy : expire after last access, or expire after 
> last write.
> 2) cache TTL.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (DRILL-4161) Make Hive Metastore client caching user configurable.

[Zelaine Fong (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15068378#comment-15068378
 ] 

Zelaine Fong edited comment on DRILL-4161 at 12/22/15 4:53 PM:
---

The following example shows how to change caching policy to 600 seconds after 
every access. 
Here's an example showing the parameters that need to be changed in the Hive 
storage plugin configuration to increase the cache invalidation time to 600 
seconds, after the last time a cached object was accessed.


"configProps": {
"hive.metastore.uris": "thrift://hive.ip.address.com:9083",
... // other setting, 
 
"hive.metastore.cache-ttl-seconds": "600",
"hive.metastore.cache-expire-after": "access"
  }





was (Author: zfong):
The following example shows how to change caching policy to 600 seconds after 
every access. 
Here's an example showing the parameters that need to be changed in the Hive 
storage plugin configuration to increase the cache invalidation time to 600 
seconds, after the last time a cached object was accessed.

"configProps": {
"hive.metastore.uris": "thrift://hive.ip.address.com:9083",
... // other setting, 
 
"hive.metastore.cache-ttl-seconds": "600",
"hive.metastore.cache-expire-after": "access"
  }



> Make Hive Metastore client caching user configurable.
> -
>
> Key: DRILL-4161
> URL: https://issues.apache.org/jira/browse/DRILL-4161
> Project: Apache Drill
>  Issue Type: Bug
>Reporter: Jinfeng Ni
> Fix For: 1.5.0
>
>
> Drill leverages LoadingCache in hive metastore client, in order to avoid the 
> long access time to hive metastore server. However, there is a tradeoff 
> between caching stale data and the possibility of cache hit. 
> For instance, DRILL-3893 changes cache invalidation policy to "1 minute after 
> last write", to avoid the chances of hitting stale data. However, it also 
> implies that the cached data would be only valid for 1 minute after 
> loading/write.
> It's desirable to allow user to configure the caching policy, per their 
> individual use case requirement. In particular, we probably should allow user 
> to specify:
> 1) caching invalidation policy : expire after last access, or expire after 
> last write.
> 2) cache TTL.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (DRILL-4161) Make Hive Metastore client caching user configurable.

[Zelaine Fong (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15068378#comment-15068378
 ] 

Zelaine Fong edited comment on DRILL-4161 at 12/22/15 4:59 PM:
---

You'll need to change a couple of parameters in the Hive storage plugin 
configuration to leverage the changes in this Jira.

Here's an example showing the parameters that need to be changed to increase 
the cache invalidation time to 600 seconds, after the last time a cached object 
was accessed.
{code}
"configProps": {
"hive.metastore.uris": "thrift://hive.ip.address.com:9083",
... // other setting, 
 
"hive.metastore.cache-ttl-seconds": "600",
"hive.metastore.cache-expire-after": "access"
  }
{code}



was (Author: zfong):
The following example shows how to change caching policy to 600 seconds after 
every access. 
Here's an example showing the parameters that need to be changed in the Hive 
storage plugin configuration to increase the cache invalidation time to 600 
seconds, after the last time a cached object was accessed.
{code}
"configProps": {
"hive.metastore.uris": "thrift://hive.ip.address.com:9083",
... // other setting, 
 
"hive.metastore.cache-ttl-seconds": "600",
"hive.metastore.cache-expire-after": "access"
  }
{code}


> Make Hive Metastore client caching user configurable.
> -
>
> Key: DRILL-4161
> URL: https://issues.apache.org/jira/browse/DRILL-4161
> Project: Apache Drill
>  Issue Type: Bug
>Reporter: Jinfeng Ni
> Fix For: 1.5.0
>
>
> Drill leverages LoadingCache in hive metastore client, in order to avoid the 
> long access time to hive metastore server. However, there is a tradeoff 
> between caching stale data and the possibility of cache hit. 
> For instance, DRILL-3893 changes cache invalidation policy to "1 minute after 
> last write", to avoid the chances of hitting stale data. However, it also 
> implies that the cached data would be only valid for 1 minute after 
> loading/write.
> It's desirable to allow user to configure the caching policy, per their 
> individual use case requirement. In particular, we probably should allow user 
> to specify:
> 1) caching invalidation policy : expire after last access, or expire after 
> last write.
> 2) cache TTL.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (DRILL-4216) Aggregate Window Function COUNT() With GROUP BY Clause expected: range(0, 32768)

[Deneche A. Hakim (JIRA)]

 [ 
https://issues.apache.org/jira/browse/DRILL-4216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deneche A. Hakim reassigned DRILL-4216:
---

Assignee: Deneche A. Hakim

> Aggregate Window Function COUNT() With GROUP BY Clause expected: range(0, 
> 32768)
> 
>
> Key: DRILL-4216
> URL: https://issues.apache.org/jira/browse/DRILL-4216
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Functions - Drill
>Affects Versions: 1.3.0
> Environment: Hadoop-2.5.2
> Hbase-0.9.15
> Java-1.7.0_85
>Reporter: PIPELINE
>Assignee: Deneche A. Hakim
>
> *When column is row_key,it work well !*
> 0: jdbc:drill:> select count(row_key) over() from hbase.web_initial_20151222 
> wi group by row_key limit 3;
> +-+
> | EXPR$0  |
> +-+
> | 102906  |
> | 102906  |
> | 102906  |
> +-+
> 3 rows selected (1.645 seconds)
> *When column is Hbase.Talbename.ColumnFamily.Qualify, and count(column) less 
> than 32768,it work well !*
> 0: jdbc:drill:> select count(wi.cf1.q5) over() from 
> hbase.web_initial_20151214 wi group by wi.cf1.q5 limit 3;
> +-+
> | EXPR$0  |
> +-+
> | 10383   |
> | 10383   |
> | 10383   |
> +-+
> 3 rows selected (1.044 seconds)
> {color:red}
> When column is Hbase.Talbename.ColumnFamily.Qualify, and count(column) 
> more than 32768,IndexOutOfBoundsException
> {color}
> 0: jdbc:drill:> select count(wi.cf1.q5) over() from 
> hbase.web_initial_20151222 wi group by wi.cf1.q5 limit 3;
> Error: SYSTEM ERROR: IndexOutOfBoundsException: index: 0, length: 62784 
> (expected: range(0, 32768))
> Fragment 0:0
> [Error Id: 77406a8a-8389-4f1b-af6c-d26d811379b7 on slave4.hadoop:31010] 
> (state=,code=0)
> java.sql.SQLException: SYSTEM ERROR: IndexOutOfBoundsException: index: 0, 
> length: 62784 (expected: range(0, 32768))
> Fragment 0:0
> [Error Id: 77406a8a-8389-4f1b-af6c-d26d811379b7 on slave4.hadoop:31010]
>   at 
> org.apache.drill.jdbc.impl.DrillCursor.nextRowInternally(DrillCursor.java:247)
>   at org.apache.drill.jdbc.impl.DrillCursor.next(DrillCursor.java:320)
>   at 
> net.hydromatic.avatica.AvaticaResultSet.next(AvaticaResultSet.java:187)
>   at 
> org.apache.drill.jdbc.impl.DrillResultSetImpl.next(DrillResultSetImpl.java:160)
>   at sqlline.IncrementalRows.hasNext(IncrementalRows.java:62)
>   at 
> sqlline.TableOutputFormat$ResizingRowsProvider.next(TableOutputFormat.java:87)
>   at sqlline.TableOutputFormat.print(TableOutputFormat.java:118)
>   at sqlline.SqlLine.print(SqlLine.java:1593)
>   at sqlline.Commands.execute(Commands.java:852)
>   at sqlline.Commands.sql(Commands.java:751)
>   at sqlline.SqlLine.dispatch(SqlLine.java:746)
>   at sqlline.SqlLine.begin(SqlLine.java:621)
>   at sqlline.SqlLine.start(SqlLine.java:375)
>   at sqlline.SqlLine.main(SqlLine.java:268)
> Caused by: org.apache.drill.common.exceptions.UserRemoteException: SYSTEM 
> ERROR: IndexOutOfBoundsException: index: 0, length: 62784 (expected: range(0, 
> 32768))
> Fragment 0:0
> [Error Id: 77406a8a-8389-4f1b-af6c-d26d811379b7 on slave4.hadoop:31010]
>   at 
> org.apache.drill.exec.rpc.user.QueryResultHandler.resultArrived(QueryResultHandler.java:118)
>   at 
> org.apache.drill.exec.rpc.user.UserClient.handleReponse(UserClient.java:112)
>   at 
> org.apache.drill.exec.rpc.BasicClientWithConnection.handle(BasicClientWithConnection.java:47)
>   at 
> org.apache.drill.exec.rpc.BasicClientWithConnection.handle(BasicClientWithConnection.java:32)
>   at org.apache.drill.exec.rpc.RpcBus.handle(RpcBus.java:69)
>   at org.apache.drill.exec.rpc.RpcBus$RequestEvent.run(RpcBus.java:400)
>   at 
> org.apache.drill.common.SerializedExecutor$RunnableProcessor.run(SerializedExecutor.java:105)
>   at 
> org.apache.drill.exec.rpc.RpcBus$SameExecutor.execute(RpcBus.java:264)
>   at 
> org.apache.drill.common.SerializedExecutor.execute(SerializedExecutor.java:142)
>   at 
> org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:298)
>   at 
> org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:269)
>   at 
> io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:89)
>   at 
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
>   at 
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
>   at 
> io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:254)
>   at 
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
>   at 
> 

[jira] [Commented] (DRILL-4216) Aggregate Window Function COUNT() With GROUP BY Clause expected: range(0, 32768)

[Deneche A. Hakim (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15068412#comment-15068412
 ] 

Deneche A. Hakim commented on DRILL-4216:
-

[~PIPE] can you run the failing query again after setting verbose errors to 
true:
{noformat}
set `exec.errors.verbose`=true;
{noformat}

This should give us the stack trace on the server side.

Thanks

> Aggregate Window Function COUNT() With GROUP BY Clause expected: range(0, 
> 32768)
> 
>
> Key: DRILL-4216
> URL: https://issues.apache.org/jira/browse/DRILL-4216
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Functions - Drill
>Affects Versions: 1.3.0
> Environment: Hadoop-2.5.2
> Hbase-0.9.15
> Java-1.7.0_85
>Reporter: PIPELINE
>Assignee: Deneche A. Hakim
>
> *When column is row_key,it work well !*
> 0: jdbc:drill:> select count(row_key) over() from hbase.web_initial_20151222 
> wi group by row_key limit 3;
> +-+
> | EXPR$0  |
> +-+
> | 102906  |
> | 102906  |
> | 102906  |
> +-+
> 3 rows selected (1.645 seconds)
> *When column is Hbase.Talbename.ColumnFamily.Qualify, and count(column) less 
> than 32768,it work well !*
> 0: jdbc:drill:> select count(wi.cf1.q5) over() from 
> hbase.web_initial_20151214 wi group by wi.cf1.q5 limit 3;
> +-+
> | EXPR$0  |
> +-+
> | 10383   |
> | 10383   |
> | 10383   |
> +-+
> 3 rows selected (1.044 seconds)
> {color:red}
> When column is Hbase.Talbename.ColumnFamily.Qualify, and count(column) 
> more than 32768,IndexOutOfBoundsException
> {color}
> 0: jdbc:drill:> select count(wi.cf1.q5) over() from 
> hbase.web_initial_20151222 wi group by wi.cf1.q5 limit 3;
> Error: SYSTEM ERROR: IndexOutOfBoundsException: index: 0, length: 62784 
> (expected: range(0, 32768))
> Fragment 0:0
> [Error Id: 77406a8a-8389-4f1b-af6c-d26d811379b7 on slave4.hadoop:31010] 
> (state=,code=0)
> java.sql.SQLException: SYSTEM ERROR: IndexOutOfBoundsException: index: 0, 
> length: 62784 (expected: range(0, 32768))
> Fragment 0:0
> [Error Id: 77406a8a-8389-4f1b-af6c-d26d811379b7 on slave4.hadoop:31010]
>   at 
> org.apache.drill.jdbc.impl.DrillCursor.nextRowInternally(DrillCursor.java:247)
>   at org.apache.drill.jdbc.impl.DrillCursor.next(DrillCursor.java:320)
>   at 
> net.hydromatic.avatica.AvaticaResultSet.next(AvaticaResultSet.java:187)
>   at 
> org.apache.drill.jdbc.impl.DrillResultSetImpl.next(DrillResultSetImpl.java:160)
>   at sqlline.IncrementalRows.hasNext(IncrementalRows.java:62)
>   at 
> sqlline.TableOutputFormat$ResizingRowsProvider.next(TableOutputFormat.java:87)
>   at sqlline.TableOutputFormat.print(TableOutputFormat.java:118)
>   at sqlline.SqlLine.print(SqlLine.java:1593)
>   at sqlline.Commands.execute(Commands.java:852)
>   at sqlline.Commands.sql(Commands.java:751)
>   at sqlline.SqlLine.dispatch(SqlLine.java:746)
>   at sqlline.SqlLine.begin(SqlLine.java:621)
>   at sqlline.SqlLine.start(SqlLine.java:375)
>   at sqlline.SqlLine.main(SqlLine.java:268)
> Caused by: org.apache.drill.common.exceptions.UserRemoteException: SYSTEM 
> ERROR: IndexOutOfBoundsException: index: 0, length: 62784 (expected: range(0, 
> 32768))
> Fragment 0:0
> [Error Id: 77406a8a-8389-4f1b-af6c-d26d811379b7 on slave4.hadoop:31010]
>   at 
> org.apache.drill.exec.rpc.user.QueryResultHandler.resultArrived(QueryResultHandler.java:118)
>   at 
> org.apache.drill.exec.rpc.user.UserClient.handleReponse(UserClient.java:112)
>   at 
> org.apache.drill.exec.rpc.BasicClientWithConnection.handle(BasicClientWithConnection.java:47)
>   at 
> org.apache.drill.exec.rpc.BasicClientWithConnection.handle(BasicClientWithConnection.java:32)
>   at org.apache.drill.exec.rpc.RpcBus.handle(RpcBus.java:69)
>   at org.apache.drill.exec.rpc.RpcBus$RequestEvent.run(RpcBus.java:400)
>   at 
> org.apache.drill.common.SerializedExecutor$RunnableProcessor.run(SerializedExecutor.java:105)
>   at 
> org.apache.drill.exec.rpc.RpcBus$SameExecutor.execute(RpcBus.java:264)
>   at 
> org.apache.drill.common.SerializedExecutor.execute(SerializedExecutor.java:142)
>   at 
> org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:298)
>   at 
> org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:269)
>   at 
> io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:89)
>   at 
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
>   at 
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
>   at 
> io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:254)
>   at 
> 

[jira] [Commented] (DRILL-4216) Aggregate Window Function COUNT() With GROUP BY Clause expected: range(0, 32768)

[Deneche A. Hakim (JIRA)]

[ 
https://issues.apache.org/jira/browse/DRILL-4216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15068452#comment-15068452
 ] 

Deneche A. Hakim commented on DRILL-4216:
-

as a side note, window functions are applied after group by so "count() over()" 
in your query will basically just count the number of groups returned by "group 
by row_key" 

> Aggregate Window Function COUNT() With GROUP BY Clause expected: range(0, 
> 32768)
> 
>
> Key: DRILL-4216
> URL: https://issues.apache.org/jira/browse/DRILL-4216
> Project: Apache Drill
>  Issue Type: Bug
>  Components: Functions - Drill
>Affects Versions: 1.3.0
> Environment: Hadoop-2.5.2
> Hbase-0.9.15
> Java-1.7.0_85
>Reporter: PIPELINE
>Assignee: Deneche A. Hakim
>
> *When column is row_key,it work well !*
> 0: jdbc:drill:> select count(row_key) over() from hbase.web_initial_20151222 
> wi group by row_key limit 3;
> +-+
> | EXPR$0  |
> +-+
> | 102906  |
> | 102906  |
> | 102906  |
> +-+
> 3 rows selected (1.645 seconds)
> *When column is Hbase.Talbename.ColumnFamily.Qualify, and count(column) less 
> than 32768,it work well !*
> 0: jdbc:drill:> select count(wi.cf1.q5) over() from 
> hbase.web_initial_20151214 wi group by wi.cf1.q5 limit 3;
> +-+
> | EXPR$0  |
> +-+
> | 10383   |
> | 10383   |
> | 10383   |
> +-+
> 3 rows selected (1.044 seconds)
> {color:red}
> When column is Hbase.Talbename.ColumnFamily.Qualify, and count(column) 
> more than 32768,IndexOutOfBoundsException
> {color}
> 0: jdbc:drill:> select count(wi.cf1.q5) over() from 
> hbase.web_initial_20151222 wi group by wi.cf1.q5 limit 3;
> Error: SYSTEM ERROR: IndexOutOfBoundsException: index: 0, length: 62784 
> (expected: range(0, 32768))
> Fragment 0:0
> [Error Id: 77406a8a-8389-4f1b-af6c-d26d811379b7 on slave4.hadoop:31010] 
> (state=,code=0)
> java.sql.SQLException: SYSTEM ERROR: IndexOutOfBoundsException: index: 0, 
> length: 62784 (expected: range(0, 32768))
> Fragment 0:0
> [Error Id: 77406a8a-8389-4f1b-af6c-d26d811379b7 on slave4.hadoop:31010]
>   at 
> org.apache.drill.jdbc.impl.DrillCursor.nextRowInternally(DrillCursor.java:247)
>   at org.apache.drill.jdbc.impl.DrillCursor.next(DrillCursor.java:320)
>   at 
> net.hydromatic.avatica.AvaticaResultSet.next(AvaticaResultSet.java:187)
>   at 
> org.apache.drill.jdbc.impl.DrillResultSetImpl.next(DrillResultSetImpl.java:160)
>   at sqlline.IncrementalRows.hasNext(IncrementalRows.java:62)
>   at 
> sqlline.TableOutputFormat$ResizingRowsProvider.next(TableOutputFormat.java:87)
>   at sqlline.TableOutputFormat.print(TableOutputFormat.java:118)
>   at sqlline.SqlLine.print(SqlLine.java:1593)
>   at sqlline.Commands.execute(Commands.java:852)
>   at sqlline.Commands.sql(Commands.java:751)
>   at sqlline.SqlLine.dispatch(SqlLine.java:746)
>   at sqlline.SqlLine.begin(SqlLine.java:621)
>   at sqlline.SqlLine.start(SqlLine.java:375)
>   at sqlline.SqlLine.main(SqlLine.java:268)
> Caused by: org.apache.drill.common.exceptions.UserRemoteException: SYSTEM 
> ERROR: IndexOutOfBoundsException: index: 0, length: 62784 (expected: range(0, 
> 32768))
> Fragment 0:0
> [Error Id: 77406a8a-8389-4f1b-af6c-d26d811379b7 on slave4.hadoop:31010]
>   at 
> org.apache.drill.exec.rpc.user.QueryResultHandler.resultArrived(QueryResultHandler.java:118)
>   at 
> org.apache.drill.exec.rpc.user.UserClient.handleReponse(UserClient.java:112)
>   at 
> org.apache.drill.exec.rpc.BasicClientWithConnection.handle(BasicClientWithConnection.java:47)
>   at 
> org.apache.drill.exec.rpc.BasicClientWithConnection.handle(BasicClientWithConnection.java:32)
>   at org.apache.drill.exec.rpc.RpcBus.handle(RpcBus.java:69)
>   at org.apache.drill.exec.rpc.RpcBus$RequestEvent.run(RpcBus.java:400)
>   at 
> org.apache.drill.common.SerializedExecutor$RunnableProcessor.run(SerializedExecutor.java:105)
>   at 
> org.apache.drill.exec.rpc.RpcBus$SameExecutor.execute(RpcBus.java:264)
>   at 
> org.apache.drill.common.SerializedExecutor.execute(SerializedExecutor.java:142)
>   at 
> org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:298)
>   at 
> org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:269)
>   at 
> io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:89)
>   at 
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
>   at 
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
>   at 
> io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:254)
>   at 
>