Khurram Faraaz created DRILL-4686:
-------------------------------------
Summary: Aggregation query over HBase table results in
IllegalStateException: Failure while reading vector
Key: DRILL-4686
URL: https://issues.apache.org/jira/browse/DRILL-4686
Project: Apache Drill
Issue Type: Bug
Components: Query Planning & Optimization
Affects Versions: 1.7.0
Environment: 4 node cluster CentOS
Reporter: Khurram Faraaz
Aggregation query over HBase table from Drill 1.7.0 returns
IllegalStateException
Drill version 1.7.0-SNAPSHOT, commit ID : 09b26277
{noformat}
put 'browser_action2', '1','v:e0', 'abc1';
put 'browser_action2', '2','v:e0', 'abc2';
put 'browser_action2', '3','v:e0', 'abc3';
put 'browser_action2', '4','v:e0', 'abc4';
put 'browser_action2', '5','v:e0', 'abc5';
put 'browser_action2', '6','v:e0', 'abc6';
put 'browser_action2', '7','v:e0', 'abc7';
put 'browser_action2', '8','v:e0', 'abc8';
put 'browser_action2', '9','v:e0', 'abc9';
put 'browser_action2', '10','v:e0', 'abc10';
{noformat}
{noformat}
[root@centos-01 ~]# hbase shell
HBase Shell; enter 'help<RETURN>' for list of supported commands.
Type "exit<RETURN>" to leave the HBase Shell
Version 1.1.1-mapr-1602-SNAPSHOT, r05ceb750d7ac9decac18e92650fedc0e86d85c7a,
Mon Mar 28 18:32:45 UTC 2016
Not all HBase shell commands are applicable to MapR tables.
Consult MapR documentation for the list of supported commands.
hbase(main):001:0> scan 'browser_action2'
ROW COLUMN+CELL
1 column=v:e0, timestamp=1463589516782,
value=abc1
10 column=v:e0, timestamp=1463589516916,
value=abc10
2 column=v:e0, timestamp=1463589516809,
value=abc2
3 column=v:e0, timestamp=1463589516829,
value=abc3
4 column=v:e0, timestamp=1463589516834,
value=abc4
5 column=v:e0, timestamp=1463589516847,
value=abc5
6 column=v:e0, timestamp=1463589516861,
value=abc6
7 column=v:e0, timestamp=1463589516874,
value=abc7
8 column=v:e0, timestamp=1463589516896,
value=abc8
9 column=v:e0, timestamp=1463589516905,
value=abc9
10 row(s) in 0.7970 seconds
hbase(main):002:0>
{noformat}
{noformat}
0: jdbc:drill:schema=dfs.tmp> use hbase;
+-------+------------------------------------+
| ok | summary |
+-------+------------------------------------+
| true | Default schema changed to [hbase] |
+-------+------------------------------------+
1 row selected (0.327 seconds)
0: jdbc:drill:schema=dfs.tmp> show tables;
+---------------+---------------------+
| TABLE_SCHEMA | TABLE_NAME |
+---------------+---------------------+
| hbase | browser_action2 |
| hbase | fewtypes_null |
| hbase | fewtypes_null_hive |
| hbase | hbase_student |
| hbase | hbase_voter |
| hbase | hbvoter |
| hbase | hive_hbase |
| hbase | student |
| hbase | voter |
+---------------+---------------------+
9 rows selected (1.597 seconds)
0: jdbc:drill:schema=dfs.tmp> describe browser_action2;
+--------------+------------+--------------+
| COLUMN_NAME | DATA_TYPE | IS_NULLABLE |
+--------------+------------+--------------+
| row_key | ANY | NO |
| v | MAP | NO |
+--------------+------------+--------------+
2 rows selected (1.669 seconds)
0: jdbc:drill:schema=dfs.tmp> select CONVERT_FROM(a.`v`.`e0`, 'UTF8') as k
from hbase.browser_action2 a;
+--------+
| k |
+--------+
| abc6 |
| abc5 |
| abc3 |
| abc7 |
| abc1 |
| abc10 |
| abc9 |
| abc2 |
| abc8 |
| abc4 |
+--------+
10 rows selected (0.924 seconds)
0: jdbc:drill:schema=dfs.tmp> select CONVERT_FROM(a.`v`.`e0`, 'UTF8') as k,
count(a.`v`.`e0`) p from hbase.browser_action2 a where a.row_key > '0' group
by a.`v`.`e0`;
Error: SYSTEM ERROR: IllegalStateException: Failure while reading vector.
Expected vector class of org.apache.drill.exec.vector.NullableIntVector but was
holding vector class org.apache.drill.exec.vector.NullableVarBinaryVector,
field= $f0(VARBINARY:OPTIONAL)[$bits$(UINT1:REQUIRED),
$f0(VARBINARY:OPTIONAL)[$offsets$(UINT4:REQUIRED)]]
Fragment 2:3
[Error Id: 5c0c8dc2-141b-4a2c-ba7e-008693fe4273 on centos-02.qa.lab:31010]
(state=,code=0)
0: jdbc:drill:schema=dfs.tmp>
{noformat}
Explain plan for the above failing query
{noformat}
0: jdbc:drill:schema=dfs.tmp> explain plan for select CONVERT_FROM(a.`v`.`e0`,
'UTF8') as k, count(a.`v`.`e0`) p from hbase.browser_action2 a where a.row_key
> '0' group by a.`v`.`e0`;
+------+------+
| text | json |
+------+------+
| 00-00 Screen
00-01 Project(k=[$0], p=[$1])
00-02 UnionExchange
01-01 Project(k=[CONVERT_FROMUTF8($0)], p=[$1])
01-02 HashAgg(group=[{0}], p=[$SUM0($1)])
01-03 Project($f0=[$0], p=[$1])
01-04 HashToRandomExchange(dist0=[[$0]])
02-01 UnorderedMuxExchange
03-01 Project($f0=[$0], p=[$1],
E_X_P_R_H_A_S_H_F_I_E_L_D=[hash32AsDouble($0)])
03-02 HashAgg(group=[{0}], p=[COUNT($0)])
03-03 Project($f0=[ITEM($1, 'e0')])
03-04 Scan(groupscan=[HBaseGroupScan
[HBaseScanSpec=HBaseScanSpec [tableName=browser_action2, startRow=0\x00,
stopRow=, filter=null], columns=[`*`]]])
{noformat}
Java code used to create table in HBase
{noformat}
public class PutIntDataToHBase {
public static void main(String args[]) throws IOException {
Configuration conf = HBaseConfiguration.create();
conf.set("hbase.zookeeper.property.clientPort","5181");
HBaseAdmin admin = new HBaseAdmin(conf);
if (admin.tableExists("browser_action2")) {
admin.disableTable("browser_action2");
admin.deleteTable("browser_action2");
}
byte[][] SPLIT_KEYS =
{{'0'},{'1'},{'2'},{'3'},{'4'},{'5'},{'6'},{'7'},{'8'},{'9'}};
HTableDescriptor tableDesc = new
HTableDescriptor(TableName.valueOf("browser_action2"));
tableDesc.addFamily(new HColumnDescriptor("v"));
admin.createTable(tableDesc,SPLIT_KEYS);
}
}
{noformat}
Stack trace from drillbit.log
{noformat}
2016-05-19 06:11:31,906 [28c2a6ec-c264-23a9-c1d3-d8d0d174ca57:frag:2:3] ERROR
o.a.d.e.w.fragment.FragmentExecutor - SYSTEM ERROR: IllegalStateException:
Failure while reading vector. Expected vector class of
org.apache.drill.exec.vector.NullableIntVector but was holding vector class
org.apache.drill.exec.vector.NullableVarBinaryVector, field=
$f0(VARBINARY:OPTIONAL)[$bits$(UINT1:REQUIRED),
$f0(VARBINARY:OPTIONAL)[$offsets$(UINT4:REQUIRED)]]
Fragment 2:3
[Error Id: 5c0c8dc2-141b-4a2c-ba7e-008693fe4273 on centos-02.qa.lab:31010]
org.apache.drill.common.exceptions.UserException: SYSTEM ERROR:
IllegalStateException: Failure while reading vector. Expected vector class of
org.apache.drill.exec.vector.NullableIntVector but was holding vector class
org.apache.drill.exec.vector.NullableVarBinaryVector, field=
$f0(VARBINARY:OPTIONAL)[$bits$(UINT1:REQUIRED),
$f0(VARBINARY:OPTIONAL)[$offsets$(UINT4:REQUIRED)]]
Fragment 2:3
[Error Id: 5c0c8dc2-141b-4a2c-ba7e-008693fe4273 on centos-02.qa.lab:31010]
at
org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:543)
~[drill-common-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:318)
[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:185)
[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:287)
[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
at
org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
[drill-common-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[na:1.7.0_45]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_45]
at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45]
Caused by: java.lang.IllegalStateException: Failure while reading vector.
Expected vector class of org.apache.drill.exec.vector.NullableIntVector but was
holding vector class org.apache.drill.exec.vector.NullableVarBinaryVector,
field= $f0(VARBINARY:OPTIONAL)[$bits$(UINT1:REQUIRED),
$f0(VARBINARY:OPTIONAL)[$offsets$(UINT4:REQUIRED)]]
at
org.apache.drill.exec.record.VectorContainer.getValueAccessorById(VectorContainer.java:290)
~[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
at
org.apache.drill.exec.record.RecordBatchLoader.getValueAccessorById(RecordBatchLoader.java:178)
~[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.unorderedreceiver.UnorderedReceiverBatch.getValueAccessorById(UnorderedReceiverBatch.java:135)
~[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
at
org.apache.drill.exec.test.generated.PartitionerGen953$OutgoingRecordBatch.doSetup(PartitionerTemplate.java:64)
~[na:na]
at
org.apache.drill.exec.test.generated.PartitionerGen953$OutgoingRecordBatch.initializeBatch(PartitionerTemplate.java:358)
~[na:na]
at
org.apache.drill.exec.test.generated.PartitionerGen953.flushOutgoingBatches(PartitionerTemplate.java:163)
~[na:na]
at
org.apache.drill.exec.physical.impl.partitionsender.PartitionerDecorator$FlushBatchesHandlingClass.execute(PartitionerDecorator.java:266)
~[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.partitionsender.PartitionerDecorator.executeMethodLogic(PartitionerDecorator.java:138)
~[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.partitionsender.PartitionerDecorator.flushOutgoingBatches(PartitionerDecorator.java:82)
~[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.partitionsender.PartitionSenderRootExec.innerNext(PartitionSenderRootExec.java:183)
~[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:94)
~[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
at
org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:257)
~[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
at
org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:251)
~[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
at java.security.AccessController.doPrivileged(Native Method)
~[na:1.7.0_45]
at javax.security.auth.Subject.doAs(Subject.java:415) ~[na:1.7.0_45]
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1595)
~[hadoop-common-2.7.0-mapr-1602.jar:na]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:251)
[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
... 4 common frames omitted
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
