[
https://issues.apache.org/jira/browse/DRILL-4255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15126597#comment-15126597
]
Khurram Faraaz commented on DRILL-4255:
---------------------------------------
Here are the test results when we set `store.json.all_text_mode` is set to
true, we see an IllegalStateException being returned
{noformat}
0: jdbc:drill:schema=dfs.tmp> select distinct t.operation from `auditlogs` t;
Error: UNSUPPORTED_OPERATION ERROR: Hash aggregate does not support schema
changes
Fragment 1:0
[Error Id: ca0ffebd-4cee-41ba-9b56-c637128516a4 on centos-02.qa.lab:31010]
(state=,code=0)
# Set option to true
0: jdbc:drill:schema=dfs.tmp> alter session set
`store.json.all_text_mode`=true;
+-------+------------------------------------+
| ok | summary |
+-------+------------------------------------+
| true | store.json.all_text_mode updated. |
+-------+------------------------------------+
1 row selected (0.213 seconds)
0: jdbc:drill:schema=dfs.tmp> select distinct t.operation from `auditlogs` t;
Error: SYSTEM ERROR: IllegalStateException: Failure while reading vector.
Expected vector class of org.apache.drill.exec.vector.NullableIntVector but was
holding vector class org.apache.drill.exec.vector.NullableVarCharVector.
Fragment 2:0
[Error Id: eb354a62-235f-47a4-a112-0e8f45758d9e on centos-02.qa.lab:31010]
(state=,code=0)
{noformat}
{noformat}
2016-02-01 17:16:04,423 [29506d2e-577e-8bbd-3b8d-fd2d97fabb70:frag:2:0] ERROR
o.a.d.e.w.fragment.FragmentExecutor - SYSTEM ERROR: IllegalStateException:
Failure while reading vector. Expected vector class of
org.apache.drill.exec.vector.NullableIntVector but was holding vector class
org.apache.drill.exec.vector.NullableVarCharVector.
Fragment 2:0
[Error Id: eb354a62-235f-47a4-a112-0e8f45758d9e on centos-02.qa.lab:31010]
org.apache.drill.common.exceptions.UserException: SYSTEM ERROR:
IllegalStateException: Failure while reading vector. Expected vector class of
org.apache.drill.exec.vector.NullableIntVector but was holding vector class
org.apache.drill.exec.vector.NullableVarCharVector.
Fragment 2:0
[Error Id: eb354a62-235f-47a4-a112-0e8f45758d9e on centos-02.qa.lab:31010]
at
org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:534)
~[drill-common-1.4.0.jar:1.4.0]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:321)
[drill-java-exec-1.4.0.jar:1.4.0]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:184)
[drill-java-exec-1.4.0.jar:1.4.0]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:290)
[drill-java-exec-1.4.0.jar:1.4.0]
at
org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
[drill-common-1.4.0.jar:1.4.0]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[na:1.8.0_65]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_65]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_65]
Caused by: java.lang.IllegalStateException: Failure while reading vector.
Expected vector class of org.apache.drill.exec.vector.NullableIntVector but was
holding vector class org.apache.drill.exec.vector.NullableVarCharVector.
at
org.apache.drill.exec.record.VectorContainer.getValueAccessorById(VectorContainer.java:284)
~[drill-java-exec-1.4.0.jar:1.4.0]
at
org.apache.drill.exec.record.RecordBatchLoader.getValueAccessorById(RecordBatchLoader.java:179)
~[drill-java-exec-1.4.0.jar:1.4.0]
at
org.apache.drill.exec.physical.impl.unorderedreceiver.UnorderedReceiverBatch.getValueAccessorById(UnorderedReceiverBatch.java:135)
~[drill-java-exec-1.4.0.jar:1.4.0]
at
org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.getValueAccessorById(IteratorValidatorBatchIterator.java:188)
~[drill-java-exec-1.4.0.jar:1.4.0]
at
org.apache.drill.exec.test.generated.PartitionerGen11$OutgoingRecordBatch.doSetup(PartitionerTemplate.java:61)
~[na:na]
at
org.apache.drill.exec.test.generated.PartitionerGen11$OutgoingRecordBatch.initializeBatch(PartitionerTemplate.java:358)
~[na:na]
at
org.apache.drill.exec.test.generated.PartitionerGen11.flushOutgoingBatches(PartitionerTemplate.java:163)
~[na:na]
at
org.apache.drill.exec.physical.impl.partitionsender.PartitionerDecorator$FlushBatchesHandlingClass.execute(PartitionerDecorator.java:266)
~[drill-java-exec-1.4.0.jar:1.4.0]
at
org.apache.drill.exec.physical.impl.partitionsender.PartitionerDecorator.executeMethodLogic(PartitionerDecorator.java:138)
~[drill-java-exec-1.4.0.jar:1.4.0]
at
org.apache.drill.exec.physical.impl.partitionsender.PartitionerDecorator.flushOutgoingBatches(PartitionerDecorator.java:82)
~[drill-java-exec-1.4.0.jar:1.4.0]
at
org.apache.drill.exec.physical.impl.partitionsender.PartitionSenderRootExec.innerNext(PartitionSenderRootExec.java:184)
~[drill-java-exec-1.4.0.jar:1.4.0]
at
org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:94)
~[drill-java-exec-1.4.0.jar:1.4.0]
at
org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:256)
~[drill-java-exec-1.4.0.jar:1.4.0]
at
org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:250)
~[drill-java-exec-1.4.0.jar:1.4.0]
at java.security.AccessController.doPrivileged(Native Method)
~[na:1.8.0_65]
at javax.security.auth.Subject.doAs(Subject.java:422) ~[na:1.8.0_65]
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1595)
~[hadoop-common-2.7.0-mapr-1506.jar:na]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:250)
[drill-java-exec-1.4.0.jar:1.4.0]
... 4 common frames omitted
2016-02-01 17:16:04,446 [CONTROL-rpc-event-queue] INFO
o.a.d.e.w.fragment.FragmentExecutor - 29506d2e-577e-8bbd-3b8d-fd2d97fabb70:1:0:
State change requested RUNNING --> CANCELLATION_REQUESTED
{noformat}
> SELECT DISTINCT query over JSON data returns UNSUPPORTED OPERATION
> ------------------------------------------------------------------
>
> Key: DRILL-4255
> URL: https://issues.apache.org/jira/browse/DRILL-4255
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Flow
> Affects Versions: 1.4.0
> Environment: CentOS
> Reporter: Khurram Faraaz
>
> SELECT DISTINCT over mapr fs generated audit logs (JSON files) results in
> unsupported operation. An exact query over another set of JSON data returns
> correct results.
> MapR Drill 1.4.0, commit ID : 9627a80f
> MapRBuildVersion : 5.1.0.36488.GA
> OS : CentOS x86_64 GNU/Linux
> {noformat}
> 0: jdbc:drill:schema=dfs.tmp> select distinct t.operation from `auditlogs` t;
> Error: UNSUPPORTED_OPERATION ERROR: Hash aggregate does not support schema
> changes
> Fragment 3:3
> [Error Id: 1233bf68-13da-4043-a162-cf6d98c07ec9 on example.com:31010]
> (state=,code=0)
> {noformat}
> Stack trace from drillbit.log
> {noformat}
> 2016-01-08 11:35:35,093 [297060f9-1c7a-b32c-09e8-24b5ad863e73:frag:3:3] INFO
> o.a.d.e.p.i.aggregate.HashAggBatch - User Error Occurred
> org.apache.drill.common.exceptions.UserException: UNSUPPORTED_OPERATION
> ERROR: Hash aggregate does not support schema changes
> [Error Id: 1233bf68-13da-4043-a162-cf6d98c07ec9 ]
> at
> org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:534)
> ~[drill-common-1.4.0.jar:1.4.0]
> at
> org.apache.drill.exec.physical.impl.aggregate.HashAggBatch.innerNext(HashAggBatch.java:144)
> [drill-java-exec-1.4.0.jar:1.4.0]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:162)
> [drill-java-exec-1.4.0.jar:1.4.0]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
> [drill-java-exec-1.4.0.jar:1.4.0]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:109)
> [drill-java-exec-1.4.0.jar:1.4.0]
> at
> org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:51)
> [drill-java-exec-1.4.0.jar:1.4.0]
> at
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:132)
> [drill-java-exec-1.4.0.jar:1.4.0]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:162)
> [drill-java-exec-1.4.0.jar:1.4.0]
> at
> org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:104)
> [drill-java-exec-1.4.0.jar:1.4.0]
> at
> org.apache.drill.exec.physical.impl.SingleSenderCreator$SingleSenderRootExec.innerNext(SingleSenderCreator.java:93)
> [drill-java-exec-1.4.0.jar:1.4.0]
> at
> org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:94)
> [drill-java-exec-1.4.0.jar:1.4.0]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:256)
> [drill-java-exec-1.4.0.jar:1.4.0]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:250)
> [drill-java-exec-1.4.0.jar:1.4.0]
> at java.security.AccessController.doPrivileged(Native Method)
> [na:1.7.0_65]
> at javax.security.auth.Subject.doAs(Subject.java:415) [na:1.7.0_65]
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1595)
> [hadoop-common-2.7.0-mapr-1506.jar:na]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:250)
> [drill-java-exec-1.4.0.jar:1.4.0]
> at
> org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
> [drill-common-1.4.0.jar:1.4.0]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> [na:1.7.0_65]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> [na:1.7.0_65]
> at java.lang.Thread.run(Thread.java:745) [na:1.7.0_65]
> {noformat}
> Query plan for above query.
> {noformat}
> 00-00 Screen : rowType = RecordType(ANY operation): rowcount = 141437.16,
> cumulative cost = {3.4100499276E7 rows, 1.69455861396E8 cpu, 0.0 io,
> 1.2165858754560001E10 network, 2.7382234176000005E8 memory}, id = 7572
> 00-01 UnionExchange : rowType = RecordType(ANY operation): rowcount =
> 141437.16, cumulative cost = {3.408635556E7 rows, 1.6944171768E8 cpu, 0.0 io,
> 1.2165858754560001E10 network, 2.7382234176000005E8 memory}, id = 7571
> 01-01 Project(operation=[$0]) : rowType = RecordType(ANY operation):
> rowcount = 141437.16, cumulative cost = {3.3944918400000006E7 rows,
> 1.683102204E8 cpu, 0.0 io, 1.15865321472E10 network, 2.7382234176000005E8
> memory}, id = 7570
> 01-02 HashAgg(group=[{0}]) : rowType = RecordType(ANY operation):
> rowcount = 141437.16, cumulative cost = {3.3944918400000006E7 rows,
> 1.683102204E8 cpu, 0.0 io, 1.15865321472E10 network, 2.7382234176000005E8
> memory}, id = 7569
> 01-03 Project(operation=[$0]) : rowType = RecordType(ANY
> operation): rowcount = 1414371.6, cumulative cost = {3.2530546800000004E7
> rows, 1.569952476E8 cpu, 0.0 io, 1.15865321472E10 network,
> 2.4892940160000002E8 memory}, id = 7568
> 01-04 HashToRandomExchange(dist0=[[$0]]) : rowType =
> RecordType(ANY operation, ANY E_X_P_R_H_A_S_H_F_I_E_L_D): rowcount =
> 1414371.6, cumulative cost = {3.2530546800000004E7 rows, 1.569952476E8 cpu,
> 0.0 io, 1.15865321472E10 network, 2.4892940160000002E8 memory}, id = 7567
> 02-01 UnorderedMuxExchange : rowType = RecordType(ANY
> operation, ANY E_X_P_R_H_A_S_H_F_I_E_L_D): rowcount = 1414371.6, cumulative
> cost = {3.1116175200000003E7 rows, 1.34365302E8 cpu, 0.0 io, 0.0 network,
> 2.4892940160000002E8 memory}, id = 7566
> 03-01 Project(operation=[$0],
> E_X_P_R_H_A_S_H_F_I_E_L_D=[hash32AsDouble($0)]) : rowType = RecordType(ANY
> operation, ANY E_X_P_R_H_A_S_H_F_I_E_L_D): rowcount = 1414371.6, cumulative
> cost = {2.97018036E7 rows, 1.329509304E8 cpu, 0.0 io, 0.0 network,
> 2.4892940160000002E8 memory}, id = 7565
> 03-02 HashAgg(group=[{0}]) : rowType = RecordType(ANY
> operation): rowcount = 1414371.6, cumulative cost = {2.8287432E7 rows,
> 1.27293444E8 cpu, 0.0 io, 0.0 network, 2.4892940160000002E8 memory}, id = 7564
> 03-03 Scan(groupscan=[EasyGroupScan
> [selectionRoot=maprfs:/tmp/auditlogs, numFiles=31, columns=[`operation`],
> files=[maprfs:/tmp/auditlogs/DBAudit.log-2015-12-30-001.json,
> maprfs:/tmp/auditlogs/DBAudit.log-2015-12-28-002.json,
> maprfs:/tmp/auditlogs/FSAudit.log-2015-12-31-001.json,
> maprfs:/tmp/auditlogs/FSAudit.log-2016-01-06-003.json,
> maprfs:/tmp/auditlogs/FSAudit.log-2015-12-28-002.json,
> maprfs:/tmp/auditlogs/DBAudit.log-2015-12-28-001.json,
> maprfs:/tmp/auditlogs/FSAudit.log-2015-12-30-001.json,
> maprfs:/tmp/auditlogs/FSAudit.log-2015-12-28-003.json,
> maprfs:/tmp/auditlogs/DBAudit.log-2015-12-31-002.json,
> maprfs:/tmp/auditlogs/FSAudit.log-2016-01-04-001.json,
> maprfs:/tmp/auditlogs/DBAudit.log-2016-01-06-001.json,
> maprfs:/tmp/auditlogs/DBAudit.log-2015-12-28-003.json,
> maprfs:/tmp/auditlogs/FSAudit.log-2015-12-31-002.json,
> maprfs:/tmp/auditlogs/DBAudit.log-2016-01-06-003.json,
> maprfs:/tmp/auditlogs/FSAudit.log-2015-12-31-003.json,
> maprfs:/tmp/auditlogs/FSAudit.log-2016-01-06-001.json,
> maprfs:/tmp/auditlogs/FSAudit.log-2016-01-03-001.json,
> maprfs:/tmp/auditlogs/DBAudit.log-2015-12-31-001.json,
> maprfs:/tmp/auditlogs/DBAudit.log-2015-12-29-001.json,
> maprfs:/tmp/auditlogs/DBAudit.log-2015-12-28-004.json,
> maprfs:/tmp/auditlogs/FSAudit.log-2016-01-01-001.json,
> maprfs:/tmp/auditlogs/FSAudit.log-2015-12-28-004.json,
> maprfs:/tmp/auditlogs/FSAudit.log-2015-12-29-001.json,
> maprfs:/tmp/auditlogs/FSAudit.log-2015-12-28-001.json,
> maprfs:/tmp/auditlogs/DBAudit.log-2016-01-01-001.json,
> maprfs:/tmp/auditlogs/FSAudit.log-2016-01-06-004.json,
> maprfs:/tmp/auditlogs/DBAudit.log-2016-01-06-004.json,
> maprfs:/tmp/auditlogs/FSAudit.log-2016-01-06-002.json,
> maprfs:/tmp/auditlogs/FSAudit.log-2016-01-07-001.json,
> maprfs:/tmp/auditlogs/DBAudit.log-2016-01-06-002.json,
> maprfs:/tmp/auditlogs/FSAudit.log-2016-01-08-001.json]]]) : rowType =
> RecordType(ANY operation): rowcount = 1.4143716E7, cumulative cost =
> {1.4143716E7 rows, 1.4143716E7 cpu, 0.0 io, 0.0 network, 0.0 memory}, id =
> 7563
> {noformat}
> Another query that is exactly like the failing query reported here, this one
> returns correct results though.
> {noformat}
> 0: jdbc:drill:schema=dfs.tmp> select distinct t.key2 from `twoKeyJsn.json` t;
> +-------+
> | key2 |
> +-------+
> | d |
> | c |
> | b |
> | 1 |
> | a |
> | 0 |
> | k |
> | m |
> | j |
> | h |
> | e |
> | n |
> | g |
> | f |
> | l |
> | i |
> +-------+
> 16 rows selected (27.097 seconds)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
