Khurram Faraaz created DRILL-5458:
-------------------------------------
Summary: REPEATED_COUNT over varchar data results in
SchemaChangeException
Key: DRILL-5458
URL: https://issues.apache.org/jira/browse/DRILL-5458
Project: Apache Drill
Issue Type: Bug
Affects Versions: 1.11.0
Reporter: Khurram Faraaz
REPEATED_COUNT over varchar type data results in SchemaChangeException
Apache Drill 1.11.0 commit id: 3e8b01d5
{noformat}
// CTAS over JSON file.
0: jdbc:drill:schema=dfs.tmp> create table tblwarr01 as select
convert_to(t.arr,'JSON') arr, id from `rptd_count.json` t;
+-----------+----------------------------+
| Fragment | Number of records written |
+-----------+----------------------------+
| 0_0 | 10 |
+-----------+----------------------------+
1 row selected (0.221 seconds)
0: jdbc:drill:schema=dfs.tmp> select * from tblwarr01;
+--------------+-----+
| arr | id |
+--------------+-----+
| [B@67cf1438 | 1 |
| [B@4c389dc6 | 2 |
| [B@18fe5942 | 3 |
| [B@629608df | 4 |
| [B@68209b09 | 5 |
| [B@34a2a147 | 6 |
| [B@210a5750 | 7 |
| [B@2dea5622 | 8 |
| [B@73bce9ba | 9 |
| [B@7794edb2 | 10 |
+--------------+-----+
10 rows selected (0.209 seconds)
{noformat}
{noformat}
0: jdbc:drill:schema=dfs.tmp> select convert_from(t.arr,'UTF8') from tblwarr01
t;
+---------------------------------------+
| EXPR$0 |
+---------------------------------------+
| [ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 ] |
| [ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 ] |
| [ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 ] |
| [ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 ] |
| [ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 ] |
| [ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 ] |
| [ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 ] |
| [ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 ] |
| [ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 ] |
| [ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 ] |
+---------------------------------------+
10 rows selected (0.195 seconds)
// Performing a REPEATED_COUNT on the array results in SchemaChangeException
0: jdbc:drill:schema=dfs.tmp> select REPEATED_COUNT(convert_from(t.arr,'UTF8'))
from tblwarr01 t;
Error: SYSTEM ERROR: SchemaChangeException: Failure while trying to materialize
incoming schema. Errors:
Error in expression at index -1. Error: Missing function implementation:
[repeated_count(VARCHAR-REQUIRED)]. Full expression: --UNKNOWN EXPRESSION--..
Fragment 0:0
[Error Id: 1a891918-79f6-49d1-a594-f9ec5f28a99a on centos-01.qa.lab:31010]
(state=,code=0)
{noformat}
Data used in CTAS.
{noformat}
[root@centos-01 ~]# cat rptd_count.json
{"arr":[0,1,2,3,4,5,6,7,8,9,10],"id":1}
{"arr":[0,1,2,3,4,5,6,7,8,9,10],"id":2}
{"arr":[0,1,2,3,4,5,6,7,8,9,10],"id":3}
{"arr":[0,1,2,3,4,5,6,7,8,9,10],"id":4}
{"arr":[0,1,2,3,4,5,6,7,8,9,10],"id":5}
{"arr":[0,1,2,3,4,5,6,7,8,9,10],"id":6}
{"arr":[0,1,2,3,4,5,6,7,8,9,10],"id":7}
{"arr":[0,1,2,3,4,5,6,7,8,9,10],"id":8}
{"arr":[0,1,2,3,4,5,6,7,8,9,10],"id":9}
{"arr":[0,1,2,3,4,5,6,7,8,9,10],"id":10}
[root@centos-01 ~]#
{noformat}
Stack trace from drillbit.log
{noformat}
2017-05-02 05:17:03,651 [26f7e9b0-5f11-e764-6fa8-92c27ca2c6cf:frag:0:0] ERROR
o.a.d.e.w.fragment.FragmentExecutor - SYSTEM ERROR: SchemaChangeException:
Failure while trying to materialize incoming schema. Errors:
Error in expression at index -1. Error: Missing function implementation:
[repeated_count(VARCHAR-REQUIRED)]. Full expression: --UNKNOWN EXPRESSION--..
Fragment 0:0
[Error Id: 1a891918-79f6-49d1-a594-f9ec5f28a99a on centos-01.qa.lab:31010]
org.apache.drill.common.exceptions.UserException: SYSTEM ERROR:
SchemaChangeException: Failure while trying to materialize incoming schema.
Errors:
Error in expression at index -1. Error: Missing function implementation:
[repeated_count(VARCHAR-REQUIRED)]. Full expression: --UNKNOWN EXPRESSION--..
Fragment 0:0
[Error Id: 1a891918-79f6-49d1-a594-f9ec5f28a99a on centos-01.qa.lab:31010]
at
org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:544)
~[drill-common-1.11.0-SNAPSHOT.jar:1.11.0-SNAPSHOT]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:295)
[drill-java-exec-1.11.0-SNAPSHOT.jar:1.11.0-SNAPSHOT]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:160)
[drill-java-exec-1.11.0-SNAPSHOT.jar:1.11.0-SNAPSHOT]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:264)
[drill-java-exec-1.11.0-SNAPSHOT.jar:1.11.0-SNAPSHOT]
at
org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
[drill-common-1.11.0-SNAPSHOT.jar:1.11.0-SNAPSHOT]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[na:1.8.0_91]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_91]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91]
Caused by: org.apache.drill.exec.exception.SchemaChangeException: Failure while
trying to materialize incoming schema. Errors:
Error in expression at index -1. Error: Missing function implementation:
[repeated_count(VARCHAR-REQUIRED)]. Full expression: --UNKNOWN EXPRESSION--..
at
org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.setupNewSchema(ProjectRecordBatch.java:427)
~[drill-java-exec-1.11.0-SNAPSHOT.jar:1.11.0-SNAPSHOT]
at
org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:78)
~[drill-java-exec-1.11.0-SNAPSHOT.jar:1.11.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:135)
~[drill-java-exec-1.11.0-SNAPSHOT.jar:1.11.0-SNAPSHOT]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:162)
~[drill-java-exec-1.11.0-SNAPSHOT.jar:1.11.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:104)
~[drill-java-exec-1.11.0-SNAPSHOT.jar:1.11.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext(ScreenCreator.java:81)
~[drill-java-exec-1.11.0-SNAPSHOT.jar:1.11.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:94)
~[drill-java-exec-1.11.0-SNAPSHOT.jar:1.11.0-SNAPSHOT]
at
org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:234)
~[drill-java-exec-1.11.0-SNAPSHOT.jar:1.11.0-SNAPSHOT]
at
org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:227)
~[drill-java-exec-1.11.0-SNAPSHOT.jar:1.11.0-SNAPSHOT]
at java.security.AccessController.doPrivileged(Native Method)
~[na:1.8.0_91]
at javax.security.auth.Subject.doAs(Subject.java:422) ~[na:1.8.0_91]
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1595)
~[hadoop-common-2.7.0-mapr-1607.jar:na]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:227)
[drill-java-exec-1.11.0-SNAPSHOT.jar:1.11.0-SNAPSHOT]
... 4 common frames omitted
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
