[jira] [Updated] (DRILL-4083) Drill not using HiveDrillNativeParquetScan if no column is needed to be read from HIVE
[
https://issues.apache.org/jira/browse/DRILL-4083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Victoria Markman updated DRILL-4083:
Fix Version/s: 1.4.0
> Drill not using HiveDrillNativeParquetScan if no column is needed to be read
> from HIVE
> --
>
> Key: DRILL-4083
> URL: https://issues.apache.org/jira/browse/DRILL-4083
> Project: Apache Drill
> Issue Type: Bug
> Components: Storage - Hive
>Reporter: Sean Hsuan-Yi Chu
>Assignee: Sean Hsuan-Yi Chu
> Fix For: 1.4.0
>
>
> For example, for a query such as:
> {code}
> Select count(*) from hive.parquetTable
> {code}
> would not use HiveDrillNativeParquetScan. However, the following query will
> use:
> {code}
> Select count(*) from hive.parquetTable where column > 0
> {code}
> Ideally, both should use the same HiveDrillNativeParquetScan
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (DRILL-4125) Illegal argument exception during merge join
[
https://issues.apache.org/jira/browse/DRILL-4125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Victoria Markman updated DRILL-4125:
Attachment: drillbit.log
> Illegal argument exception during merge join
> -
>
> Key: DRILL-4125
> URL: https://issues.apache.org/jira/browse/DRILL-4125
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Relational Operators
>Affects Versions: 1.3.0, 1.4.0
>Reporter: Victoria Markman
>Priority: Blocker
> Fix For: 1.4.0
>
> Attachments: 29ac59f2-5d92-7378-bf81-e844a300efd7.sys.drill,
> drillbit.log
>
>
> Same setup as in DRILL-4109
> Query: framework/resources/Advanced/tpcds/tpcds_sf100/original/query93.sql
> Excerpt from drillbit.log
> {code}
> 2015-11-23 23:50:44,071 [29ac59f2-5d92-7378-bf81-e844a300efd7:frag:5:74]
> ERROR o.a.d.e.w.fragment.FragmentExecutor - SYSTEM ERROR:
> IllegalArgumentException
> Fragment 5:74
> [Error Id: 1ca9758d-1864-4940-9efa-b8906d4f9b52 on atsqa4-133.qa.lab:31010]
> org.apache.drill.common.exceptions.UserException: SYSTEM ERROR:
> IllegalArgumentException
> Fragment 5:74
> [Error Id: 1ca9758d-1864-4940-9efa-b8906d4f9b52 on atsqa4-133.qa.lab:31010]
> at
> org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:534)
> ~[drill-common-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:321)
> [drill-java-exec-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:184)
> [drill-java-exec-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:290)
> [drill-java-exec-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
> at
> org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
> [drill-common-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> [na:1.7.0_71]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> [na:1.7.0_71]
> at java.lang.Thread.run(Thread.java:745) [na:1.7.0_71]
> Caused by: java.lang.IllegalArgumentException: null
> at
> com.google.common.base.Preconditions.checkArgument(Preconditions.java:76)
> ~[guava-14.0.1.jar:na]
> at
> org.apache.drill.exec.record.RecordIterator.getCurrentPosition(RecordIterator.java:242)
> ~[drill-java-exec-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
> at
> org.apache.drill.exec.test.generated.JoinWorkerGen8348.doJoin(JoinTemplate.java:63)
> ~[na:na]
> at
> org.apache.drill.exec.physical.impl.join.MergeJoinBatch.innerNext(MergeJoinBatch.java:206)
> ~[drill-java-exec-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:162)
> ~[drill-java-exec-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
> ~[drill-java-exec-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:109)
> ~[drill-java-exec-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
> at
> org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:51)
> ~[drill-java-exec-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
> at
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:132)
> ~[drill-java-exec-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:162)
> ~[drill-java-exec-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
> ~[drill-java-exec-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:109)
> ~[drill-java-exec-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
> at
> org.apache.drill.exec.physical.impl.xsort.ExternalSortBatch.innerNext(ExternalSortBatch.java:276)
> ~[drill-java-exec-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:162)
> ~[drill-java-exec-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
> ~[drill-java-exec-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:109)
> ~[drill-java-exec-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
> at
> org.apache
[jira] [Updated] (DRILL-4125) Illegal argument exception during merge join
[
https://issues.apache.org/jira/browse/DRILL-4125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Victoria Markman updated DRILL-4125:
Attachment: 29ac59f2-5d92-7378-bf81-e844a300efd7.sys.drill
> Illegal argument exception during merge join
> -
>
> Key: DRILL-4125
> URL: https://issues.apache.org/jira/browse/DRILL-4125
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Relational Operators
>Affects Versions: 1.3.0, 1.4.0
>Reporter: Victoria Markman
>Priority: Blocker
> Fix For: 1.4.0
>
> Attachments: 29ac59f2-5d92-7378-bf81-e844a300efd7.sys.drill
>
>
> Same setup as in DRILL-4109
> Query: framework/resources/Advanced/tpcds/tpcds_sf100/original/query93.sql
> Excerpt from drillbit.log
> {code}
> 2015-11-23 23:50:44,071 [29ac59f2-5d92-7378-bf81-e844a300efd7:frag:5:74]
> ERROR o.a.d.e.w.fragment.FragmentExecutor - SYSTEM ERROR:
> IllegalArgumentException
> Fragment 5:74
> [Error Id: 1ca9758d-1864-4940-9efa-b8906d4f9b52 on atsqa4-133.qa.lab:31010]
> org.apache.drill.common.exceptions.UserException: SYSTEM ERROR:
> IllegalArgumentException
> Fragment 5:74
> [Error Id: 1ca9758d-1864-4940-9efa-b8906d4f9b52 on atsqa4-133.qa.lab:31010]
> at
> org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:534)
> ~[drill-common-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:321)
> [drill-java-exec-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:184)
> [drill-java-exec-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:290)
> [drill-java-exec-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
> at
> org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
> [drill-common-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> [na:1.7.0_71]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> [na:1.7.0_71]
> at java.lang.Thread.run(Thread.java:745) [na:1.7.0_71]
> Caused by: java.lang.IllegalArgumentException: null
> at
> com.google.common.base.Preconditions.checkArgument(Preconditions.java:76)
> ~[guava-14.0.1.jar:na]
> at
> org.apache.drill.exec.record.RecordIterator.getCurrentPosition(RecordIterator.java:242)
> ~[drill-java-exec-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
> at
> org.apache.drill.exec.test.generated.JoinWorkerGen8348.doJoin(JoinTemplate.java:63)
> ~[na:na]
> at
> org.apache.drill.exec.physical.impl.join.MergeJoinBatch.innerNext(MergeJoinBatch.java:206)
> ~[drill-java-exec-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:162)
> ~[drill-java-exec-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
> ~[drill-java-exec-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:109)
> ~[drill-java-exec-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
> at
> org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:51)
> ~[drill-java-exec-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
> at
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:132)
> ~[drill-java-exec-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:162)
> ~[drill-java-exec-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
> ~[drill-java-exec-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:109)
> ~[drill-java-exec-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
> at
> org.apache.drill.exec.physical.impl.xsort.ExternalSortBatch.innerNext(ExternalSortBatch.java:276)
> ~[drill-java-exec-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:162)
> ~[drill-java-exec-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
> ~[drill-java-exec-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:109)
> ~[drill-java-exec-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
>
[jira] [Created] (DRILL-4125) Illegal argument exception during merge join
Victoria Markman created DRILL-4125:
---
Summary: Illegal argument exception during merge join
Key: DRILL-4125
URL: https://issues.apache.org/jira/browse/DRILL-4125
Project: Apache Drill
Issue Type: Bug
Components: Execution - Relational Operators
Affects Versions: 1.3.0, 1.4.0
Reporter: Victoria Markman
Priority: Blocker
Fix For: 1.4.0
Same setup as in DRILL-4109
Query: framework/resources/Advanced/tpcds/tpcds_sf100/original/query93.sql
Excerpt from drillbit.log
{code}
2015-11-23 23:50:44,071 [29ac59f2-5d92-7378-bf81-e844a300efd7:frag:5:74] ERROR
o.a.d.e.w.fragment.FragmentExecutor - SYSTEM ERROR: IllegalArgumentException
Fragment 5:74
[Error Id: 1ca9758d-1864-4940-9efa-b8906d4f9b52 on atsqa4-133.qa.lab:31010]
org.apache.drill.common.exceptions.UserException: SYSTEM ERROR:
IllegalArgumentException
Fragment 5:74
[Error Id: 1ca9758d-1864-4940-9efa-b8906d4f9b52 on atsqa4-133.qa.lab:31010]
at
org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:534)
~[drill-common-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:321)
[drill-java-exec-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:184)
[drill-java-exec-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:290)
[drill-java-exec-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
at
org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
[drill-common-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[na:1.7.0_71]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_71]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_71]
Caused by: java.lang.IllegalArgumentException: null
at
com.google.common.base.Preconditions.checkArgument(Preconditions.java:76)
~[guava-14.0.1.jar:na]
at
org.apache.drill.exec.record.RecordIterator.getCurrentPosition(RecordIterator.java:242)
~[drill-java-exec-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
at
org.apache.drill.exec.test.generated.JoinWorkerGen8348.doJoin(JoinTemplate.java:63)
~[na:na]
at
org.apache.drill.exec.physical.impl.join.MergeJoinBatch.innerNext(MergeJoinBatch.java:206)
~[drill-java-exec-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:162)
~[drill-java-exec-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
~[drill-java-exec-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:109)
~[drill-java-exec-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
at
org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:51)
~[drill-java-exec-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:132)
~[drill-java-exec-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:162)
~[drill-java-exec-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
~[drill-java-exec-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:109)
~[drill-java-exec-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.xsort.ExternalSortBatch.innerNext(ExternalSortBatch.java:276)
~[drill-java-exec-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:162)
~[drill-java-exec-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:119)
~[drill-java-exec-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:109)
~[drill-java-exec-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
at
org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:51)
~[drill-java-exec-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.svremover.RemovingRecordBatch.innerNext(RemovingRecordBatch.java:94)
~[drill-java-exec-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:162)
~[drill-java-exec-1.4.0-SNAPSHOT.jar:1.
[jira] [Updated] (DRILL-4109) NPE in external sort
[
https://issues.apache.org/jira/browse/DRILL-4109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Victoria Markman updated DRILL-4109:
Attachment: 29ac6c1b-9b33-3457-8bc8-9e2dff6ad438.sys.drill
> NPE in external sort
> -
>
> Key: DRILL-4109
> URL: https://issues.apache.org/jira/browse/DRILL-4109
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.4.0
>Reporter: Victoria Markman
>Priority: Blocker
> Attachments: 29ac6c1b-9b33-3457-8bc8-9e2dff6ad438.sys.drill,
> 29b41f37-4803-d7ce-e05f-912d1f65da79.sys.drill, drillbit.log,
> drillbit.log.debug
>
>
> 4 node cluster
> 36GB of direct memory
> 4GB heap memory
> planner.memory.max_query_memory_per_node=2GB (default)
> planner.enable_hashjoin = false
> Spill directory has 6.4T of memory available:
> {noformat}
> [Tue Nov 17 18:23:18 /tmp/drill ] # df -H .
> Filesystem Size Used Avail Use% Mounted on
> localhost:/mapr 7.7T 1.4T 6.4T 18% /mapr
> {noformat}
> Run query below:
> framework/resources/Advanced/tpcds/tpcds_sf100/original/query15.sql
> drillbit.log
> {code}
> 2015-11-18 02:22:12,639 [29b41f37-4803-d7ce-e05f-912d1f65da79:frag:3:9] INFO
> o.a.d.e.p.i.xsort.ExternalSortBatch - Merging and spilling to
> /tmp/drill/spill/29b41f37-4803-d7ce-e05f-912d1f65da79/major_fragment_3/minor_fragment_9/operator_17/7
> 2015-11-18 02:22:12,770 [29b41f37-4803-d7ce-e05f-912d1f65da79:frag:3:5] INFO
> o.a.d.e.p.i.xsort.ExternalSortBatch - Merging and spilling to
> /tmp/drill/spill/29b41f37-4803-d7ce-e05f-912d1f65da79/major_fragment_3/minor_fragment_5/operator_17/7
> 2015-11-18 02:22:13,345 [29b41f37-4803-d7ce-e05f-912d1f65da79:frag:3:17] INFO
> o.a.d.e.p.i.xsort.ExternalSortBatch - Completed spilling to
> /tmp/drill/spill/29b41f37-4803-d7ce-e05f-912d1f65da79/major_fragment_3/minor_fragment_17/operator_17/7
> 2015-11-18 02:22:13,346 [29b41f37-4803-d7ce-e05f-912d1f65da79:frag:3:13] INFO
> o.a.d.e.p.i.xsort.ExternalSortBatch - Completed spilling to
> /tmp/drill/spill/29b41f37-4803-d7ce-e05f-912d1f65da79/major_fragment_3/minor_fragment_13/operator_16/1
> 2015-11-18 02:22:13,346 [29b41f37-4803-d7ce-e05f-912d1f65da79:frag:3:13] WARN
> o.a.d.e.p.i.xsort.ExternalSortBatch - Starting to merge. 34 batch groups.
> Current allocated memory: 2252186
> 2015-11-18 02:22:13,363 [29b41f37-4803-d7ce-e05f-912d1f65da79:frag:3:13] INFO
> o.a.d.e.w.fragment.FragmentExecutor -
> 29b41f37-4803-d7ce-e05f-912d1f65da79:3:13: State change requested RUNNING -->
> FAILED
> 2015-11-18 02:22:13,370 [29b41f37-4803-d7ce-e05f-912d1f65da79:frag:3:13] INFO
> o.a.d.e.w.fragment.FragmentExecutor -
> 29b41f37-4803-d7ce-e05f-912d1f65da79:3:13: State change requested FAILED -->
> FINISHED
> 2015-11-18 02:22:13,371 [29b41f37-4803-d7ce-e05f-912d1f65da79:frag:3:13]
> ERROR o.a.d.e.w.fragment.FragmentExecutor - SYSTEM ERROR: NullPointerException
> Fragment 3:13
> [Error Id: c5d67dcb-16aa-4951-89f5-599b4b4eb54d on atsqa4-133.qa.lab:31010]
> org.apache.drill.common.exceptions.UserException: SYSTEM ERROR:
> NullPointerException
> Fragment 3:13
> [Error Id: c5d67dcb-16aa-4951-89f5-599b4b4eb54d on atsqa4-133.qa.lab:31010]
> at
> org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:534)
> ~[drill-common-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:321)
> [drill-java-exec-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:184)
> [drill-java-exec-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:290)
> [drill-java-exec-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
> at
> org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
> [drill-common-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> [na:1.7.0_71]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> [na:1.7.0_71]
> at java.lang.Thread.run(Thread.java:745) [na:1.7.0_71]
> java.lang.NullPointerException: null
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (DRILL-4109) NPE in external sort
[
https://issues.apache.org/jira/browse/DRILL-4109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Victoria Markman updated DRILL-4109:
Attachment: drillbit.log.debug
> NPE in external sort
> -
>
> Key: DRILL-4109
> URL: https://issues.apache.org/jira/browse/DRILL-4109
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.4.0
>Reporter: Victoria Markman
>Priority: Blocker
> Attachments: 29b41f37-4803-d7ce-e05f-912d1f65da79.sys.drill,
> drillbit.log, drillbit.log.debug
>
>
> 4 node cluster
> 36GB of direct memory
> 4GB heap memory
> planner.memory.max_query_memory_per_node=2GB (default)
> planner.enable_hashjoin = false
> Spill directory has 6.4T of memory available:
> {noformat}
> [Tue Nov 17 18:23:18 /tmp/drill ] # df -H .
> Filesystem Size Used Avail Use% Mounted on
> localhost:/mapr 7.7T 1.4T 6.4T 18% /mapr
> {noformat}
> Run query below:
> framework/resources/Advanced/tpcds/tpcds_sf100/original/query15.sql
> drillbit.log
> {code}
> 2015-11-18 02:22:12,639 [29b41f37-4803-d7ce-e05f-912d1f65da79:frag:3:9] INFO
> o.a.d.e.p.i.xsort.ExternalSortBatch - Merging and spilling to
> /tmp/drill/spill/29b41f37-4803-d7ce-e05f-912d1f65da79/major_fragment_3/minor_fragment_9/operator_17/7
> 2015-11-18 02:22:12,770 [29b41f37-4803-d7ce-e05f-912d1f65da79:frag:3:5] INFO
> o.a.d.e.p.i.xsort.ExternalSortBatch - Merging and spilling to
> /tmp/drill/spill/29b41f37-4803-d7ce-e05f-912d1f65da79/major_fragment_3/minor_fragment_5/operator_17/7
> 2015-11-18 02:22:13,345 [29b41f37-4803-d7ce-e05f-912d1f65da79:frag:3:17] INFO
> o.a.d.e.p.i.xsort.ExternalSortBatch - Completed spilling to
> /tmp/drill/spill/29b41f37-4803-d7ce-e05f-912d1f65da79/major_fragment_3/minor_fragment_17/operator_17/7
> 2015-11-18 02:22:13,346 [29b41f37-4803-d7ce-e05f-912d1f65da79:frag:3:13] INFO
> o.a.d.e.p.i.xsort.ExternalSortBatch - Completed spilling to
> /tmp/drill/spill/29b41f37-4803-d7ce-e05f-912d1f65da79/major_fragment_3/minor_fragment_13/operator_16/1
> 2015-11-18 02:22:13,346 [29b41f37-4803-d7ce-e05f-912d1f65da79:frag:3:13] WARN
> o.a.d.e.p.i.xsort.ExternalSortBatch - Starting to merge. 34 batch groups.
> Current allocated memory: 2252186
> 2015-11-18 02:22:13,363 [29b41f37-4803-d7ce-e05f-912d1f65da79:frag:3:13] INFO
> o.a.d.e.w.fragment.FragmentExecutor -
> 29b41f37-4803-d7ce-e05f-912d1f65da79:3:13: State change requested RUNNING -->
> FAILED
> 2015-11-18 02:22:13,370 [29b41f37-4803-d7ce-e05f-912d1f65da79:frag:3:13] INFO
> o.a.d.e.w.fragment.FragmentExecutor -
> 29b41f37-4803-d7ce-e05f-912d1f65da79:3:13: State change requested FAILED -->
> FINISHED
> 2015-11-18 02:22:13,371 [29b41f37-4803-d7ce-e05f-912d1f65da79:frag:3:13]
> ERROR o.a.d.e.w.fragment.FragmentExecutor - SYSTEM ERROR: NullPointerException
> Fragment 3:13
> [Error Id: c5d67dcb-16aa-4951-89f5-599b4b4eb54d on atsqa4-133.qa.lab:31010]
> org.apache.drill.common.exceptions.UserException: SYSTEM ERROR:
> NullPointerException
> Fragment 3:13
> [Error Id: c5d67dcb-16aa-4951-89f5-599b4b4eb54d on atsqa4-133.qa.lab:31010]
> at
> org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:534)
> ~[drill-common-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:321)
> [drill-java-exec-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:184)
> [drill-java-exec-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:290)
> [drill-java-exec-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
> at
> org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
> [drill-common-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> [na:1.7.0_71]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> [na:1.7.0_71]
> at java.lang.Thread.run(Thread.java:745) [na:1.7.0_71]
> java.lang.NullPointerException: null
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (DRILL-4109) NPE in external sort
[
https://issues.apache.org/jira/browse/DRILL-4109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Victoria Markman updated DRILL-4109:
Attachment: drillbit.log
> NPE in external sort
> -
>
> Key: DRILL-4109
> URL: https://issues.apache.org/jira/browse/DRILL-4109
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.4.0
>Reporter: Victoria Markman
>Priority: Blocker
> Attachments: 29b41f37-4803-d7ce-e05f-912d1f65da79.sys.drill,
> drillbit.log
>
>
> 4 node cluster
> 36GB of direct memory
> 4GB heap memory
> planner.memory.max_query_memory_per_node=2GB (default)
> planner.enable_hashjoin = false
> Spill directory has 6.4T of memory available:
> {noformat}
> [Tue Nov 17 18:23:18 /tmp/drill ] # df -H .
> Filesystem Size Used Avail Use% Mounted on
> localhost:/mapr 7.7T 1.4T 6.4T 18% /mapr
> {noformat}
> Run query below:
> framework/resources/Advanced/tpcds/tpcds_sf100/original/query15.sql
> drillbit.log
> {code}
> 2015-11-18 02:22:12,639 [29b41f37-4803-d7ce-e05f-912d1f65da79:frag:3:9] INFO
> o.a.d.e.p.i.xsort.ExternalSortBatch - Merging and spilling to
> /tmp/drill/spill/29b41f37-4803-d7ce-e05f-912d1f65da79/major_fragment_3/minor_fragment_9/operator_17/7
> 2015-11-18 02:22:12,770 [29b41f37-4803-d7ce-e05f-912d1f65da79:frag:3:5] INFO
> o.a.d.e.p.i.xsort.ExternalSortBatch - Merging and spilling to
> /tmp/drill/spill/29b41f37-4803-d7ce-e05f-912d1f65da79/major_fragment_3/minor_fragment_5/operator_17/7
> 2015-11-18 02:22:13,345 [29b41f37-4803-d7ce-e05f-912d1f65da79:frag:3:17] INFO
> o.a.d.e.p.i.xsort.ExternalSortBatch - Completed spilling to
> /tmp/drill/spill/29b41f37-4803-d7ce-e05f-912d1f65da79/major_fragment_3/minor_fragment_17/operator_17/7
> 2015-11-18 02:22:13,346 [29b41f37-4803-d7ce-e05f-912d1f65da79:frag:3:13] INFO
> o.a.d.e.p.i.xsort.ExternalSortBatch - Completed spilling to
> /tmp/drill/spill/29b41f37-4803-d7ce-e05f-912d1f65da79/major_fragment_3/minor_fragment_13/operator_16/1
> 2015-11-18 02:22:13,346 [29b41f37-4803-d7ce-e05f-912d1f65da79:frag:3:13] WARN
> o.a.d.e.p.i.xsort.ExternalSortBatch - Starting to merge. 34 batch groups.
> Current allocated memory: 2252186
> 2015-11-18 02:22:13,363 [29b41f37-4803-d7ce-e05f-912d1f65da79:frag:3:13] INFO
> o.a.d.e.w.fragment.FragmentExecutor -
> 29b41f37-4803-d7ce-e05f-912d1f65da79:3:13: State change requested RUNNING -->
> FAILED
> 2015-11-18 02:22:13,370 [29b41f37-4803-d7ce-e05f-912d1f65da79:frag:3:13] INFO
> o.a.d.e.w.fragment.FragmentExecutor -
> 29b41f37-4803-d7ce-e05f-912d1f65da79:3:13: State change requested FAILED -->
> FINISHED
> 2015-11-18 02:22:13,371 [29b41f37-4803-d7ce-e05f-912d1f65da79:frag:3:13]
> ERROR o.a.d.e.w.fragment.FragmentExecutor - SYSTEM ERROR: NullPointerException
> Fragment 3:13
> [Error Id: c5d67dcb-16aa-4951-89f5-599b4b4eb54d on atsqa4-133.qa.lab:31010]
> org.apache.drill.common.exceptions.UserException: SYSTEM ERROR:
> NullPointerException
> Fragment 3:13
> [Error Id: c5d67dcb-16aa-4951-89f5-599b4b4eb54d on atsqa4-133.qa.lab:31010]
> at
> org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:534)
> ~[drill-common-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:321)
> [drill-java-exec-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:184)
> [drill-java-exec-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:290)
> [drill-java-exec-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
> at
> org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
> [drill-common-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> [na:1.7.0_71]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> [na:1.7.0_71]
> at java.lang.Thread.run(Thread.java:745) [na:1.7.0_71]
> java.lang.NullPointerException: null
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (DRILL-4109) NPE in external sort
Victoria Markman created DRILL-4109:
---
Summary: NPE in external sort
Key: DRILL-4109
URL: https://issues.apache.org/jira/browse/DRILL-4109
Project: Apache Drill
Issue Type: Bug
Affects Versions: 1.4.0
Reporter: Victoria Markman
Priority: Blocker
Attachments: 29b41f37-4803-d7ce-e05f-912d1f65da79.sys.drill
4 node cluster
36GB of direct memory
4GB heap memory
planner.memory.max_query_memory_per_node=2GB (default)
planner.enable_hashjoin = false
Spill directory has 6.4T of memory available:
{noformat}
[Tue Nov 17 18:23:18 /tmp/drill ] # df -H .
Filesystem Size Used Avail Use% Mounted on
localhost:/mapr 7.7T 1.4T 6.4T 18% /mapr
{noformat}
Run query below:
framework/resources/Advanced/tpcds/tpcds_sf100/original/query15.sql
drillbit.log
{code}
2015-11-18 02:22:12,639 [29b41f37-4803-d7ce-e05f-912d1f65da79:frag:3:9] INFO
o.a.d.e.p.i.xsort.ExternalSortBatch - Merging and spilling to
/tmp/drill/spill/29b41f37-4803-d7ce-e05f-912d1f65da79/major_fragment_3/minor_fragment_9/operator_17/7
2015-11-18 02:22:12,770 [29b41f37-4803-d7ce-e05f-912d1f65da79:frag:3:5] INFO
o.a.d.e.p.i.xsort.ExternalSortBatch - Merging and spilling to
/tmp/drill/spill/29b41f37-4803-d7ce-e05f-912d1f65da79/major_fragment_3/minor_fragment_5/operator_17/7
2015-11-18 02:22:13,345 [29b41f37-4803-d7ce-e05f-912d1f65da79:frag:3:17] INFO
o.a.d.e.p.i.xsort.ExternalSortBatch - Completed spilling to
/tmp/drill/spill/29b41f37-4803-d7ce-e05f-912d1f65da79/major_fragment_3/minor_fragment_17/operator_17/7
2015-11-18 02:22:13,346 [29b41f37-4803-d7ce-e05f-912d1f65da79:frag:3:13] INFO
o.a.d.e.p.i.xsort.ExternalSortBatch - Completed spilling to
/tmp/drill/spill/29b41f37-4803-d7ce-e05f-912d1f65da79/major_fragment_3/minor_fragment_13/operator_16/1
2015-11-18 02:22:13,346 [29b41f37-4803-d7ce-e05f-912d1f65da79:frag:3:13] WARN
o.a.d.e.p.i.xsort.ExternalSortBatch - Starting to merge. 34 batch groups.
Current allocated memory: 2252186
2015-11-18 02:22:13,363 [29b41f37-4803-d7ce-e05f-912d1f65da79:frag:3:13] INFO
o.a.d.e.w.fragment.FragmentExecutor -
29b41f37-4803-d7ce-e05f-912d1f65da79:3:13: State change requested RUNNING -->
FAILED
2015-11-18 02:22:13,370 [29b41f37-4803-d7ce-e05f-912d1f65da79:frag:3:13] INFO
o.a.d.e.w.fragment.FragmentExecutor -
29b41f37-4803-d7ce-e05f-912d1f65da79:3:13: State change requested FAILED -->
FINISHED
2015-11-18 02:22:13,371 [29b41f37-4803-d7ce-e05f-912d1f65da79:frag:3:13] ERROR
o.a.d.e.w.fragment.FragmentExecutor - SYSTEM ERROR: NullPointerException
Fragment 3:13
[Error Id: c5d67dcb-16aa-4951-89f5-599b4b4eb54d on atsqa4-133.qa.lab:31010]
org.apache.drill.common.exceptions.UserException: SYSTEM ERROR:
NullPointerException
Fragment 3:13
[Error Id: c5d67dcb-16aa-4951-89f5-599b4b4eb54d on atsqa4-133.qa.lab:31010]
at
org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:534)
~[drill-common-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:321)
[drill-java-exec-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:184)
[drill-java-exec-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:290)
[drill-java-exec-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
at
org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
[drill-common-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[na:1.7.0_71]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_71]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_71]
java.lang.NullPointerException: null
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (DRILL-4109) NPE in external sort
[
https://issues.apache.org/jira/browse/DRILL-4109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Victoria Markman updated DRILL-4109:
Attachment: 29b41f37-4803-d7ce-e05f-912d1f65da79.sys.drill
> NPE in external sort
> -
>
> Key: DRILL-4109
> URL: https://issues.apache.org/jira/browse/DRILL-4109
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.4.0
>Reporter: Victoria Markman
>Priority: Blocker
> Attachments: 29b41f37-4803-d7ce-e05f-912d1f65da79.sys.drill
>
>
> 4 node cluster
> 36GB of direct memory
> 4GB heap memory
> planner.memory.max_query_memory_per_node=2GB (default)
> planner.enable_hashjoin = false
> Spill directory has 6.4T of memory available:
> {noformat}
> [Tue Nov 17 18:23:18 /tmp/drill ] # df -H .
> Filesystem Size Used Avail Use% Mounted on
> localhost:/mapr 7.7T 1.4T 6.4T 18% /mapr
> {noformat}
> Run query below:
> framework/resources/Advanced/tpcds/tpcds_sf100/original/query15.sql
> drillbit.log
> {code}
> 2015-11-18 02:22:12,639 [29b41f37-4803-d7ce-e05f-912d1f65da79:frag:3:9] INFO
> o.a.d.e.p.i.xsort.ExternalSortBatch - Merging and spilling to
> /tmp/drill/spill/29b41f37-4803-d7ce-e05f-912d1f65da79/major_fragment_3/minor_fragment_9/operator_17/7
> 2015-11-18 02:22:12,770 [29b41f37-4803-d7ce-e05f-912d1f65da79:frag:3:5] INFO
> o.a.d.e.p.i.xsort.ExternalSortBatch - Merging and spilling to
> /tmp/drill/spill/29b41f37-4803-d7ce-e05f-912d1f65da79/major_fragment_3/minor_fragment_5/operator_17/7
> 2015-11-18 02:22:13,345 [29b41f37-4803-d7ce-e05f-912d1f65da79:frag:3:17] INFO
> o.a.d.e.p.i.xsort.ExternalSortBatch - Completed spilling to
> /tmp/drill/spill/29b41f37-4803-d7ce-e05f-912d1f65da79/major_fragment_3/minor_fragment_17/operator_17/7
> 2015-11-18 02:22:13,346 [29b41f37-4803-d7ce-e05f-912d1f65da79:frag:3:13] INFO
> o.a.d.e.p.i.xsort.ExternalSortBatch - Completed spilling to
> /tmp/drill/spill/29b41f37-4803-d7ce-e05f-912d1f65da79/major_fragment_3/minor_fragment_13/operator_16/1
> 2015-11-18 02:22:13,346 [29b41f37-4803-d7ce-e05f-912d1f65da79:frag:3:13] WARN
> o.a.d.e.p.i.xsort.ExternalSortBatch - Starting to merge. 34 batch groups.
> Current allocated memory: 2252186
> 2015-11-18 02:22:13,363 [29b41f37-4803-d7ce-e05f-912d1f65da79:frag:3:13] INFO
> o.a.d.e.w.fragment.FragmentExecutor -
> 29b41f37-4803-d7ce-e05f-912d1f65da79:3:13: State change requested RUNNING -->
> FAILED
> 2015-11-18 02:22:13,370 [29b41f37-4803-d7ce-e05f-912d1f65da79:frag:3:13] INFO
> o.a.d.e.w.fragment.FragmentExecutor -
> 29b41f37-4803-d7ce-e05f-912d1f65da79:3:13: State change requested FAILED -->
> FINISHED
> 2015-11-18 02:22:13,371 [29b41f37-4803-d7ce-e05f-912d1f65da79:frag:3:13]
> ERROR o.a.d.e.w.fragment.FragmentExecutor - SYSTEM ERROR: NullPointerException
> Fragment 3:13
> [Error Id: c5d67dcb-16aa-4951-89f5-599b4b4eb54d on atsqa4-133.qa.lab:31010]
> org.apache.drill.common.exceptions.UserException: SYSTEM ERROR:
> NullPointerException
> Fragment 3:13
> [Error Id: c5d67dcb-16aa-4951-89f5-599b4b4eb54d on atsqa4-133.qa.lab:31010]
> at
> org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:534)
> ~[drill-common-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:321)
> [drill-java-exec-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:184)
> [drill-java-exec-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:290)
> [drill-java-exec-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
> at
> org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
> [drill-common-1.4.0-SNAPSHOT.jar:1.4.0-SNAPSHOT]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> [na:1.7.0_71]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> [na:1.7.0_71]
> at java.lang.Thread.run(Thread.java:745) [na:1.7.0_71]
> java.lang.NullPointerException: null
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-4096) Incorrect result when we use coalesce in a join condition along with other filters
[
https://issues.apache.org/jira/browse/DRILL-4096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15009128#comment-15009128
]
Victoria Markman commented on DRILL-4096:
-
[~rkins] Is this specific to Hive only ?
> Incorrect result when we use coalesce in a join condition along with other
> filters
> --
>
> Key: DRILL-4096
> URL: https://issues.apache.org/jira/browse/DRILL-4096
> Project: Apache Drill
> Issue Type: Bug
> Components: Functions - Drill, Storage - Hive
>Affects Versions: 1.3.0
>Reporter: Rahul Challapalli
>Priority: Critical
>
> git.commit.id.abbrev=447f8ba
> The below query returns no results which is wrong based on the data set.
> Interestingly if we remove the second filter we get a cannot plan exception
> from drill. Will raise a different jira, if I cannot find an existing one
> {code}
> select * from hive.null_schemachange d, hive.onlynulls n where d.date_col =
> coalesce(n.date_col, date '2038-04-10', n.date_col) and d.date_col >
> '2015-01-01';
> {code}
> Hive DDL :
> {code}
> drop table if exists null_schemachange;
> create external table null_schemachange (
> int_col int,
> bigint_col bigint,
> date_col date,
> time_col string,
> timestamp_col timestamp,
> interval_col string,
> varchar_col string,
> float_col float,
> double_col double,
> bool_col boolean
> )
> ROW FORMAT DELIMITED FIELDS TERMINATED BY "|"
> LOCATION '/drill/testdata/hive_storage/null_schemachange.tbl'
> TBLPROPERTIES ("serialization.null.format"="null");
> drop table if exists onlynulls;
> create external table onlynulls (
> int_col int,
> bigint_col bigint,
> date_col date,
> time_col string,
> timestamp_col timestamp,
> interval_col string,
> varchar_col string,
> float_col float,
> double_col double,
> bool_col boolean
> )
> ROW FORMAT DELIMITED FIELDS TERMINATED BY "|"
> LOCATION '/drill/testdata/hive_storage/onlynulls.tbl'
> TBLPROPERTIES ("serialization.null.format"="null");
> {code}
> The data files are attached
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-4095) Regression: TPCDS query 30 throws can not plan exception
[
https://issues.apache.org/jira/browse/DRILL-4095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15007539#comment-15007539
]
Victoria Markman commented on DRILL-4095:
-
Note: when bug is fixed, enable
Advanced/tpcds/tpcds_sf1/original/text/query30.sql.drill-4095.fail
> Regression: TPCDS query 30 throws can not plan exception
>
>
> Key: DRILL-4095
> URL: https://issues.apache.org/jira/browse/DRILL-4095
> Project: Apache Drill
> Issue Type: New Feature
> Components: Query Planning & Optimization
>Affects Versions: 1.2.0, 1.3.0
>Reporter: Victoria Markman
>
> Fails in: mapr-drill-1.2.0.201510190924-1.noarch.rpm
> mapr-drill-1.3.0.20152348-1.noarch.rpm
>
> It looks like this bug was fixed as part of DRILL-2949 , but has regressed
> since then.
> Reproduction with data shipped with drill (TPCH):
> {code}
> 0: jdbc:drill:schema=dfs> select
> . . . . . . . . . . . . > count(*)
> . . . . . . . . . . . . > from
> . . . . . . . . . . . . > cp.`tpch/nation.parquet` a,
> . . . . . . . . . . . . > cp.`tpch/region.parquet` b
> . . . . . . . . . . . . > where
> . . . . . . . . . . . . > a.n_regionkey > (select max(b.r_regionkey)
> from cp.`tpch/region.parquet` b where b.r_nationkey = a.n_nationkey);
> Error: UNSUPPORTED_OPERATION ERROR: This query cannot be planned possibly due
> to either a cartesian join or an inequality join
> [Error Id: a52ca497-f654-46ba-b1a7-20d1c0147129 on atsqa4-133.qa.lab:31010]
> (state=,code=0)
> {code}
> If you remove table from the join, query executes:
> {code}
> 0: jdbc:drill:schema=dfs> select
> . . . . . . . . . . . . > count(*)
> . . . . . . . . . . . . > from
> . . . . . . . . . . . . > cp.`tpch/nation.parquet` a
> . . . . . . . . . . . . > --cp.`tpch/region.parquet` b
> . . . . . . . . . . . . > where
> . . . . . . . . . . . . > a.n_regionkey > (select max(b.r_regionkey)
> from cp.`tpch/region.parquet` b where b.r_nationkey = a.n_nationkey);
> +-+
> | EXPR$0 |
> +-+
> | 0 |
> +-+
> 1 row selected (0.921 seconds)
> {code}
> This affects TPCDS query 30:
> {code}
> 0: jdbc:drill:schema=dfs> WITH customer_total_return
> . . . . . . . . . . . . > AS (SELECT wr_returning_customer_sk AS
> ctr_customer_sk,
> . . . . . . . . . . . . > ca_state AS
> ctr_state,
> . . . . . . . . . . . . > Sum(wr_return_amt) AS
> ctr_total_return
> . . . . . . . . . . . . > FROM web_returns,
> . . . . . . . . . . . . > date_dim,
> . . . . . . . . . . . . > customer_address
> . . . . . . . . . . . . > WHERE wr_returned_date_sk = d_date_sk
> . . . . . . . . . . . . > AND d_year = 2000
> . . . . . . . . . . . . > AND wr_returning_addr_sk =
> ca_address_sk
> . . . . . . . . . . . . > GROUP BY wr_returning_customer_sk,
> . . . . . . . . . . . . >ca_state)
> . . . . . . . . . . . . > SELECT c_customer_id,
> . . . . . . . . . . . . >c_salutation,
> . . . . . . . . . . . . >c_first_name,
> . . . . . . . . . . . . >c_last_name,
> . . . . . . . . . . . . >c_preferred_cust_flag,
> . . . . . . . . . . . . >c_birth_day,
> . . . . . . . . . . . . >c_birth_month,
> . . . . . . . . . . . . >c_birth_year,
> . . . . . . . . . . . . >c_birth_country,
> . . . . . . . . . . . . >c_login,
> . . . . . . . . . . . . >c_email_address,
> . . . . . . . . . . . . >c_last_review_date,
> . . . . . . . . . . . . >ctr_total_return
> . . . . . . . . . . . . > FROM customer_total_return ctr1,
> . . . . . . . . . . . . >customer_address,
> . . . . . . . . . . . . >customer
> . . . . . . . . . . . . > WHERE ctr1.ctr_total_return > (SELECT
> Avg(ctr_total_return) * 1.2
> . . . . . . . . . . . . > FROM
> customer_total_return ctr2
> . . . . . . . . . . . . > WHERE
> ctr1.ctr_state = ctr2.ctr_state)
> . . . . . . . . . . . . >AND ca_address_sk = c_current_addr_sk
> . . . . . . . . . . . . >AND ca_state = 'IN'
> . . . . . . . . . . . . >AND ctr1.ctr_customer_sk = c_customer_sk
> . . . . . . . . . . . . > LIMIT 100;
> Error: UNSUPPORTED_OPERATION ERROR: This query cannot be planned possibly due
> to either a cartesian join or an inequality join
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (DRILL-4095) Regression: TPCDS query 30 throws can not plan exception
Victoria Markman created DRILL-4095:
---
Summary: Regression: TPCDS query 30 throws can not plan exception
Key: DRILL-4095
URL: https://issues.apache.org/jira/browse/DRILL-4095
Project: Apache Drill
Issue Type: New Feature
Components: Query Planning & Optimization
Affects Versions: 1.2.0, 1.3.0
Reporter: Victoria Markman
Fails in: mapr-drill-1.2.0.201510190924-1.noarch.rpm
mapr-drill-1.3.0.20152348-1.noarch.rpm
It looks like this bug was fixed as part of DRILL-2949 , but has regressed
since then.
Reproduction with data shipped with drill (TPCH):
{code}
0: jdbc:drill:schema=dfs> select
. . . . . . . . . . . . > count(*)
. . . . . . . . . . . . > from
. . . . . . . . . . . . > cp.`tpch/nation.parquet` a,
. . . . . . . . . . . . > cp.`tpch/region.parquet` b
. . . . . . . . . . . . > where
. . . . . . . . . . . . > a.n_regionkey > (select max(b.r_regionkey)
from cp.`tpch/region.parquet` b where b.r_nationkey = a.n_nationkey);
Error: UNSUPPORTED_OPERATION ERROR: This query cannot be planned possibly due
to either a cartesian join or an inequality join
[Error Id: a52ca497-f654-46ba-b1a7-20d1c0147129 on atsqa4-133.qa.lab:31010]
(state=,code=0)
{code}
If you remove table from the join, query executes:
{code}
0: jdbc:drill:schema=dfs> select
. . . . . . . . . . . . > count(*)
. . . . . . . . . . . . > from
. . . . . . . . . . . . > cp.`tpch/nation.parquet` a
. . . . . . . . . . . . > --cp.`tpch/region.parquet` b
. . . . . . . . . . . . > where
. . . . . . . . . . . . > a.n_regionkey > (select max(b.r_regionkey)
from cp.`tpch/region.parquet` b where b.r_nationkey = a.n_nationkey);
+-+
| EXPR$0 |
+-+
| 0 |
+-+
1 row selected (0.921 seconds)
{code}
This affects TPCDS query 30:
{code}
0: jdbc:drill:schema=dfs> WITH customer_total_return
. . . . . . . . . . . . > AS (SELECT wr_returning_customer_sk AS
ctr_customer_sk,
. . . . . . . . . . . . > ca_state AS ctr_state,
. . . . . . . . . . . . > Sum(wr_return_amt) AS
ctr_total_return
. . . . . . . . . . . . > FROM web_returns,
. . . . . . . . . . . . > date_dim,
. . . . . . . . . . . . > customer_address
. . . . . . . . . . . . > WHERE wr_returned_date_sk = d_date_sk
. . . . . . . . . . . . > AND d_year = 2000
. . . . . . . . . . . . > AND wr_returning_addr_sk =
ca_address_sk
. . . . . . . . . . . . > GROUP BY wr_returning_customer_sk,
. . . . . . . . . . . . >ca_state)
. . . . . . . . . . . . > SELECT c_customer_id,
. . . . . . . . . . . . >c_salutation,
. . . . . . . . . . . . >c_first_name,
. . . . . . . . . . . . >c_last_name,
. . . . . . . . . . . . >c_preferred_cust_flag,
. . . . . . . . . . . . >c_birth_day,
. . . . . . . . . . . . >c_birth_month,
. . . . . . . . . . . . >c_birth_year,
. . . . . . . . . . . . >c_birth_country,
. . . . . . . . . . . . >c_login,
. . . . . . . . . . . . >c_email_address,
. . . . . . . . . . . . >c_last_review_date,
. . . . . . . . . . . . >ctr_total_return
. . . . . . . . . . . . > FROM customer_total_return ctr1,
. . . . . . . . . . . . >customer_address,
. . . . . . . . . . . . >customer
. . . . . . . . . . . . > WHERE ctr1.ctr_total_return > (SELECT
Avg(ctr_total_return) * 1.2
. . . . . . . . . . . . > FROM
customer_total_return ctr2
. . . . . . . . . . . . > WHERE ctr1.ctr_state
= ctr2.ctr_state)
. . . . . . . . . . . . >AND ca_address_sk = c_current_addr_sk
. . . . . . . . . . . . >AND ca_state = 'IN'
. . . . . . . . . . . . >AND ctr1.ctr_customer_sk = c_customer_sk
. . . . . . . . . . . . > LIMIT 100;
Error: UNSUPPORTED_OPERATION ERROR: This query cannot be planned possibly due
to either a cartesian join or an inequality join
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (DRILL-867) tpcds queries 6, 9 and 10 fail to plan
[
https://issues.apache.org/jira/browse/DRILL-867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Victoria Markman resolved DRILL-867.
Resolution: Fixed
> tpcds queries 6, 9 and 10 fail to plan
> --
>
> Key: DRILL-867
> URL: https://issues.apache.org/jira/browse/DRILL-867
> Project: Apache Drill
> Issue Type: Bug
> Components: SQL Parser
>Reporter: Krystal
>Priority: Minor
> Fix For: Future
>
>
> git.commit.id.abbrev=e1e5ea0
> git.commit.time=29.05.2014 @ 15\:32\:29 PDT
> query 6:
> {code}
> select * from (select a.ca_state state, count(*) cnt
> from customer_address a
> ,customer c
> ,store_sales s
> ,date_dim d
> ,item i
> where a.ca_address_sk = c.c_current_addr_sk
> and c.c_customer_sk = s.ss_customer_sk
> and s.ss_sold_date_sk = d.d_date_sk
> and s.ss_item_sk = i.i_item_sk
> and d.d_month_seq =
>(select distinct (d.d_month_seq)
> from date_dim d
>where d.d_year = 1998
> and d.d_moy = 5 )
> and i.i_current_price > 1.2 *
> (select avg(j.i_current_price)
>from item j
>where j.i_category = i.i_category)
> group by a.ca_state
> having count(*) >= 10
> order by cnt
> ) limit 100;
> {code}
> query 7:
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-867) tpcds queries 6, 9 and 10 fail to plan
[
https://issues.apache.org/jira/browse/DRILL-867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15007335#comment-15007335
]
Victoria Markman commented on DRILL-867:
I'm going to add these queries to the regression suite as negative tests to
make sure we don't regress anything. Resolving as fixed for now.
> tpcds queries 6, 9 and 10 fail to plan
> --
>
> Key: DRILL-867
> URL: https://issues.apache.org/jira/browse/DRILL-867
> Project: Apache Drill
> Issue Type: Bug
> Components: SQL Parser
>Reporter: Krystal
>Priority: Minor
> Fix For: Future
>
>
> git.commit.id.abbrev=e1e5ea0
> git.commit.time=29.05.2014 @ 15\:32\:29 PDT
> query 6:
> {code}
> select * from (select a.ca_state state, count(*) cnt
> from customer_address a
> ,customer c
> ,store_sales s
> ,date_dim d
> ,item i
> where a.ca_address_sk = c.c_current_addr_sk
> and c.c_customer_sk = s.ss_customer_sk
> and s.ss_sold_date_sk = d.d_date_sk
> and s.ss_item_sk = i.i_item_sk
> and d.d_month_seq =
>(select distinct (d.d_month_seq)
> from date_dim d
>where d.d_year = 1998
> and d.d_moy = 5 )
> and i.i_current_price > 1.2 *
> (select avg(j.i_current_price)
>from item j
>where j.i_category = i.i_category)
> group by a.ca_state
> having count(*) >= 10
> order by cnt
> ) limit 100;
> {code}
> query 7:
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-867) tpcds queries 6, 9 and 10 fail to plan
[
https://issues.apache.org/jira/browse/DRILL-867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15007330#comment-15007330
]
Victoria Markman commented on DRILL-867:
All these queries throw proper unsupported errors now and have enhancement
requests associated with the unsupported features:
query 6 (DRILL-3801)
{code}
0: jdbc:drill:schema=dfs> select * from (select a.ca_state state, count(*) cnt
. . . . . . . . . . . . > from customer_address a
. . . . . . . . . . . . > ,customer c
. . . . . . . . . . . . > ,store_sales s
. . . . . . . . . . . . > ,date_dim d
. . . . . . . . . . . . > ,item i
. . . . . . . . . . . . > where a.ca_address_sk = c.c_current_addr_sk
. . . . . . . . . . . . > and c.c_customer_sk = s.ss_customer_sk
. . . . . . . . . . . . > and s.ss_sold_date_sk = d.d_date_sk
. . . . . . . . . . . . > and s.ss_item_sk = i.i_item_sk
. . . . . . . . . . . . > and d.d_month_seq =
. . . . . . . . . . . . >(select distinct (d.d_month_seq)
. . . . . . . . . . . . > from date_dim d
. . . . . . . . . . . . >where d.d_year = 1998
. . . . . . . . . . . . > and d.d_moy = 5 )
. . . . . . . . . . . . > and i.i_current_price > 1.2 *
. . . . . . . . . . . . > (select avg(j.i_current_price)
. . . . . . . . . . . . >from item j
. . . . . . . . . . . . >where j.i_category = i.i_category)
. . . . . . . . . . . . > group by a.ca_state
. . . . . . . . . . . . > having count(*) >= 10
. . . . . . . . . . . . > order by cnt
. . . . . . . . . . . . > ) limit 100;
Error: UNSUPPORTED_OPERATION ERROR: Non-scalar sub-query used in an expression
See Apache Drill JIRA: DRILL-1937
[Error Id: 841dae77-8b74-484e-a0f0-a01a2c07fbd8 on atsqa4-133.qa.lab:31010]
(state=,code=0)
{code}
query 9 ( DRILL-4092)
{code}
0: jdbc:drill:schema=dfs> select * from (select store.s_store_name
. . . . . . . . . . . . > ,sum(ss.ss_net_profit)
. . . . . . . . . . . . > from store_sales ss
. . . . . . . . . . . . > ,date_dim dd
. . . . . . . . . . . . > ,store,
. . . . . . . . . . . . > (select ca_zip
. . . . . . . . . . . . > from (
. . . . . . . . . . . . > (SELECT substr(ca.ca_zip,1,5) ca_zip
. . . . . . . . . . . . > FROM customer_address ca
. . . . . . . . . . . . > WHERE substr(ca.ca_zip,1,5) IN (
. . . . . . . . . . . . >
'16733','50732','51878','16933','33177','55974',
. . . . . . . . . . . . >
'21338','90455','63106','78712','45114',
. . . . . . . . . . . . >
'51090','44881','35526','91360','34986',
. . . . . . . . . . . . >
'31893','28853','84061','25483','84541',
. . . . . . . . . . . . >
'39275','56211','51199','85189','24292',
. . . . . . . . . . . . >
'27477','46388','77218','21137','43660',
. . . . . . . . . . . . >
'36509','77925','11691','26790','35256',
. . . . . . . . . . . . >
'59221','42491','39214','35273','27293',
. . . . . . . . . . . . >
'74258','68798','50936','19136','25240',
. . . . . . . . . . . . >
'89163','21667','30941','61680','10425',
. . . . . . . . . . . . >
'96787','84569','37596','84291','44843',
. . . . . . . . . . . . >
'31487','24949','31269','62115','79494',
. . . . . . . . . . . . >
'32194','62531','61655','40724','29091',
. . . . . . . . . . . . >
'81608','77126','32704','79045','19008',
. . . . . . . . . . . . >
'81581','59693','24689','79355','19635',
. . . . . . . . . . . . >
'52025','83585','56103','80150','26203',
. . . . . . . . . . . . >
'81571','85657','39672','62868','33498',
. . . . . . . . . . . . >
'69453','25748','44145','35695','57860',
. . . . . . . . . . . . >
'59532','76967','81235','22004','34487',
. . . . . . . . . . . . >
'48499','47318','63039','77728','89774',
. . . . . . . . . . . . >
'91640','76501','70137','37512','48507',
. . . . . . . . . . . . >
'51980','34851','54884','30905','12745',
. . . . . . . . . . . . >
'60630','42798','39923','47591','82518',
. . . . . . . . . . . . >
'32982','14233','56444','79278','57791',
. . . . . . . . . . . . >
'37395','93812','14062','21556','58923',
. . . . . . . . . . . . >
'13595','87261','79484','24492','10389',
. . . . . . . . . . . . >
'89526','21733','85078','35187','68025',
. . .
[jira] [Created] (DRILL-4092) Support for INTERSECT
Victoria Markman created DRILL-4092: --- Summary: Support for INTERSECT Key: DRILL-4092 URL: https://issues.apache.org/jira/browse/DRILL-4092 Project: Apache Drill Issue Type: New Feature Reporter: Victoria Markman -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-3549) Default value for planner.memory.max_query_memory_per_node needs to be increased
[ https://issues.apache.org/jira/browse/DRILL-3549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Victoria Markman updated DRILL-3549: Labels: usability (was: ) > Default value for planner.memory.max_query_memory_per_node needs to be > increased > > > Key: DRILL-3549 > URL: https://issues.apache.org/jira/browse/DRILL-3549 > Project: Apache Drill > Issue Type: Bug > Components: Query Planning & Optimization >Affects Versions: 1.2.0 >Reporter: Abhishek Girish >Assignee: Deneche A. Hakim >Priority: Critical > Labels: usability > Fix For: 1.4.0 > > > The current default value for planner.memory.max_query_memory_per_node is > 2147483648 (2 GB). This value is not enough, given the addition of window > function support. Most queries on reasonably sized data & cluster setup fail > with OOM due to insufficient memory. > The improve usability, the default needs to be increased to a reasonably > sized value (could be determined based on Drill Max Direct Memory). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-3543) Add stats for external sort to a query profile
[
https://issues.apache.org/jira/browse/DRILL-3543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Victoria Markman updated DRILL-3543:
Priority: Critical (was: Major)
> Add stats for external sort to a query profile
> --
>
> Key: DRILL-3543
> URL: https://issues.apache.org/jira/browse/DRILL-3543
> Project: Apache Drill
> Issue Type: Improvement
> Components: Execution - Relational Operators
>Affects Versions: 1.1.0
>Reporter: Victoria Markman
>Assignee: Deneche A. Hakim
>Priority: Critical
> Labels: usability
> Fix For: Future
>
>
> The only indication if sort spilled to disk today is info from the
> drillbit.log.
> It would be great if this information was displayed in the query profile.
> {code}
> 015-07-22 23:47:29,907 [2a4fd46e-f8c3-6b96-b165-b665a41be311:frag:0:0] INFO
> o.a.d.e.p.i.xsort.ExternalSortBatch - Completed spilling to
> /tmp/drill/spill/2a4fd46e-f8c3-6b96-b165-b665a41be311/major_fragment_0/minor_fragment_0/operator_7/92
> 2015-07-22 23:47:29,919 [2a4fd46e-f8c3-6b96-b165-b665a41be311:frag:0:0] INFO
> o.a.d.e.p.i.xsort.ExternalSortBatch - Merging and spilling to
> /tmp/drill/spill/2a4fd46e-f8c3-6b96-b165-b665a41be311/major_fragment_0/minor_fragment_0/operator_7/93
> 2015-07-22 23:47:29,919 [2a4fd46e-f8c3-6b96-b165-b665a41be311:frag:0:0] INFO
> o.a.d.e.p.i.xsort.ExternalSortBatch - Completed spilling to
> /tmp/drill/spill/2a4fd46e-f8c3-6b96-b165-b665a41be311/major_fragment_0/minor_fragment_0/operator_7/93
> 2015-07-22 23:47:29,919 [2a4fd46e-f8c3-6b96-b165-b665a41be311:frag:0:0] WARN
> o.a.d.e.p.i.xsort.ExternalSortBatch - Starting to merge. 7 batch groups.
> Current allocated memory: 11566787
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (DRILL-3543) Add stats for external sort to a query profile
[
https://issues.apache.org/jira/browse/DRILL-3543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Victoria Markman updated DRILL-3543:
Labels: usability (was: )
> Add stats for external sort to a query profile
> --
>
> Key: DRILL-3543
> URL: https://issues.apache.org/jira/browse/DRILL-3543
> Project: Apache Drill
> Issue Type: Improvement
> Components: Execution - Relational Operators
>Affects Versions: 1.1.0
>Reporter: Victoria Markman
>Assignee: Deneche A. Hakim
>Priority: Critical
> Labels: usability
> Fix For: Future
>
>
> The only indication if sort spilled to disk today is info from the
> drillbit.log.
> It would be great if this information was displayed in the query profile.
> {code}
> 015-07-22 23:47:29,907 [2a4fd46e-f8c3-6b96-b165-b665a41be311:frag:0:0] INFO
> o.a.d.e.p.i.xsort.ExternalSortBatch - Completed spilling to
> /tmp/drill/spill/2a4fd46e-f8c3-6b96-b165-b665a41be311/major_fragment_0/minor_fragment_0/operator_7/92
> 2015-07-22 23:47:29,919 [2a4fd46e-f8c3-6b96-b165-b665a41be311:frag:0:0] INFO
> o.a.d.e.p.i.xsort.ExternalSortBatch - Merging and spilling to
> /tmp/drill/spill/2a4fd46e-f8c3-6b96-b165-b665a41be311/major_fragment_0/minor_fragment_0/operator_7/93
> 2015-07-22 23:47:29,919 [2a4fd46e-f8c3-6b96-b165-b665a41be311:frag:0:0] INFO
> o.a.d.e.p.i.xsort.ExternalSortBatch - Completed spilling to
> /tmp/drill/spill/2a4fd46e-f8c3-6b96-b165-b665a41be311/major_fragment_0/minor_fragment_0/operator_7/93
> 2015-07-22 23:47:29,919 [2a4fd46e-f8c3-6b96-b165-b665a41be311:frag:0:0] WARN
> o.a.d.e.p.i.xsort.ExternalSortBatch - Starting to merge. 7 batch groups.
> Current allocated memory: 11566787
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-4071) Partition pruning fails when a Coalesce() function appears with partition filter
[
https://issues.apache.org/jira/browse/DRILL-4071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14999819#comment-14999819
]
Victoria Markman commented on DRILL-4071:
-
Similar bug: https://issues.apache.org/jira/browse/DRILL-3862
> Partition pruning fails when a Coalesce() function appears with partition
> filter
>
>
> Key: DRILL-4071
> URL: https://issues.apache.org/jira/browse/DRILL-4071
> Project: Apache Drill
> Issue Type: Bug
> Components: Query Planning & Optimization
>Reporter: Aman Sinha
>Assignee: Mehant Baid
>
> Pruning fails for this query:
> {code}
> 0: jdbc:drill:zk=local> explain plan for select count(*) from
> dfs.`/Users/asinha/data/multilevel/parquet` where dir0 = 1994 and
> coalesce(o_clerk, 'Clerk') = '';
> +--+--+
> | text | json |
> +--+--+
> | 00-00Screen
> 00-01 Project(EXPR$0=[$0])
> 00-02StreamAgg(group=[{}], EXPR$0=[COUNT()])
> 00-03 Project($f0=[0])
> 00-04SelectionVectorRemover
> 00-05 Filter(condition=[AND(=($0, 1994), =(CASE(IS NOT NULL($1),
> $1, 'Clerk'), ''))])
> 00-06Scan(groupscan=[ParquetGroupScan
> [entries=[ReadEntryWithPath
> [path=file:/Users/asinha/data/multilevel/parquet/1994/Q1/orders_94_q1.parquet],
> ReadEntryWithPath
> [path=file:/Users/asinha/data/multilevel/parquet/1994/Q2/orders_94_q2.parquet],
> ReadEntryWithPath
> [path=file:/Users/asinha/data/multilevel/parquet/1994/Q3/orders_94_q3.parquet],
> ReadEntryWithPath
> [path=file:/Users/asinha/data/multilevel/parquet/1994/Q4/orders_94_q4.parquet],
> ReadEntryWithPath
> [path=file:/Users/asinha/data/multilevel/parquet/1995/Q1/orders_95_q1.parquet],
> ReadEntryWithPath
> [path=file:/Users/asinha/data/multilevel/parquet/1995/Q2/orders_95_q2.parquet],
> ReadEntryWithPath
> [path=file:/Users/asinha/data/multilevel/parquet/1995/Q3/orders_95_q3.parquet],
> ReadEntryWithPath
> [path=file:/Users/asinha/data/multilevel/parquet/1995/Q4/orders_95_q4.parquet],
> ReadEntryWithPath
> [path=file:/Users/asinha/data/multilevel/parquet/1996/Q1/orders_96_q1.parquet],
> ReadEntryWithPath
> [path=file:/Users/asinha/data/multilevel/parquet/1996/Q2/orders_96_q2.parquet],
> ReadEntryWithPath
> [path=file:/Users/asinha/data/multilevel/parquet/1996/Q3/orders_96_q3.parquet],
> ReadEntryWithPath
> [path=file:/Users/asinha/data/multilevel/parquet/1996/Q4/orders_96_q4.parquet]],
> selectionRoot=file:/Users/asinha/data/multilevel/parquet, numFiles=12,
> usedMetadataFile=false, columns=[`dir0`, `o_clerk`]]])
> {code}
> The log indicates no partition filters were found:
> {code}
> ...
> o.a.d.e.p.l.partition.PruneScanRule - No conditions were found eligible for
> partition pruning.Total pruning elapsed time: 0 ms
> {code}
> A preliminary analysis indicates that since the Coalesce gets converted to a
> CASE(IS NOT NULL) expression, the filter analysis does not correctly
> process the full expression tree. At one point in
> {{FindPartitionConditions.analyzeCall()}} I saw the operandStack had 3
> elements in it: [NO_PUSH, NO_PUSH, PUSH] which seemed strange since I would
> expect even number of elements.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (DRILL-3847) Exception during partition pruning
[
https://issues.apache.org/jira/browse/DRILL-3847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Victoria Markman updated DRILL-3847:
Attachment: drillbit.log
This is the new error I got with the latest 1.3.0
#Sat Nov 07 18:09:40 UTC 2015
git.commit.id.abbrev=3a73f09
git.commit.user.email=jacq...@apache.org
> Exception during partition pruning
> --
>
> Key: DRILL-3847
> URL: https://issues.apache.org/jira/browse/DRILL-3847
> Project: Apache Drill
> Issue Type: Bug
> Components: Query Planning & Optimization
>Affects Versions: 1.2.0
>Reporter: Victoria Markman
> Attachments: drillbit.log, drillbit.log
>
>
> * This is happening when I run query below on a table partitioned by
> cd_demo_sk which resulted in ~1.9 million files in the single directory.
> * Metadata file is present customer_demographics_part_1
> * If cast to varchar is replaced with the star in projection list: query works
> * If cast to varchar replaced with integer columns: query works
> * Same query runs to completion on the original table.
> {code}
> select
> cast(cd_gender as varchar(10)),
> cast(cd_credit_rating as varchar(10)),
> cast(cd_marital_status as varchar(10)),
> cd_dep_count
> from
> customer_demographics_part_1
> where cd_demo_sk = 1;
> {code}
> drillbit.log
> {code}
> 2015-09-28 20:39:20,653 [29f65bb1-6f25-9d81-3225-c7ae40adbbf3:foreman] WARN
> o.a.d.e.p.l.partition.PruneScanRule - Exception while trying to prune
> partition.
> java.lang.ClassCastException: java.util.LinkedHashMap cannot be cast to
> parquet.io.api.Binary
> at
> org.apache.drill.exec.store.parquet.ParquetGroupScan.populatePruningVector(ParquetGroupScan.java:403)
> ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at
> org.apache.drill.exec.planner.ParquetPartitionDescriptor.populatePartitionVectors(ParquetPartitionDescriptor.java:96)
> ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at
> org.apache.drill.exec.planner.logical.partition.PruneScanRule.doOnMatch(PruneScanRule.java:212)
> ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at
> org.apache.drill.exec.planner.logical.partition.ParquetPruneScanRule$2.onMatch(ParquetPruneScanRule.java:87)
> [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at
> org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:228)
> [calcite-core-1.4.0-drill-r5.jar:1.4.0-drill-r5]
> at
> org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:808)
> [calcite-core-1.4.0-drill-r5.jar:1.4.0-drill-r5]
> at
> org.apache.calcite.tools.Programs$RuleSetProgram.run(Programs.java:303)
> [calcite-core-1.4.0-drill-r5.jar:1.4.0-drill-r5]
> at
> org.apache.calcite.prepare.PlannerImpl.transform(PlannerImpl.java:303)
> [calcite-core-1.4.0-drill-r5.jar:1.4.0-drill-r5]
> at
> org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.logicalPlanningVolcanoAndLopt(DefaultSqlHandler.java:545)
> [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at
> org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel(DefaultSqlHandler.java:213)
> [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at
> org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel(DefaultSqlHandler.java:248)
> [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at
> org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan(DefaultSqlHandler.java:164)
> [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at
> org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan(DrillSqlWorker.java:178)
> [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.foreman.Foreman.runSQL(Foreman.java:905)
> [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:244)
> [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> [na:1.7.0_71]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> [na:1.7.0_71]
> at java.lang.Thread.run(Thread.java:745) [na:1.7.0_71]
> 2015-09-28 20:39:24,779 [29f65bb1-6f25-9d81-3225-c7ae40adbbf3:foreman] WARN
> o.a.d.e.p.l.partition.PruneScanRule - Exception while trying to prune
> partition.
> java.lang.ClassCastException: java.util.LinkedHashMap cannot be cast to
> parquet.io.api.Binary
> at
> org.apache.drill.exec.store.parquet.ParquetGroupScan.populatePruningVector(ParquetGroupScan.java:403)
> ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at
> org.apache.drill.exec.planner.ParquetPartitionDescriptor.pop
[jira] [Updated] (DRILL-4060) CTAS to csv or json files gives incorrect time values
[ https://issues.apache.org/jira/browse/DRILL-4060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Victoria Markman updated DRILL-4060: Priority: Critical (was: Major) > CTAS to csv or json files gives incorrect time values > - > > Key: DRILL-4060 > URL: https://issues.apache.org/jira/browse/DRILL-4060 > Project: Apache Drill > Issue Type: Bug > Components: Storage - Writer >Reporter: Krystal >Priority: Critical > > I have a csv file with the following data: > select columns[8], columns[10] from `interval_data.csv`; > +---+--+ > | EXPR$0 |EXPR$1| > +---+--+ > | 01:35:56 | 2015-01-24 07:27:05 | > | 12:22:07 | 2014-05-25 03:41:54 | > | 00:01:28 | 2014-07-30 08:03:11 | > | 00:00:01 | 2014-09-15 02:33:11 | > | 06:59:59 | 2014-06-17 13:04:09 | > | 23:38:16 | 2015-02-01 02:02:37 | > | 15:00:00 | 2014-08-16 13:11:12 | > | 05:55:36 | 2014-04-13 15:06:36 | > | 10:48:36 | 2013-03-23 01:24:20 | > +---+--+ > I created a json file using CTAS: > alter session set `store.format` = 'json'; > create table `test1.json` as select cast(columns[8] as time) > c_time,cast(columns[10] as timestamp) c_timestamp from `interval_data.csv`; > select c_time, c_timestamp from `test1.json`; > +---+--+ > |c_time | c_timestamp| > +---+--+ > | 09:35:56.000 | 2015-01-24 15:27:05.000 | > | 20:22:07.000 | 2014-05-25 10:41:54.000 | > | 08:01:28.000 | 2014-07-30 15:03:11.000 | > | 08:00:01.000 | 2014-09-15 09:33:11.000 | > | 14:59:59.000 | 2014-06-17 20:04:09.000 | > | 07:38:16.000 | 2015-02-01 10:02:37.000 | > | 23:00:00.000 | 2014-08-16 20:11:12.000 | > | 13:55:36.000 | 2014-04-13 22:06:36.000 | > | 18:48:36.000 | 2013-03-23 08:24:20.000 | > +---+--+ > Notice that the times have 8 hours added to the original values. > I got the same result when create another CSV file using CTAS on the same > data. > For CTAS as parquet, however, the resulting data is the same as the orginal > data. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-4041) Parquet library update causing random "Buffer has negative reference count"
[
https://issues.apache.org/jira/browse/DRILL-4041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14992919#comment-14992919
]
Victoria Markman commented on DRILL-4041:
-
I've seen it in a different scenario:
https://issues.apache.org/jira/browse/DRILL-3665
> Parquet library update causing random "Buffer has negative reference count"
> ---
>
> Key: DRILL-4041
> URL: https://issues.apache.org/jira/browse/DRILL-4041
> Project: Apache Drill
> Issue Type: Bug
> Components: Storage - Parquet
>Affects Versions: 1.3.0
>Reporter: Rahul Challapalli
>Assignee: Steven Phillips
>Priority: Critical
>
> git commit # 39582bd60c9e9b16aba4f099d434e927e7e5
> After the parquet library update commit, we started seeing the below error
> randomly causing failures in the Extended Functional Suite.
> {code}
> Failed with exception
> java.lang.IllegalArgumentException: Buffer has negative reference count.
> at
> oadd.com.google.common.base.Preconditions.checkArgument(Preconditions.java:92)
> at oadd.io.netty.buffer.DrillBuf.release(DrillBuf.java:250)
> at oadd.io.netty.buffer.DrillBuf.release(DrillBuf.java:259)
> at oadd.io.netty.buffer.DrillBuf.release(DrillBuf.java:259)
> at oadd.io.netty.buffer.DrillBuf.release(DrillBuf.java:259)
> at oadd.io.netty.buffer.DrillBuf.release(DrillBuf.java:239)
> at
> oadd.org.apache.drill.exec.vector.BaseDataValueVector.clear(BaseDataValueVector.java:39)
> at
> oadd.org.apache.drill.exec.vector.NullableIntVector.clear(NullableIntVector.java:150)
> at
> oadd.org.apache.drill.exec.record.SimpleVectorWrapper.clear(SimpleVectorWrapper.java:84)
> at
> oadd.org.apache.drill.exec.record.VectorContainer.zeroVectors(VectorContainer.java:312)
> at
> oadd.org.apache.drill.exec.record.VectorContainer.clear(VectorContainer.java:296)
> at
> oadd.org.apache.drill.exec.record.RecordBatchLoader.clear(RecordBatchLoader.java:183)
> at
> org.apache.drill.jdbc.impl.DrillResultSetImpl.cleanup(DrillResultSetImpl.java:139)
> at org.apache.drill.jdbc.impl.DrillCursor.close(DrillCursor.java:333)
> at
> oadd.net.hydromatic.avatica.AvaticaResultSet.close(AvaticaResultSet.java:110)
> at
> org.apache.drill.jdbc.impl.DrillResultSetImpl.close(DrillResultSetImpl.java:169)
> at
> org.apache.drill.test.framework.DrillTestJdbc.executeQuery(DrillTestJdbc.java:233)
> at
> org.apache.drill.test.framework.DrillTestJdbc.run(DrillTestJdbc.java:89)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:744)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-3807) [Regression] Query with equality join and a FALSE condition fails to plan
[
https://issues.apache.org/jira/browse/DRILL-3807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14992191#comment-14992191
]
Victoria Markman commented on DRILL-3807:
-
*For QA reference*
This bug is affecting following tests in Advanced suite:
{code}
/root/drillAutomation/framework-master/framework/resources/Advanced/mondrian/query6146.q
/root/drillAutomation/framework-master/framework/resources/Advanced/mondrian/query860.q
/root/drillAutomation/framework-master/framework/resources/Advanced/mondrian/query862.q
/root/drillAutomation/framework-master/framework/resources/Advanced/mondrian/query6330.q
/root/drillAutomation/framework-master/framework/resources/Advanced/mondrian/query771.q
/root/drillAutomation/framework-master/framework/resources/Advanced/mondrian/query808.q
/root/drillAutomation/framework-master/framework/resources/Advanced/mondrian/query540.q
/root/drillAutomation/framework-master/framework/resources/Advanced/mondrian/query863.q
/root/drillAutomation/framework-master/framework/resources/Advanced/mondrian/query861.q
{code}
> [Regression] Query with equality join and a FALSE condition fails to plan
> -
>
> Key: DRILL-3807
> URL: https://issues.apache.org/jira/browse/DRILL-3807
> Project: Apache Drill
> Issue Type: Bug
> Components: Query Planning & Optimization
>Affects Versions: 1.2.0
>Reporter: Aman Sinha
>Assignee: Sean Hsuan-Yi Chu
> Fix For: 1.4.0
>
>
> 1.2.0-SNAPSHOT behavior:
> {code}
> 0: jdbc:drill:zk=local> explain plan for select l.l_quantity from
> cp.`tpch/lineitem.parquet` l, cp.`tpch/part.parquet` p where l.l_partkey =
> p.p_partkey and (1 = 0);
> Error: UNSUPPORTED_OPERATION ERROR: This query cannot be planned possibly due
> to either a cartesian join or an inequality join
> [Error Id: f7466d86-b709-465e-bb49-d3c51ecf941b on 172.16.0.160:31010]
> (state=,code=0)
> {code}
> The simplification of ' l.l_partkey = p.p_partkey and (1 = 0)' to a False
> condition is valid and accordingly Drill fails to plan due to the cartesian
> join introduced by the False condition. However, in 1.1.0 apparently the
> 1=0 was converted to a LIMIT 0 which was pushed below the Join and the query
> successfully planned and executed:
> 1.1.0 behavior:
> {code}
> 0: jdbc:drill:zk=local> explain plan for select l.l_quantity from
> cp.`tpch/lineitem.parquet` l, cp.`tpch/part.parquet` p where l.l_partkey =
> p.p_partkey and (1 = 0);
> +--+--+
> | text | json |
> +--+--+
> | 00-00Screen
> 00-01 Project(l_quantity=[$1])
> 00-02HashJoin(condition=[=($0, $2)], joinType=[inner])
> 00-04 SelectionVectorRemover
> 00-05Limit(offset=[0], fetch=[0])
> 00-06 Scan(groupscan=[ParquetGroupScan
> [entries=[ReadEntryWithPath [path=classpath:/tpch/lineitem.parquet]],
> selectionRoot=classpath:/tpch/lineitem.parquet, numFiles=1,
> columns=[`l_partkey`, `l_quantity`]]])
> 00-03 Scan(groupscan=[ParquetGroupScan [entries=[ReadEntryWithPath
> [path=classpath:/tpch/part.parquet]],
> selectionRoot=classpath:/tpch/part.parquet, numFiles=1,
> columns=[`p_partkey`]]])
> {code}
> [~cchang] and I looked at the commit history and it appears that the
> regression started somewhere between Aug 24 and Aug 28, which is the time
> when we rebased on Calcite 1.4.0. So we need to narrow down further the
> change that may have caused this.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-3937) We are not pruning when we have a metadata cache and auto partitioned data in some cases
[
https://issues.apache.org/jira/browse/DRILL-3937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14992151#comment-14992151
]
Victoria Markman commented on DRILL-3937:
-
yep, will do that. Thanks !
> We are not pruning when we have a metadata cache and auto partitioned data in
> some cases
>
>
> Key: DRILL-3937
> URL: https://issues.apache.org/jira/browse/DRILL-3937
> Project: Apache Drill
> Issue Type: Bug
> Components: Metadata
>Reporter: Rahul Challapalli
>Assignee: Aman Sinha
> Fix For: 1.3.0
>
> Attachments: 1_0_9998.parquet, 1_0_.parquet
>
>
> git.commit.id.abbrev=2736412
> The below plan indicates that we are not pruning
> {code}
> explain plan for select count(*) from dfs.`/drill/comscore/orders2` where
> o_clerk='Clerk#79443';
> +--+--+
> | text | json |
> +--+--+
> | 00-00Screen
> 00-01 Project(EXPR$0=[$0])
> 00-02StreamAgg(group=[{}], EXPR$0=[COUNT()])
> 00-03 Project($f0=[0])
> 00-04SelectionVectorRemover
> 00-05 Filter(condition=[=($0, 'Clerk#79443')])
> 00-06Scan(groupscan=[ParquetGroupScan
> [entries=[ReadEntryWithPath
> [path=maprfs:///drill/comscore/orders2/1_0_.parquet], ReadEntryWithPath
> [path=maprfs:///drill/comscore/orders2/1_0_9998.parquet]],
> selectionRoot=/drill/comscore/orders2, numFiles=2, usedMetadataFile=true,
> columns=[`o_clerk`]]])
> {code}
> Error from the logs
> {code}
> 2015-10-15 01:24:28,467 [29e0ffb4-1c91-f40a-8bf0-5e3665dcf107:foreman] WARN
> o.a.d.e.p.l.partition.PruneScanRule - Exception while trying to prune
> partition.
> java.lang.ClassCastException: java.util.LinkedHashMap cannot be cast to
> parquet.io.api.Binary
> at
> org.apache.drill.exec.store.parquet.ParquetGroupScan.populatePruningVector(ParquetGroupScan.java:414)
> ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at
> org.apache.drill.exec.planner.ParquetPartitionDescriptor.populatePartitionVectors(ParquetPartitionDescriptor.java:96)
> ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at
> org.apache.drill.exec.planner.logical.partition.PruneScanRule.doOnMatch(PruneScanRule.java:212)
> ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at
> org.apache.drill.exec.planner.logical.partition.ParquetPruneScanRule$2.onMatch(ParquetPruneScanRule.java:87)
> [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at
> org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:228)
> [calcite-core-1.4.0-drill-r6.jar:1.4.0-drill-r6]
> at
> org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:808)
> [calcite-core-1.4.0-drill-r6.jar:1.4.0-drill-r6]
> at
> org.apache.calcite.tools.Programs$RuleSetProgram.run(Programs.java:303)
> [calcite-core-1.4.0-drill-r6.jar:1.4.0-drill-r6]
> at
> org.apache.calcite.prepare.PlannerImpl.transform(PlannerImpl.java:303)
> [calcite-core-1.4.0-drill-r6.jar:1.4.0-drill-r6]
> at
> org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.logicalPlanningVolcanoAndLopt(DefaultSqlHandler.java:545)
> [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at
> org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel(DefaultSqlHandler.java:213)
> [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at
> org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel(DefaultSqlHandler.java:248)
> [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at
> org.apache.drill.exec.planner.sql.handlers.ExplainHandler.getPlan(ExplainHandler.java:61)
> [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at
> org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan(DrillSqlWorker.java:178)
> [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.foreman.Foreman.runSQL(Foreman.java:905)
> [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:244)
> [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> [na:1.7.0_71]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> [na:1.7.0_71]
> at java.lang.Thread.run(Thread.java:745) [na:1.7.0_71]
> {code}
> The partition column type in this case is binary which could be causing the
> issue.
> Partition pruning seems to be working when we have Metadata Caching + Auto
> Partitioned Files with integer partition column
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-3937) We are not pruning when we have a metadata cache and auto partitioned data in some cases
[
https://issues.apache.org/jira/browse/DRILL-3937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14992105#comment-14992105
]
Victoria Markman commented on DRILL-3937:
-
[~zfong] this will be my next step.
> We are not pruning when we have a metadata cache and auto partitioned data in
> some cases
>
>
> Key: DRILL-3937
> URL: https://issues.apache.org/jira/browse/DRILL-3937
> Project: Apache Drill
> Issue Type: Bug
> Components: Metadata
>Reporter: Rahul Challapalli
>Assignee: Aman Sinha
> Fix For: 1.3.0
>
> Attachments: 1_0_9998.parquet, 1_0_.parquet
>
>
> git.commit.id.abbrev=2736412
> The below plan indicates that we are not pruning
> {code}
> explain plan for select count(*) from dfs.`/drill/comscore/orders2` where
> o_clerk='Clerk#79443';
> +--+--+
> | text | json |
> +--+--+
> | 00-00Screen
> 00-01 Project(EXPR$0=[$0])
> 00-02StreamAgg(group=[{}], EXPR$0=[COUNT()])
> 00-03 Project($f0=[0])
> 00-04SelectionVectorRemover
> 00-05 Filter(condition=[=($0, 'Clerk#79443')])
> 00-06Scan(groupscan=[ParquetGroupScan
> [entries=[ReadEntryWithPath
> [path=maprfs:///drill/comscore/orders2/1_0_.parquet], ReadEntryWithPath
> [path=maprfs:///drill/comscore/orders2/1_0_9998.parquet]],
> selectionRoot=/drill/comscore/orders2, numFiles=2, usedMetadataFile=true,
> columns=[`o_clerk`]]])
> {code}
> Error from the logs
> {code}
> 2015-10-15 01:24:28,467 [29e0ffb4-1c91-f40a-8bf0-5e3665dcf107:foreman] WARN
> o.a.d.e.p.l.partition.PruneScanRule - Exception while trying to prune
> partition.
> java.lang.ClassCastException: java.util.LinkedHashMap cannot be cast to
> parquet.io.api.Binary
> at
> org.apache.drill.exec.store.parquet.ParquetGroupScan.populatePruningVector(ParquetGroupScan.java:414)
> ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at
> org.apache.drill.exec.planner.ParquetPartitionDescriptor.populatePartitionVectors(ParquetPartitionDescriptor.java:96)
> ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at
> org.apache.drill.exec.planner.logical.partition.PruneScanRule.doOnMatch(PruneScanRule.java:212)
> ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at
> org.apache.drill.exec.planner.logical.partition.ParquetPruneScanRule$2.onMatch(ParquetPruneScanRule.java:87)
> [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at
> org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:228)
> [calcite-core-1.4.0-drill-r6.jar:1.4.0-drill-r6]
> at
> org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:808)
> [calcite-core-1.4.0-drill-r6.jar:1.4.0-drill-r6]
> at
> org.apache.calcite.tools.Programs$RuleSetProgram.run(Programs.java:303)
> [calcite-core-1.4.0-drill-r6.jar:1.4.0-drill-r6]
> at
> org.apache.calcite.prepare.PlannerImpl.transform(PlannerImpl.java:303)
> [calcite-core-1.4.0-drill-r6.jar:1.4.0-drill-r6]
> at
> org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.logicalPlanningVolcanoAndLopt(DefaultSqlHandler.java:545)
> [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at
> org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel(DefaultSqlHandler.java:213)
> [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at
> org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel(DefaultSqlHandler.java:248)
> [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at
> org.apache.drill.exec.planner.sql.handlers.ExplainHandler.getPlan(ExplainHandler.java:61)
> [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at
> org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan(DrillSqlWorker.java:178)
> [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.foreman.Foreman.runSQL(Foreman.java:905)
> [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:244)
> [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> [na:1.7.0_71]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> [na:1.7.0_71]
> at java.lang.Thread.run(Thread.java:745) [na:1.7.0_71]
> {code}
> The partition column type in this case is binary which could be causing the
> issue.
> Partition pruning seems to be working when we have Metadata Caching + Auto
> Partitioned Files with integer partition column
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-3937) We are not pruning when we have a metadata cache and auto partitioned data in some cases
[
https://issues.apache.org/jira/browse/DRILL-3937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14992049#comment-14992049
]
Victoria Markman commented on DRILL-3937:
-
[~rkins] does this bug look like DRILL-3847 to you ? Reproductions are
different, but the stack trace looks very similar to me.
> We are not pruning when we have a metadata cache and auto partitioned data in
> some cases
>
>
> Key: DRILL-3937
> URL: https://issues.apache.org/jira/browse/DRILL-3937
> Project: Apache Drill
> Issue Type: Bug
> Components: Metadata
>Reporter: Rahul Challapalli
>Assignee: Aman Sinha
> Fix For: 1.3.0
>
> Attachments: 1_0_9998.parquet, 1_0_.parquet
>
>
> git.commit.id.abbrev=2736412
> The below plan indicates that we are not pruning
> {code}
> explain plan for select count(*) from dfs.`/drill/comscore/orders2` where
> o_clerk='Clerk#79443';
> +--+--+
> | text | json |
> +--+--+
> | 00-00Screen
> 00-01 Project(EXPR$0=[$0])
> 00-02StreamAgg(group=[{}], EXPR$0=[COUNT()])
> 00-03 Project($f0=[0])
> 00-04SelectionVectorRemover
> 00-05 Filter(condition=[=($0, 'Clerk#79443')])
> 00-06Scan(groupscan=[ParquetGroupScan
> [entries=[ReadEntryWithPath
> [path=maprfs:///drill/comscore/orders2/1_0_.parquet], ReadEntryWithPath
> [path=maprfs:///drill/comscore/orders2/1_0_9998.parquet]],
> selectionRoot=/drill/comscore/orders2, numFiles=2, usedMetadataFile=true,
> columns=[`o_clerk`]]])
> {code}
> Error from the logs
> {code}
> 2015-10-15 01:24:28,467 [29e0ffb4-1c91-f40a-8bf0-5e3665dcf107:foreman] WARN
> o.a.d.e.p.l.partition.PruneScanRule - Exception while trying to prune
> partition.
> java.lang.ClassCastException: java.util.LinkedHashMap cannot be cast to
> parquet.io.api.Binary
> at
> org.apache.drill.exec.store.parquet.ParquetGroupScan.populatePruningVector(ParquetGroupScan.java:414)
> ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at
> org.apache.drill.exec.planner.ParquetPartitionDescriptor.populatePartitionVectors(ParquetPartitionDescriptor.java:96)
> ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at
> org.apache.drill.exec.planner.logical.partition.PruneScanRule.doOnMatch(PruneScanRule.java:212)
> ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at
> org.apache.drill.exec.planner.logical.partition.ParquetPruneScanRule$2.onMatch(ParquetPruneScanRule.java:87)
> [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at
> org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:228)
> [calcite-core-1.4.0-drill-r6.jar:1.4.0-drill-r6]
> at
> org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:808)
> [calcite-core-1.4.0-drill-r6.jar:1.4.0-drill-r6]
> at
> org.apache.calcite.tools.Programs$RuleSetProgram.run(Programs.java:303)
> [calcite-core-1.4.0-drill-r6.jar:1.4.0-drill-r6]
> at
> org.apache.calcite.prepare.PlannerImpl.transform(PlannerImpl.java:303)
> [calcite-core-1.4.0-drill-r6.jar:1.4.0-drill-r6]
> at
> org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.logicalPlanningVolcanoAndLopt(DefaultSqlHandler.java:545)
> [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at
> org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel(DefaultSqlHandler.java:213)
> [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at
> org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToDrel(DefaultSqlHandler.java:248)
> [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at
> org.apache.drill.exec.planner.sql.handlers.ExplainHandler.getPlan(ExplainHandler.java:61)
> [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at
> org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan(DrillSqlWorker.java:178)
> [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.foreman.Foreman.runSQL(Foreman.java:905)
> [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:244)
> [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> [na:1.7.0_71]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> [na:1.7.0_71]
> at java.lang.Thread.run(Thread.java:745) [na:1.7.0_71]
> {code}
> The partition column type in this case is binary which could be causing the
> issue.
> Partition pruning seems to be working when we have Metadata Caching + Auto
> Partitioned Files with intege
[jira] [Updated] (DRILL-4021) Cannot subract or add between two timestamps
[
https://issues.apache.org/jira/browse/DRILL-4021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Victoria Markman updated DRILL-4021:
Attachment: photo-2.JPG
> Cannot subract or add between two timestamps
>
>
> Key: DRILL-4021
> URL: https://issues.apache.org/jira/browse/DRILL-4021
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Data Types
>Reporter: Krystal
> Attachments: Screen Shot 2015-11-03 at 3.44.30 PM.png, Screen Shot
> 2015-11-04 at 7.35.07 AM.png, Screen Shot 2015-11-04 at 8.12.34 AM.png,
> photo-2.JPG
>
>
> The following subtraction between 2 now() function works:
> select now() - now()from voter_hive limit 1;
> +-+
> | EXPR$0 |
> +-+
> | PT0S|
> +-+
>
> However, the following queries fail:
> select now() - create_time from voter_hive where voter_id=1;
> Error: VALIDATION ERROR: From line 1, column 8 to line 1, column 26: Cannot
> apply '-' to arguments of type ' - '. Supported form(s):
> ' - '
> ' - '
> ' - '
> select create_time - cast('1997-02-12 15:18:31.072' as timestamp) from
> voter_hive where voter_id=1;
> Error: VALIDATION ERROR: From line 1, column 8 to line 1, column 65: Cannot
> apply '-' to arguments of type ' - '. Supported
> form(s): ' - '
> ' - '
> ' - '
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-4021) Cannot subract or add between two timestamps
[
https://issues.apache.org/jira/browse/DRILL-4021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14990300#comment-14990300
]
Victoria Markman commented on DRILL-4021:
-
You are correct !
Now I can proudly say 'yes' to the question on a slide (attached)
> Cannot subract or add between two timestamps
>
>
> Key: DRILL-4021
> URL: https://issues.apache.org/jira/browse/DRILL-4021
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Data Types
>Reporter: Krystal
> Attachments: Screen Shot 2015-11-03 at 3.44.30 PM.png, Screen Shot
> 2015-11-04 at 7.35.07 AM.png, Screen Shot 2015-11-04 at 8.12.34 AM.png
>
>
> The following subtraction between 2 now() function works:
> select now() - now()from voter_hive limit 1;
> +-+
> | EXPR$0 |
> +-+
> | PT0S|
> +-+
>
> However, the following queries fail:
> select now() - create_time from voter_hive where voter_id=1;
> Error: VALIDATION ERROR: From line 1, column 8 to line 1, column 26: Cannot
> apply '-' to arguments of type ' - '. Supported form(s):
> ' - '
> ' - '
> ' - '
> select create_time - cast('1997-02-12 15:18:31.072' as timestamp) from
> voter_hive where voter_id=1;
> Error: VALIDATION ERROR: From line 1, column 8 to line 1, column 65: Cannot
> apply '-' to arguments of type ' - '. Supported
> form(s): ' - '
> ' - '
> ' - '
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (DRILL-4021) Cannot subract or add between two timestamps
[
https://issues.apache.org/jira/browse/DRILL-4021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Victoria Markman updated DRILL-4021:
Attachment: Screen Shot 2015-11-04 at 8.12.34 AM.png
> Cannot subract or add between two timestamps
>
>
> Key: DRILL-4021
> URL: https://issues.apache.org/jira/browse/DRILL-4021
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Data Types
>Reporter: Krystal
> Attachments: Screen Shot 2015-11-03 at 3.44.30 PM.png, Screen Shot
> 2015-11-04 at 7.35.07 AM.png, Screen Shot 2015-11-04 at 8.12.34 AM.png
>
>
> The following subtraction between 2 now() function works:
> select now() - now()from voter_hive limit 1;
> +-+
> | EXPR$0 |
> +-+
> | PT0S|
> +-+
>
> However, the following queries fail:
> select now() - create_time from voter_hive where voter_id=1;
> Error: VALIDATION ERROR: From line 1, column 8 to line 1, column 26: Cannot
> apply '-' to arguments of type ' - '. Supported form(s):
> ' - '
> ' - '
> ' - '
> select create_time - cast('1997-02-12 15:18:31.072' as timestamp) from
> voter_hive where voter_id=1;
> Error: VALIDATION ERROR: From line 1, column 8 to line 1, column 65: Cannot
> apply '-' to arguments of type ' - '. Supported
> form(s): ' - '
> ' - '
> ' - '
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-4021) Cannot subract or add between two timestamps
[
https://issues.apache.org/jira/browse/DRILL-4021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14989820#comment-14989820
]
Victoria Markman commented on DRILL-4021:
-
Ha ! We are SQL 2011 compliant :) it does not require parenthesis.
Anyways, we need to deal with Drill's inconsistent behavior somehow. I'm going
to add some more tests on top of views and see what I find out.
Thanks Julian.
> Cannot subract or add between two timestamps
>
>
> Key: DRILL-4021
> URL: https://issues.apache.org/jira/browse/DRILL-4021
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Data Types
>Reporter: Krystal
> Attachments: Screen Shot 2015-11-03 at 3.44.30 PM.png, Screen Shot
> 2015-11-04 at 7.35.07 AM.png, Screen Shot 2015-11-04 at 8.12.34 AM.png
>
>
> The following subtraction between 2 now() function works:
> select now() - now()from voter_hive limit 1;
> +-+
> | EXPR$0 |
> +-+
> | PT0S|
> +-+
>
> However, the following queries fail:
> select now() - create_time from voter_hive where voter_id=1;
> Error: VALIDATION ERROR: From line 1, column 8 to line 1, column 26: Cannot
> apply '-' to arguments of type ' - '. Supported form(s):
> ' - '
> ' - '
> ' - '
> select create_time - cast('1997-02-12 15:18:31.072' as timestamp) from
> voter_hive where voter_id=1;
> Error: VALIDATION ERROR: From line 1, column 8 to line 1, column 65: Cannot
> apply '-' to arguments of type ' - '. Supported
> form(s): ' - '
> ' - '
> ' - '
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (DRILL-4021) Cannot subract or add between two timestamps
[
https://issues.apache.org/jira/browse/DRILL-4021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Victoria Markman updated DRILL-4021:
Attachment: Screen Shot 2015-11-03 at 3.44.30 PM.png
> Cannot subract or add between two timestamps
>
>
> Key: DRILL-4021
> URL: https://issues.apache.org/jira/browse/DRILL-4021
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Data Types
>Reporter: Krystal
> Attachments: Screen Shot 2015-11-03 at 3.44.30 PM.png
>
>
> The following subtraction between 2 now() function works:
> select now() - now()from voter_hive limit 1;
> +-+
> | EXPR$0 |
> +-+
> | PT0S|
> +-+
>
> However, the following queries fail:
> select now() - create_time from voter_hive where voter_id=1;
> Error: VALIDATION ERROR: From line 1, column 8 to line 1, column 26: Cannot
> apply '-' to arguments of type ' - '. Supported form(s):
> ' - '
> ' - '
> ' - '
> select create_time - cast('1997-02-12 15:18:31.072' as timestamp) from
> voter_hive where voter_id=1;
> Error: VALIDATION ERROR: From line 1, column 8 to line 1, column 65: Cannot
> apply '-' to arguments of type ' - '. Supported
> form(s): ' - '
> ' - '
> ' - '
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-4021) Cannot subract or add between two timestamps
[
https://issues.apache.org/jira/browse/DRILL-4021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14988604#comment-14988604
]
Victoria Markman commented on DRILL-4021:
-
Julian,
We need to remove addition from the title, it is not allowed, period :)
However, subtraction according to SQL 2011 ISO/IEC 9075-2:2011(E) is allowed
between two timestamps ( see excerpt attached and hopefully I'm understanding
it correctly )
It is supported in Postgres 9.3
{code}
postgres=# select c_timestamp, now() - c_timestamp as interval from test;
c_timestamp |interval
-+-
2015-03-01 00:11:15 | 247 days 16:11:45.11652
2015-03-01 00:20:46 | 247 days 16:02:14.11652
2015-03-01 00:38:12 | 247 days 15:44:48.11652
2015-03-01 00:53:00 | 247 days 15:30:00.11652
2015-03-01 01:07:15 | 247 days 15:15:45.11652
2015-03-01 01:14:14 | 247 days 15:08:46.11652
2015-03-01 01:30:41 | 247 days 14:52:19.11652
2015-03-01 01:42:47 | 247 days 14:40:13.11652
2015-03-01 01:49:46 | 247 days 14:33:14.11652
|
(10 rows)
{code}
It is also supported in drill:
{code}
0: jdbc:drill:schema=dfs> select c_timestamp, now() - c_timestamp from j2;
++---+
| c_timestamp | EXPR$1 |
++---+
| 2015-03-01 00:11:15.0 | P247DT86100.498S |
| 2015-03-01 00:20:46.0 | P247DT85529.498S |
| 2015-03-01 00:38:12.0 | P247DT84483.498S |
| 2015-03-01 00:53:00.0 | P247DT83595.498S |
| 2015-03-01 01:07:15.0 | P247DT82740.498S |
| 2015-03-01 01:14:14.0 | P247DT82321.498S |
| 2015-03-01 01:30:41.0 | P247DT81334.498S |
| 2015-03-01 01:42:47.0 | P247DT80608.498S |
| 2015-03-01 01:49:46.0 | P247DT80189.498S |
| null | null |
++---+
10 rows selected (0.283 seconds)
{code}
j2 is a parquet file:
{code}
[Tue Nov 03 16:10:29 ] # ~/parquet-tools/parquet-schema 0_0_0.parquet
message root {
optional binary c_varchar (UTF8);
optional int32 c_integer;
optional int64 c_bigint;
optional float c_float;
optional double c_double;
optional int32 c_date (DATE);
optional int32 c_time (TIME_MILLIS);
optional int64 c_timestamp (TIMESTAMP_MILLIS);
optional boolean c_boolean;
optional double d9;
optional double d18;
optional double d28;
optional double d38;
}
{code}
The reason this case failed for Krystal, I believe, is because hive table is a
"strongly typed data source", and this behavior is inconsistent with the rest
of our product.
In fact, if you create a drill view with cast to timestamp, you are going to
get the same error:
{code}
0: jdbc:drill:schema=dfs> create or replace view test_timestamp(c1) as select
CAST(c_timestamp as TIMESTAMP) from j2;
+---++
| ok |summary
|
+---++
| true | View 'test_timestamp' created successfully in 'dfs.subqueries' schema
|
+---++
1 row selected (0.337 seconds)
0: jdbc:drill:schema=dfs> select c1, now() - c1 from test_timestamp;
Error: VALIDATION ERROR: From line 1, column 12 to line 1, column 21: Cannot
apply '-' to arguments of type ' - '. Supported form(s):
' - '
' - '
' - '
[Error Id: 5928b2fc-0650-4096-a0a3-8be8337a1b8e on atsqa4-133.qa.lab:31010]
(state=,code=0)
{code}
We do have a problem: drill behaves inconsistently. [~jnadeau] mentioned during
our last hangout that he was trying to fix parquet types to be used during
planning and he got tons of errors. Jacques, you probably got hundreds of those
?
> Cannot subract or add between two timestamps
>
>
> Key: DRILL-4021
> URL: https://issues.apache.org/jira/browse/DRILL-4021
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Data Types
>Reporter: Krystal
>
> The following subtraction between 2 now() function works:
> select now() - now()from voter_hive limit 1;
> +-+
> | EXPR$0 |
> +-+
> | PT0S|
> +-+
>
> However, the following queries fail:
> select now() - create_time from voter_hive where voter_id=1;
> Error: VALIDATION ERROR: From line 1, column 8 to line 1, column 26: Cannot
> apply '-' to arguments of type ' - '. Supported form(s):
> ' - '
> ' - '
> ' - '
> select create_time - cast('1997-02-12 15:18:31.072' as timestamp) from
> voter_hive where voter_id=1;
> Error: VALIDATION ERROR: From line 1, column 8 to line 1, column 65: Cannot
> apply '-' to arguments of type ' - '. Supported
> form(s): ' - '
> ' - '
> ' - '
--
This mes
[jira] [Updated] (DRILL-4019) Limit 0 with two flatten operators in a query fails with unsupported exception
[
https://issues.apache.org/jira/browse/DRILL-4019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Victoria Markman updated DRILL-4019:
Affects Version/s: (was: 1.2.0)
> Limit 0 with two flatten operators in a query fails with unsupported exception
> --
>
> Key: DRILL-4019
> URL: https://issues.apache.org/jira/browse/DRILL-4019
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Relational Operators
>Reporter: Victoria Markman
> Labels: private_branch
>
> I think this is manifestation of DRILL-2256 in a different SQL scenario.
> test.json
> {code}
> {
> "c1" : [[1,2,3],[10,20,30]]
> }
> {code}
> Two flatten operators with limit zero fails:
> {code}
> 0: jdbc:drill:schema=dfs> select flatten(c1[0]), flatten(c1[1]) from
> `test.json` limit 0;
> Error: SYSTEM ERROR: UnsupportedOperationException: Unable to get value
> vector class for minor type [LATE] and mode [OPTIONAL]
> Fragment 0:0
> [Error Id: 0f19b566-ca53-43c7-ae0f-fec39c909cfd on atsqa4-133.qa.lab:31010]
> (state=,code=0)
> {code}
> Single flatten: works
> {code}
> 0: jdbc:drill:schema=dfs> select flatten(c1[0]) from `test.json` limit 0;
> +-+
> | EXPR$0 |
> +-+
> +-+
> No rows selected (0.258 seconds)
> {code}
> Without limit 0:
> {code}
> 0: jdbc:drill:schema=dfs> select flatten(c1[0]) from `test.json`;
> +-+
> | EXPR$0 |
> +-+
> | 1 |
> | 2 |
> | 3 |
> +-+
> 3 rows selected (0.268 seconds)
> 0: jdbc:drill:schema=dfs> select flatten(c1[0]), flatten(c1[1]) from
> `test.json`;
> +-+-+
> | EXPR$0 | EXPR$1 |
> +-+-+
> | 1 | 10 |
> | 1 | 20 |
> | 1 | 30 |
> | 2 | 10 |
> | 2 | 20 |
> | 2 | 30 |
> | 3 | 10 |
> | 3 | 20 |
> | 3 | 30 |
> +-+-+
> 9 rows selected (0.26 seconds)
> {code}
> drillbit.log
> {code}
> 2015-11-03 17:13:19,237 [29c714cf-fdc7-2ac9-5fd4-c4fdae636550:foreman] INFO
> o.a.d.e.s.schedule.BlockMapBuilder - Failure finding Drillbit running on host
> atsqa4-135.qa.lab. Skipping affinity to that host.
> 2015-11-03 17:13:19,237 [29c714cf-fdc7-2ac9-5fd4-c4fdae636550:foreman] INFO
> o.a.d.e.s.schedule.BlockMapBuilder - Failure finding Drillbit running on host
> atsqa4-134.qa.lab. Skipping affinity to that host.
> 2015-11-03 17:13:19,237 [29c714cf-fdc7-2ac9-5fd4-c4fdae636550:foreman] INFO
> o.a.d.e.s.schedule.BlockMapBuilder - Get block maps: Executed 1 out of 1
> using 1 threads. Time: 1ms total, 1.717980ms avg, 1ms max.
> 2015-11-03 17:13:19,237 [29c714cf-fdc7-2ac9-5fd4-c4fdae636550:foreman] INFO
> o.a.d.e.s.schedule.BlockMapBuilder - Get block maps: Executed 1 out of 1
> using 1 threads. Earliest start: 1.82 μs, Latest start: 1.82 μs,
> Average start: 1.82 μs .
> 2015-11-03 17:13:19,263 [29c714cf-fdc7-2ac9-5fd4-c4fdae636550:frag:0:0] INFO
> o.a.d.e.w.fragment.FragmentExecutor -
> 29c714cf-fdc7-2ac9-5fd4-c4fdae636550:0:0: State change requested
> AWAITING_ALLOCATION --> RUNNING
> 2015-11-03 17:13:19,263 [29c714cf-fdc7-2ac9-5fd4-c4fdae636550:frag:0:0] INFO
> o.a.d.e.w.f.FragmentStatusReporter -
> 29c714cf-fdc7-2ac9-5fd4-c4fdae636550:0:0: State to report: RUNNING
> 2015-11-03 17:13:19,269 [29c714cf-fdc7-2ac9-5fd4-c4fdae636550:frag:0:0] INFO
> o.a.d.e.w.fragment.FragmentExecutor -
> 29c714cf-fdc7-2ac9-5fd4-c4fdae636550:0:0: State change requested RUNNING -->
> FAILED
> 2015-11-03 17:13:19,269 [29c714cf-fdc7-2ac9-5fd4-c4fdae636550:frag:0:0] INFO
> o.a.d.e.w.fragment.FragmentExecutor -
> 29c714cf-fdc7-2ac9-5fd4-c4fdae636550:0:0: State change requested FAILED -->
> FINISHED
> 2015-11-03 17:13:19,271 [29c714cf-fdc7-2ac9-5fd4-c4fdae636550:frag:0:0] ERROR
> o.a.d.e.w.fragment.FragmentExecutor - SYSTEM ERROR:
> UnsupportedOperationException: Unable to get value vector class for minor
> type [LATE] and mode [OPTIONAL]
> Fragment 0:0
> [Error Id: 95cd27b5-065d-4714-a3e0-cd8d8ff6d519 on atsqa4-133.qa.lab:31010]
> org.apache.drill.common.exceptions.UserException: SYSTEM ERROR:
> UnsupportedOperationException: Unable to get value vector class for minor
> type [LATE] and mode [OPTIONAL]
> Fragment 0:0
> [Error Id: 95cd27b5-065d-4714-a3e0-cd8d8ff6d519 on atsqa4-133.qa.lab:31010]
> at
> org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:534)
> ~[drill-common-1.2.0.jar:1.2.0]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:323)
> [drill-java-exec-1.2.0.jar:1.2.0]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:178)
> [drill-java-exec-1.2.0.jar:1.2.0]
> at
> org.apache.drill.exec.work.fragment.F
[jira] [Updated] (DRILL-4019) Limit 0 with two flatten operators in a query fails with unsupported exception
[
https://issues.apache.org/jira/browse/DRILL-4019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Victoria Markman updated DRILL-4019:
Labels: private_branch (was: )
> Limit 0 with two flatten operators in a query fails with unsupported exception
> --
>
> Key: DRILL-4019
> URL: https://issues.apache.org/jira/browse/DRILL-4019
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Relational Operators
>Reporter: Victoria Markman
> Labels: private_branch
>
> I think this is manifestation of DRILL-2256 in a different SQL scenario.
> test.json
> {code}
> {
> "c1" : [[1,2,3],[10,20,30]]
> }
> {code}
> Two flatten operators with limit zero fails:
> {code}
> 0: jdbc:drill:schema=dfs> select flatten(c1[0]), flatten(c1[1]) from
> `test.json` limit 0;
> Error: SYSTEM ERROR: UnsupportedOperationException: Unable to get value
> vector class for minor type [LATE] and mode [OPTIONAL]
> Fragment 0:0
> [Error Id: 0f19b566-ca53-43c7-ae0f-fec39c909cfd on atsqa4-133.qa.lab:31010]
> (state=,code=0)
> {code}
> Single flatten: works
> {code}
> 0: jdbc:drill:schema=dfs> select flatten(c1[0]) from `test.json` limit 0;
> +-+
> | EXPR$0 |
> +-+
> +-+
> No rows selected (0.258 seconds)
> {code}
> Without limit 0:
> {code}
> 0: jdbc:drill:schema=dfs> select flatten(c1[0]) from `test.json`;
> +-+
> | EXPR$0 |
> +-+
> | 1 |
> | 2 |
> | 3 |
> +-+
> 3 rows selected (0.268 seconds)
> 0: jdbc:drill:schema=dfs> select flatten(c1[0]), flatten(c1[1]) from
> `test.json`;
> +-+-+
> | EXPR$0 | EXPR$1 |
> +-+-+
> | 1 | 10 |
> | 1 | 20 |
> | 1 | 30 |
> | 2 | 10 |
> | 2 | 20 |
> | 2 | 30 |
> | 3 | 10 |
> | 3 | 20 |
> | 3 | 30 |
> +-+-+
> 9 rows selected (0.26 seconds)
> {code}
> drillbit.log
> {code}
> 2015-11-03 17:13:19,237 [29c714cf-fdc7-2ac9-5fd4-c4fdae636550:foreman] INFO
> o.a.d.e.s.schedule.BlockMapBuilder - Failure finding Drillbit running on host
> atsqa4-135.qa.lab. Skipping affinity to that host.
> 2015-11-03 17:13:19,237 [29c714cf-fdc7-2ac9-5fd4-c4fdae636550:foreman] INFO
> o.a.d.e.s.schedule.BlockMapBuilder - Failure finding Drillbit running on host
> atsqa4-134.qa.lab. Skipping affinity to that host.
> 2015-11-03 17:13:19,237 [29c714cf-fdc7-2ac9-5fd4-c4fdae636550:foreman] INFO
> o.a.d.e.s.schedule.BlockMapBuilder - Get block maps: Executed 1 out of 1
> using 1 threads. Time: 1ms total, 1.717980ms avg, 1ms max.
> 2015-11-03 17:13:19,237 [29c714cf-fdc7-2ac9-5fd4-c4fdae636550:foreman] INFO
> o.a.d.e.s.schedule.BlockMapBuilder - Get block maps: Executed 1 out of 1
> using 1 threads. Earliest start: 1.82 μs, Latest start: 1.82 μs,
> Average start: 1.82 μs .
> 2015-11-03 17:13:19,263 [29c714cf-fdc7-2ac9-5fd4-c4fdae636550:frag:0:0] INFO
> o.a.d.e.w.fragment.FragmentExecutor -
> 29c714cf-fdc7-2ac9-5fd4-c4fdae636550:0:0: State change requested
> AWAITING_ALLOCATION --> RUNNING
> 2015-11-03 17:13:19,263 [29c714cf-fdc7-2ac9-5fd4-c4fdae636550:frag:0:0] INFO
> o.a.d.e.w.f.FragmentStatusReporter -
> 29c714cf-fdc7-2ac9-5fd4-c4fdae636550:0:0: State to report: RUNNING
> 2015-11-03 17:13:19,269 [29c714cf-fdc7-2ac9-5fd4-c4fdae636550:frag:0:0] INFO
> o.a.d.e.w.fragment.FragmentExecutor -
> 29c714cf-fdc7-2ac9-5fd4-c4fdae636550:0:0: State change requested RUNNING -->
> FAILED
> 2015-11-03 17:13:19,269 [29c714cf-fdc7-2ac9-5fd4-c4fdae636550:frag:0:0] INFO
> o.a.d.e.w.fragment.FragmentExecutor -
> 29c714cf-fdc7-2ac9-5fd4-c4fdae636550:0:0: State change requested FAILED -->
> FINISHED
> 2015-11-03 17:13:19,271 [29c714cf-fdc7-2ac9-5fd4-c4fdae636550:frag:0:0] ERROR
> o.a.d.e.w.fragment.FragmentExecutor - SYSTEM ERROR:
> UnsupportedOperationException: Unable to get value vector class for minor
> type [LATE] and mode [OPTIONAL]
> Fragment 0:0
> [Error Id: 95cd27b5-065d-4714-a3e0-cd8d8ff6d519 on atsqa4-133.qa.lab:31010]
> org.apache.drill.common.exceptions.UserException: SYSTEM ERROR:
> UnsupportedOperationException: Unable to get value vector class for minor
> type [LATE] and mode [OPTIONAL]
> Fragment 0:0
> [Error Id: 95cd27b5-065d-4714-a3e0-cd8d8ff6d519 on atsqa4-133.qa.lab:31010]
> at
> org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:534)
> ~[drill-common-1.2.0.jar:1.2.0]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:323)
> [drill-java-exec-1.2.0.jar:1.2.0]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:178)
> [drill-java-exec-1.2.0.jar:1.2.0]
> at
> org.apache.drill.exec.work.fragment.Fragm
[jira] [Updated] (DRILL-4012) Limit 0 on top of query with kvg/flatten results in assert
[
https://issues.apache.org/jira/browse/DRILL-4012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Victoria Markman updated DRILL-4012:
Labels: private-branch (was: )
> Limit 0 on top of query with kvg/flatten results in assert
> --
>
> Key: DRILL-4012
> URL: https://issues.apache.org/jira/browse/DRILL-4012
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Data Types
>Reporter: Victoria Markman
> Labels: private-branch
>
> I've found couple of bugs that are very similar, but none of them are quite
> the same:
> missing-map.json
> {code}
> {
> "id": 1,
> "m": {"a":1,"b":2}
> }
> {
> "id": 2
> }
> {
> "id": 3,
> "m": {"c":3,"d":4}
> }
> {code}
> 'limit 0' results in an assert:
> {code}
> 0: jdbc:drill:schema=dfs> select * from (select id, flatten(kvgen(m)) from
> `missing-map.json`) limit 0;
> Error: SYSTEM ERROR: ClassCastException: Cannot cast
> org.apache.drill.exec.vector.NullableIntVector to
> org.apache.drill.exec.vector.complex.RepeatedValueVector
> Fragment 0:0
> [Error Id: 046bb4d4-2c54-43ab-9577-cf21542ff8ef on atsqa4-133.qa.lab:31010]
> (state=,code=0)
> {code}
> 'limit 1' works:
> {code}
> 0: jdbc:drill:schema=dfs> select * from (select id, flatten(kvgen(m)) from
> `missing-map.json`) limit 1;
> +-++
> | id | EXPR$1 |
> +-++
> | 1 | {"key":"a","value":1} |
> +-++
> 1 row selected (0.247 seconds)
> {code}
> No limit, just in subquery: works
> {code}
> 0: jdbc:drill:schema=dfs> select * from (select id, flatten(kvgen(m)) from
> `json_kvgenflatten/missing-map.json`);
> +-++
> | id | EXPR$1 |
> +-++
> | 1 | {"key":"a","value":1} |
> | 1 | {"key":"b","value":2} |
> | 3 | {"key":"c","value":3} |
> | 3 | {"key":"d","value":4} |
> +-++
> 4 rows selected (0.247 seconds)
> {code}
> drillbit.log
> {code}
> 2015-11-03 15:23:20,943 [29c72e96-c9f6-9fce-ecf1-14eaa145f72b:foreman] INFO
> o.a.d.e.s.schedule.BlockMapBuilder - Failure finding Drillbit running on host
> atsqa4-136.qa.lab. Skipping affinity to that host.
> 2015-11-03 15:23:20,944 [29c72e96-c9f6-9fce-ecf1-14eaa145f72b:foreman] INFO
> o.a.d.e.s.schedule.BlockMapBuilder - Failure finding Drillbit running on host
> atsqa4-134.qa.lab. Skipping affinity to that host.
> 2015-11-03 15:23:20,944 [29c72e96-c9f6-9fce-ecf1-14eaa145f72b:foreman] INFO
> o.a.d.e.s.schedule.BlockMapBuilder - Get block maps: Executed 1 out of 1
> using 1 threads. Time: 1ms total, 1.530719ms avg, 1ms max.
> 2015-11-03 15:23:20,944 [29c72e96-c9f6-9fce-ecf1-14eaa145f72b:foreman] INFO
> o.a.d.e.s.schedule.BlockMapBuilder - Get block maps: Executed 1 out of 1
> using 1 threads. Earliest start: 1.744000 μs, Latest start: 1.744000 μs,
> Average start: 1.744000 μs .
> 2015-11-03 15:23:20,968 [29c72e96-c9f6-9fce-ecf1-14eaa145f72b:frag:0:0] INFO
> o.a.d.e.w.fragment.FragmentExecutor -
> 29c72e96-c9f6-9fce-ecf1-14eaa145f72b:0:0: State change requested
> AWAITING_ALLOCATION --> RUNNING
> 2015-11-03 15:23:20,968 [29c72e96-c9f6-9fce-ecf1-14eaa145f72b:frag:0:0] INFO
> o.a.d.e.w.f.FragmentStatusReporter -
> 29c72e96-c9f6-9fce-ecf1-14eaa145f72b:0:0: State to report: RUNNING
> 2015-11-03 15:23:20,974 [29c72e96-c9f6-9fce-ecf1-14eaa145f72b:frag:0:0] WARN
> o.a.d.e.e.ExpressionTreeMaterializer - Unable to find value vector of path
> `EXPR$3`, returning null instance.
> 2015-11-03 15:23:20,975 [29c72e96-c9f6-9fce-ecf1-14eaa145f72b:frag:0:0] WARN
> o.a.d.e.e.ExpressionTreeMaterializer - Unable to find value vector of path
> `EXPR$3`, returning null instance.
> 2015-11-03 15:23:20,976 [29c72e96-c9f6-9fce-ecf1-14eaa145f72b:frag:0:0] INFO
> o.a.d.e.w.fragment.FragmentExecutor -
> 29c72e96-c9f6-9fce-ecf1-14eaa145f72b:0:0: State change requested RUNNING -->
> FAILED
> 2015-11-03 15:23:20,976 [29c72e96-c9f6-9fce-ecf1-14eaa145f72b:frag:0:0] INFO
> o.a.d.e.w.fragment.FragmentExecutor -
> 29c72e96-c9f6-9fce-ecf1-14eaa145f72b:0:0: State change requested FAILED -->
> FINISHED
> 2015-11-03 15:23:20,978 [29c72e96-c9f6-9fce-ecf1-14eaa145f72b:frag:0:0] ERROR
> o.a.d.e.w.fragment.FragmentExecutor - SYSTEM ERROR: ClassCastException:
> Cannot cast org.apache.drill.exec.vector.NullableIntVector to
> org.apache.drill.exec.vector.complex.RepeatedValueVector
> Fragment 0:0
> [Error Id: c82c7f59-4dad-47ad-8901-5a2261c81279 on atsqa4-133.qa.lab:31010]
> org.apache.drill.common.exceptions.UserException: SYSTEM ERROR:
> ClassCastException: Cannot cast
> org.apache.drill.exec.vector.NullableIntVector to
> org.apache.drill.exec.vector.complex.RepeatedValueVector
> Fragment 0:0
> [Error Id: c82c7f59-4dad-47ad-8901-5a2261c8127
[jira] [Commented] (DRILL-4012) Limit 0 on top of query with kvg/flatten results in assert
[
https://issues.apache.org/jira/browse/DRILL-4012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14987854#comment-14987854
]
Victoria Markman commented on DRILL-4012:
-
This works for me in 1.3.0 as well (pre-union type fix):
{code}
0: jdbc:drill:schema=dfs> select * from (select id, flatten(kvgen(m)) from
`json_kvgenflatten/missing-map.json`) limit 0;
+-+-+
| id | EXPR$1 |
+-+-+
+-+-+
No rows selected (0.8 seconds)
{code}
> Limit 0 on top of query with kvg/flatten results in assert
> --
>
> Key: DRILL-4012
> URL: https://issues.apache.org/jira/browse/DRILL-4012
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Data Types
>Reporter: Victoria Markman
>
> I've found couple of bugs that are very similar, but none of them are quite
> the same:
> missing-map.json
> {code}
> {
> "id": 1,
> "m": {"a":1,"b":2}
> }
> {
> "id": 2
> }
> {
> "id": 3,
> "m": {"c":3,"d":4}
> }
> {code}
> 'limit 0' results in an assert:
> {code}
> 0: jdbc:drill:schema=dfs> select * from (select id, flatten(kvgen(m)) from
> `missing-map.json`) limit 0;
> Error: SYSTEM ERROR: ClassCastException: Cannot cast
> org.apache.drill.exec.vector.NullableIntVector to
> org.apache.drill.exec.vector.complex.RepeatedValueVector
> Fragment 0:0
> [Error Id: 046bb4d4-2c54-43ab-9577-cf21542ff8ef on atsqa4-133.qa.lab:31010]
> (state=,code=0)
> {code}
> 'limit 1' works:
> {code}
> 0: jdbc:drill:schema=dfs> select * from (select id, flatten(kvgen(m)) from
> `missing-map.json`) limit 1;
> +-++
> | id | EXPR$1 |
> +-++
> | 1 | {"key":"a","value":1} |
> +-++
> 1 row selected (0.247 seconds)
> {code}
> No limit, just in subquery: works
> {code}
> 0: jdbc:drill:schema=dfs> select * from (select id, flatten(kvgen(m)) from
> `json_kvgenflatten/missing-map.json`);
> +-++
> | id | EXPR$1 |
> +-++
> | 1 | {"key":"a","value":1} |
> | 1 | {"key":"b","value":2} |
> | 3 | {"key":"c","value":3} |
> | 3 | {"key":"d","value":4} |
> +-++
> 4 rows selected (0.247 seconds)
> {code}
> drillbit.log
> {code}
> 2015-11-03 15:23:20,943 [29c72e96-c9f6-9fce-ecf1-14eaa145f72b:foreman] INFO
> o.a.d.e.s.schedule.BlockMapBuilder - Failure finding Drillbit running on host
> atsqa4-136.qa.lab. Skipping affinity to that host.
> 2015-11-03 15:23:20,944 [29c72e96-c9f6-9fce-ecf1-14eaa145f72b:foreman] INFO
> o.a.d.e.s.schedule.BlockMapBuilder - Failure finding Drillbit running on host
> atsqa4-134.qa.lab. Skipping affinity to that host.
> 2015-11-03 15:23:20,944 [29c72e96-c9f6-9fce-ecf1-14eaa145f72b:foreman] INFO
> o.a.d.e.s.schedule.BlockMapBuilder - Get block maps: Executed 1 out of 1
> using 1 threads. Time: 1ms total, 1.530719ms avg, 1ms max.
> 2015-11-03 15:23:20,944 [29c72e96-c9f6-9fce-ecf1-14eaa145f72b:foreman] INFO
> o.a.d.e.s.schedule.BlockMapBuilder - Get block maps: Executed 1 out of 1
> using 1 threads. Earliest start: 1.744000 μs, Latest start: 1.744000 μs,
> Average start: 1.744000 μs .
> 2015-11-03 15:23:20,968 [29c72e96-c9f6-9fce-ecf1-14eaa145f72b:frag:0:0] INFO
> o.a.d.e.w.fragment.FragmentExecutor -
> 29c72e96-c9f6-9fce-ecf1-14eaa145f72b:0:0: State change requested
> AWAITING_ALLOCATION --> RUNNING
> 2015-11-03 15:23:20,968 [29c72e96-c9f6-9fce-ecf1-14eaa145f72b:frag:0:0] INFO
> o.a.d.e.w.f.FragmentStatusReporter -
> 29c72e96-c9f6-9fce-ecf1-14eaa145f72b:0:0: State to report: RUNNING
> 2015-11-03 15:23:20,974 [29c72e96-c9f6-9fce-ecf1-14eaa145f72b:frag:0:0] WARN
> o.a.d.e.e.ExpressionTreeMaterializer - Unable to find value vector of path
> `EXPR$3`, returning null instance.
> 2015-11-03 15:23:20,975 [29c72e96-c9f6-9fce-ecf1-14eaa145f72b:frag:0:0] WARN
> o.a.d.e.e.ExpressionTreeMaterializer - Unable to find value vector of path
> `EXPR$3`, returning null instance.
> 2015-11-03 15:23:20,976 [29c72e96-c9f6-9fce-ecf1-14eaa145f72b:frag:0:0] INFO
> o.a.d.e.w.fragment.FragmentExecutor -
> 29c72e96-c9f6-9fce-ecf1-14eaa145f72b:0:0: State change requested RUNNING -->
> FAILED
> 2015-11-03 15:23:20,976 [29c72e96-c9f6-9fce-ecf1-14eaa145f72b:frag:0:0] INFO
> o.a.d.e.w.fragment.FragmentExecutor -
> 29c72e96-c9f6-9fce-ecf1-14eaa145f72b:0:0: State change requested FAILED -->
> FINISHED
> 2015-11-03 15:23:20,978 [29c72e96-c9f6-9fce-ecf1-14eaa145f72b:frag:0:0] ERROR
> o.a.d.e.w.fragment.FragmentExecutor - SYSTEM ERROR: ClassCastException:
> Cannot cast org.apache.drill.exec.vector.NullableIntVector to
> org.apache.drill.exec.vector.complex.RepeatedValueVector
> Fragment 0:0
> [Error Id: c82c7f59-4dad-47ad-8901-5a2261c81279 on atsqa4-133.qa.lab:31
[jira] [Commented] (DRILL-4019) Limit 0 with two flatten operators in a query fails with unsupported exception
[
https://issues.apache.org/jira/browse/DRILL-4019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14987818#comment-14987818
]
Victoria Markman commented on DRILL-4019:
-
It does work with drill 1.3.0 before Steven's check in for union type vector
(if this is what you mean, Jacques)
{code}
0: jdbc:drill:schema=dfs> select * from sys.version;
+-+---+++--++
| version | commit_id |
commit_message
|commit_time | build_email | build_time
|
+-+---+++--++
| 1.3.0-SNAPSHOT | e4b94a78487f844be4fe71c4b9bf88b16c7f42f7 | DRILL-3937:
Handle the case where min/max columns in metadata cache file are string or
binary values. | 30.10.2015 @ 01:57:33 UTC | Unknown | 31.10.2015 @
01:10:48 UTC |
+-+---+++--++
1 row selected (0.7 seconds)
{code}
> Limit 0 with two flatten operators in a query fails with unsupported exception
> --
>
> Key: DRILL-4019
> URL: https://issues.apache.org/jira/browse/DRILL-4019
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Relational Operators
>Affects Versions: 1.2.0
>Reporter: Victoria Markman
>
> I think this is manifestation of DRILL-2256 in a different SQL scenario.
> test.json
> {code}
> {
> "c1" : [[1,2,3],[10,20,30]]
> }
> {code}
> Two flatten operators with limit zero fails:
> {code}
> 0: jdbc:drill:schema=dfs> select flatten(c1[0]), flatten(c1[1]) from
> `test.json` limit 0;
> Error: SYSTEM ERROR: UnsupportedOperationException: Unable to get value
> vector class for minor type [LATE] and mode [OPTIONAL]
> Fragment 0:0
> [Error Id: 0f19b566-ca53-43c7-ae0f-fec39c909cfd on atsqa4-133.qa.lab:31010]
> (state=,code=0)
> {code}
> Single flatten: works
> {code}
> 0: jdbc:drill:schema=dfs> select flatten(c1[0]) from `test.json` limit 0;
> +-+
> | EXPR$0 |
> +-+
> +-+
> No rows selected (0.258 seconds)
> {code}
> Without limit 0:
> {code}
> 0: jdbc:drill:schema=dfs> select flatten(c1[0]) from `test.json`;
> +-+
> | EXPR$0 |
> +-+
> | 1 |
> | 2 |
> | 3 |
> +-+
> 3 rows selected (0.268 seconds)
> 0: jdbc:drill:schema=dfs> select flatten(c1[0]), flatten(c1[1]) from
> `test.json`;
> +-+-+
> | EXPR$0 | EXPR$1 |
> +-+-+
> | 1 | 10 |
> | 1 | 20 |
> | 1 | 30 |
> | 2 | 10 |
> | 2 | 20 |
> | 2 | 30 |
> | 3 | 10 |
> | 3 | 20 |
> | 3 | 30 |
> +-+-+
> 9 rows selected (0.26 seconds)
> {code}
> drillbit.log
> {code}
> 2015-11-03 17:13:19,237 [29c714cf-fdc7-2ac9-5fd4-c4fdae636550:foreman] INFO
> o.a.d.e.s.schedule.BlockMapBuilder - Failure finding Drillbit running on host
> atsqa4-135.qa.lab. Skipping affinity to that host.
> 2015-11-03 17:13:19,237 [29c714cf-fdc7-2ac9-5fd4-c4fdae636550:foreman] INFO
> o.a.d.e.s.schedule.BlockMapBuilder - Failure finding Drillbit running on host
> atsqa4-134.qa.lab. Skipping affinity to that host.
> 2015-11-03 17:13:19,237 [29c714cf-fdc7-2ac9-5fd4-c4fdae636550:foreman] INFO
> o.a.d.e.s.schedule.BlockMapBuilder - Get block maps: Executed 1 out of 1
> using 1 threads. Time: 1ms total, 1.717980ms avg, 1ms max.
> 2015-11-03 17:13:19,237 [29c714cf-fdc7-2ac9-5fd4-c4fdae636550:foreman] INFO
> o.a.d.e.s.schedule.BlockMapBuilder - Get block maps: Executed 1 out of 1
> using 1 threads. Earliest start: 1.82 μs, Latest start: 1.82 μs,
> Average start: 1.82 μs .
> 2015-11-03 17:13:19,263 [29c714cf-fdc7-2ac9-5fd4-c4fdae636550:frag:0:0] INFO
> o.a.d.e.w.fragment.FragmentExecutor -
> 29c714cf-fdc7-2ac9-5fd4-c4fdae636550:0:0: State change requested
> AWAITING_ALLOCATION --> RUNNING
> 2015-11-03 17:13:19,263 [29c714cf-fdc7-2ac9-5fd4-c4fdae636550:frag:0:0] INFO
> o.a.d.e.w.f.FragmentStatusReporter -
> 29c714cf-fdc7-2ac9-5fd4-c4fdae636550:0:0: State to report: RUNNING
> 2015-11-03 17:13:19,269 [29c714cf-fdc7-2ac9-5fd4-c4fdae636550:frag:0:0] INFO
[jira] [Created] (DRILL-4019) Limit 0 with two flatten operators in a query fails with unsupported exception
Victoria Markman created DRILL-4019:
---
Summary: Limit 0 with two flatten operators in a query fails with
unsupported exception
Key: DRILL-4019
URL: https://issues.apache.org/jira/browse/DRILL-4019
Project: Apache Drill
Issue Type: Bug
Components: Execution - Relational Operators
Affects Versions: 1.2.0
Reporter: Victoria Markman
I think this is manifestation of DRILL-2256 in a different SQL scenario.
test.json
{code}
{
"c1" : [[1,2,3],[10,20,30]]
}
{code}
Two flatten operators with limit zero fails:
{code}
0: jdbc:drill:schema=dfs> select flatten(c1[0]), flatten(c1[1]) from
`test.json` limit 0;
Error: SYSTEM ERROR: UnsupportedOperationException: Unable to get value vector
class for minor type [LATE] and mode [OPTIONAL]
Fragment 0:0
[Error Id: 0f19b566-ca53-43c7-ae0f-fec39c909cfd on atsqa4-133.qa.lab:31010]
(state=,code=0)
{code}
Single flatten: works
{code}
0: jdbc:drill:schema=dfs> select flatten(c1[0]) from `test.json` limit 0;
+-+
| EXPR$0 |
+-+
+-+
No rows selected (0.258 seconds)
{code}
Without limit 0:
{code}
0: jdbc:drill:schema=dfs> select flatten(c1[0]) from `test.json`;
+-+
| EXPR$0 |
+-+
| 1 |
| 2 |
| 3 |
+-+
3 rows selected (0.268 seconds)
0: jdbc:drill:schema=dfs> select flatten(c1[0]), flatten(c1[1]) from
`test.json`;
+-+-+
| EXPR$0 | EXPR$1 |
+-+-+
| 1 | 10 |
| 1 | 20 |
| 1 | 30 |
| 2 | 10 |
| 2 | 20 |
| 2 | 30 |
| 3 | 10 |
| 3 | 20 |
| 3 | 30 |
+-+-+
9 rows selected (0.26 seconds)
{code}
drillbit.log
{code}
2015-11-03 17:13:19,237 [29c714cf-fdc7-2ac9-5fd4-c4fdae636550:foreman] INFO
o.a.d.e.s.schedule.BlockMapBuilder - Failure finding Drillbit running on host
atsqa4-135.qa.lab. Skipping affinity to that host.
2015-11-03 17:13:19,237 [29c714cf-fdc7-2ac9-5fd4-c4fdae636550:foreman] INFO
o.a.d.e.s.schedule.BlockMapBuilder - Failure finding Drillbit running on host
atsqa4-134.qa.lab. Skipping affinity to that host.
2015-11-03 17:13:19,237 [29c714cf-fdc7-2ac9-5fd4-c4fdae636550:foreman] INFO
o.a.d.e.s.schedule.BlockMapBuilder - Get block maps: Executed 1 out of 1 using
1 threads. Time: 1ms total, 1.717980ms avg, 1ms max.
2015-11-03 17:13:19,237 [29c714cf-fdc7-2ac9-5fd4-c4fdae636550:foreman] INFO
o.a.d.e.s.schedule.BlockMapBuilder - Get block maps: Executed 1 out of 1 using
1 threads. Earliest start: 1.82 μs, Latest start: 1.82 μs, Average
start: 1.82 μs .
2015-11-03 17:13:19,263 [29c714cf-fdc7-2ac9-5fd4-c4fdae636550:frag:0:0] INFO
o.a.d.e.w.fragment.FragmentExecutor - 29c714cf-fdc7-2ac9-5fd4-c4fdae636550:0:0:
State change requested AWAITING_ALLOCATION --> RUNNING
2015-11-03 17:13:19,263 [29c714cf-fdc7-2ac9-5fd4-c4fdae636550:frag:0:0] INFO
o.a.d.e.w.f.FragmentStatusReporter - 29c714cf-fdc7-2ac9-5fd4-c4fdae636550:0:0:
State to report: RUNNING
2015-11-03 17:13:19,269 [29c714cf-fdc7-2ac9-5fd4-c4fdae636550:frag:0:0] INFO
o.a.d.e.w.fragment.FragmentExecutor - 29c714cf-fdc7-2ac9-5fd4-c4fdae636550:0:0:
State change requested RUNNING --> FAILED
2015-11-03 17:13:19,269 [29c714cf-fdc7-2ac9-5fd4-c4fdae636550:frag:0:0] INFO
o.a.d.e.w.fragment.FragmentExecutor - 29c714cf-fdc7-2ac9-5fd4-c4fdae636550:0:0:
State change requested FAILED --> FINISHED
2015-11-03 17:13:19,271 [29c714cf-fdc7-2ac9-5fd4-c4fdae636550:frag:0:0] ERROR
o.a.d.e.w.fragment.FragmentExecutor - SYSTEM ERROR:
UnsupportedOperationException: Unable to get value vector class for minor type
[LATE] and mode [OPTIONAL]
Fragment 0:0
[Error Id: 95cd27b5-065d-4714-a3e0-cd8d8ff6d519 on atsqa4-133.qa.lab:31010]
org.apache.drill.common.exceptions.UserException: SYSTEM ERROR:
UnsupportedOperationException: Unable to get value vector class for minor type
[LATE] and mode [OPTIONAL]
Fragment 0:0
[Error Id: 95cd27b5-065d-4714-a3e0-cd8d8ff6d519 on atsqa4-133.qa.lab:31010]
at
org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:534)
~[drill-common-1.2.0.jar:1.2.0]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:323)
[drill-java-exec-1.2.0.jar:1.2.0]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:178)
[drill-java-exec-1.2.0.jar:1.2.0]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:292)
[drill-java-exec-1.2.0.jar:1.2.0]
at
org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
[drill-common-1.2.0.jar:1.2.0]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[na:1.7.0_71]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_71]
at java.lang.Thread.r
[jira] [Updated] (DRILL-4012) Limit 0 on top of query with kvg/flatten results in assert
[
https://issues.apache.org/jira/browse/DRILL-4012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Victoria Markman updated DRILL-4012:
Description:
I've found couple of bugs that are very similar, but none of them are quite the
same:
missing-map.json
{code}
{
"id": 1,
"m": {"a":1,"b":2}
}
{
"id": 2
}
{
"id": 3,
"m": {"c":3,"d":4}
}
{code}
'limit 0' results in an assert:
{code}
0: jdbc:drill:schema=dfs> select * from (select id, flatten(kvgen(m)) from
`missing-map.json`) limit 0;
Error: SYSTEM ERROR: ClassCastException: Cannot cast
org.apache.drill.exec.vector.NullableIntVector to
org.apache.drill.exec.vector.complex.RepeatedValueVector
Fragment 0:0
[Error Id: 046bb4d4-2c54-43ab-9577-cf21542ff8ef on atsqa4-133.qa.lab:31010]
(state=,code=0)
{code}
'limit 1' works:
{code}
0: jdbc:drill:schema=dfs> select * from (select id, flatten(kvgen(m)) from
`missing-map.json`) limit 1;
+-++
| id | EXPR$1 |
+-++
| 1 | {"key":"a","value":1} |
+-++
1 row selected (0.247 seconds)
{code}
No limit, just in subquery: works
{code}
0: jdbc:drill:schema=dfs> select * from (select id, flatten(kvgen(m)) from
`json_kvgenflatten/missing-map.json`);
+-++
| id | EXPR$1 |
+-++
| 1 | {"key":"a","value":1} |
| 1 | {"key":"b","value":2} |
| 3 | {"key":"c","value":3} |
| 3 | {"key":"d","value":4} |
+-++
4 rows selected (0.247 seconds)
{code}
drillbit.log
{code}
2015-11-03 15:23:20,943 [29c72e96-c9f6-9fce-ecf1-14eaa145f72b:foreman] INFO
o.a.d.e.s.schedule.BlockMapBuilder - Failure finding Drillbit running on host
atsqa4-136.qa.lab. Skipping affinity to that host.
2015-11-03 15:23:20,944 [29c72e96-c9f6-9fce-ecf1-14eaa145f72b:foreman] INFO
o.a.d.e.s.schedule.BlockMapBuilder - Failure finding Drillbit running on host
atsqa4-134.qa.lab. Skipping affinity to that host.
2015-11-03 15:23:20,944 [29c72e96-c9f6-9fce-ecf1-14eaa145f72b:foreman] INFO
o.a.d.e.s.schedule.BlockMapBuilder - Get block maps: Executed 1 out of 1 using
1 threads. Time: 1ms total, 1.530719ms avg, 1ms max.
2015-11-03 15:23:20,944 [29c72e96-c9f6-9fce-ecf1-14eaa145f72b:foreman] INFO
o.a.d.e.s.schedule.BlockMapBuilder - Get block maps: Executed 1 out of 1 using
1 threads. Earliest start: 1.744000 μs, Latest start: 1.744000 μs, Average
start: 1.744000 μs .
2015-11-03 15:23:20,968 [29c72e96-c9f6-9fce-ecf1-14eaa145f72b:frag:0:0] INFO
o.a.d.e.w.fragment.FragmentExecutor - 29c72e96-c9f6-9fce-ecf1-14eaa145f72b:0:0:
State change requested AWAITING_ALLOCATION --> RUNNING
2015-11-03 15:23:20,968 [29c72e96-c9f6-9fce-ecf1-14eaa145f72b:frag:0:0] INFO
o.a.d.e.w.f.FragmentStatusReporter - 29c72e96-c9f6-9fce-ecf1-14eaa145f72b:0:0:
State to report: RUNNING
2015-11-03 15:23:20,974 [29c72e96-c9f6-9fce-ecf1-14eaa145f72b:frag:0:0] WARN
o.a.d.e.e.ExpressionTreeMaterializer - Unable to find value vector of path
`EXPR$3`, returning null instance.
2015-11-03 15:23:20,975 [29c72e96-c9f6-9fce-ecf1-14eaa145f72b:frag:0:0] WARN
o.a.d.e.e.ExpressionTreeMaterializer - Unable to find value vector of path
`EXPR$3`, returning null instance.
2015-11-03 15:23:20,976 [29c72e96-c9f6-9fce-ecf1-14eaa145f72b:frag:0:0] INFO
o.a.d.e.w.fragment.FragmentExecutor - 29c72e96-c9f6-9fce-ecf1-14eaa145f72b:0:0:
State change requested RUNNING --> FAILED
2015-11-03 15:23:20,976 [29c72e96-c9f6-9fce-ecf1-14eaa145f72b:frag:0:0] INFO
o.a.d.e.w.fragment.FragmentExecutor - 29c72e96-c9f6-9fce-ecf1-14eaa145f72b:0:0:
State change requested FAILED --> FINISHED
2015-11-03 15:23:20,978 [29c72e96-c9f6-9fce-ecf1-14eaa145f72b:frag:0:0] ERROR
o.a.d.e.w.fragment.FragmentExecutor - SYSTEM ERROR: ClassCastException: Cannot
cast org.apache.drill.exec.vector.NullableIntVector to
org.apache.drill.exec.vector.complex.RepeatedValueVector
Fragment 0:0
[Error Id: c82c7f59-4dad-47ad-8901-5a2261c81279 on atsqa4-133.qa.lab:31010]
org.apache.drill.common.exceptions.UserException: SYSTEM ERROR:
ClassCastException: Cannot cast org.apache.drill.exec.vector.NullableIntVector
to org.apache.drill.exec.vector.complex.RepeatedValueVector
Fragment 0:0
[Error Id: c82c7f59-4dad-47ad-8901-5a2261c81279 on atsqa4-133.qa.lab:31010]
at
org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:534)
~[drill-common-1.2.0.jar:1.2.0]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:323)
[drill-java-exec-1.2.0.jar:1.2.0]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:178)
[drill-java-exec-1.2.0.jar:1.2.0]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:292)
[drill-java-exec-1.2.0.jar:1.2.0]
at
org.apache.drill.common.SelfCleaningRunnable.run(Sel
[jira] [Created] (DRILL-4012) Limit 0 on top of query with kvg/flatten results in assert
Victoria Markman created DRILL-4012:
---
Summary: Limit 0 on top of query with kvg/flatten results in assert
Key: DRILL-4012
URL: https://issues.apache.org/jira/browse/DRILL-4012
Project: Apache Drill
Issue Type: Bug
Components: Execution - Data Types
Reporter: Victoria Markman
missing-map.json
{code}
{
"id": 1,
"m": {"a":1,"b":2}
}
{
"id": 2
}
{
"id": 3,
"m": {"c":3,"d":4}
}
{code}
'limit 0' results in an assert:
{code}
0: jdbc:drill:schema=dfs> select * from (select id, flatten(kvgen(m)) from
`missing-map.json`) limit 0;
Error: SYSTEM ERROR: ClassCastException: Cannot cast
org.apache.drill.exec.vector.NullableIntVector to
org.apache.drill.exec.vector.complex.RepeatedValueVector
Fragment 0:0
[Error Id: 046bb4d4-2c54-43ab-9577-cf21542ff8ef on atsqa4-133.qa.lab:31010]
(state=,code=0)
{code}
'limit 1' works:
{code}
0: jdbc:drill:schema=dfs> select * from (select id, flatten(kvgen(m)) from
`missing-map.json`) limit 1;
+-++
| id | EXPR$1 |
+-++
| 1 | {"key":"a","value":1} |
+-++
1 row selected (0.247 seconds)
{code}
No limit, just in subquery: works
{code}
0: jdbc:drill:schema=dfs> select * from (select id, flatten(kvgen(m)) from
`json_kvgenflatten/missing-map.json`);
+-++
| id | EXPR$1 |
+-++
| 1 | {"key":"a","value":1} |
| 1 | {"key":"b","value":2} |
| 3 | {"key":"c","value":3} |
| 3 | {"key":"d","value":4} |
+-++
4 rows selected (0.247 seconds)
{code}
drillbit.log
{code}
2015-11-03 15:23:20,943 [29c72e96-c9f6-9fce-ecf1-14eaa145f72b:foreman] INFO
o.a.d.e.s.schedule.BlockMapBuilder - Failure finding Drillbit running on host
atsqa4-136.qa.lab. Skipping affinity to that host.
2015-11-03 15:23:20,944 [29c72e96-c9f6-9fce-ecf1-14eaa145f72b:foreman] INFO
o.a.d.e.s.schedule.BlockMapBuilder - Failure finding Drillbit running on host
atsqa4-134.qa.lab. Skipping affinity to that host.
2015-11-03 15:23:20,944 [29c72e96-c9f6-9fce-ecf1-14eaa145f72b:foreman] INFO
o.a.d.e.s.schedule.BlockMapBuilder - Get block maps: Executed 1 out of 1 using
1 threads. Time: 1ms total, 1.530719ms avg, 1ms max.
2015-11-03 15:23:20,944 [29c72e96-c9f6-9fce-ecf1-14eaa145f72b:foreman] INFO
o.a.d.e.s.schedule.BlockMapBuilder - Get block maps: Executed 1 out of 1 using
1 threads. Earliest start: 1.744000 μs, Latest start: 1.744000 μs, Average
start: 1.744000 μs .
2015-11-03 15:23:20,968 [29c72e96-c9f6-9fce-ecf1-14eaa145f72b:frag:0:0] INFO
o.a.d.e.w.fragment.FragmentExecutor - 29c72e96-c9f6-9fce-ecf1-14eaa145f72b:0:0:
State change requested AWAITING_ALLOCATION --> RUNNING
2015-11-03 15:23:20,968 [29c72e96-c9f6-9fce-ecf1-14eaa145f72b:frag:0:0] INFO
o.a.d.e.w.f.FragmentStatusReporter - 29c72e96-c9f6-9fce-ecf1-14eaa145f72b:0:0:
State to report: RUNNING
2015-11-03 15:23:20,974 [29c72e96-c9f6-9fce-ecf1-14eaa145f72b:frag:0:0] WARN
o.a.d.e.e.ExpressionTreeMaterializer - Unable to find value vector of path
`EXPR$3`, returning null instance.
2015-11-03 15:23:20,975 [29c72e96-c9f6-9fce-ecf1-14eaa145f72b:frag:0:0] WARN
o.a.d.e.e.ExpressionTreeMaterializer - Unable to find value vector of path
`EXPR$3`, returning null instance.
2015-11-03 15:23:20,976 [29c72e96-c9f6-9fce-ecf1-14eaa145f72b:frag:0:0] INFO
o.a.d.e.w.fragment.FragmentExecutor - 29c72e96-c9f6-9fce-ecf1-14eaa145f72b:0:0:
State change requested RUNNING --> FAILED
2015-11-03 15:23:20,976 [29c72e96-c9f6-9fce-ecf1-14eaa145f72b:frag:0:0] INFO
o.a.d.e.w.fragment.FragmentExecutor - 29c72e96-c9f6-9fce-ecf1-14eaa145f72b:0:0:
State change requested FAILED --> FINISHED
2015-11-03 15:23:20,978 [29c72e96-c9f6-9fce-ecf1-14eaa145f72b:frag:0:0] ERROR
o.a.d.e.w.fragment.FragmentExecutor - SYSTEM ERROR: ClassCastException: Cannot
cast org.apache.drill.exec.vector.NullableIntVector to
org.apache.drill.exec.vector.complex.RepeatedValueVector
Fragment 0:0
[Error Id: c82c7f59-4dad-47ad-8901-5a2261c81279 on atsqa4-133.qa.lab:31010]
org.apache.drill.common.exceptions.UserException: SYSTEM ERROR:
ClassCastException: Cannot cast org.apache.drill.exec.vector.NullableIntVector
to org.apache.drill.exec.vector.complex.RepeatedValueVector
Fragment 0:0
[Error Id: c82c7f59-4dad-47ad-8901-5a2261c81279 on atsqa4-133.qa.lab:31010]
at
org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:534)
~[drill-common-1.2.0.jar:1.2.0]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:323)
[drill-java-exec-1.2.0.jar:1.2.0]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:178)
[drill-java-exec-1.2.0.jar:1.2.0]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:29
[jira] [Updated] (DRILL-3871) Off by one error while reading binary fields with one terminal null in parquet
[
https://issues.apache.org/jira/browse/DRILL-3871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Victoria Markman updated DRILL-3871:
Summary: Off by one error while reading binary fields with one terminal
null in parquet (was: Exception on inner join when join predicate is int96
field generated by impala)
> Off by one error while reading binary fields with one terminal null in parquet
> --
>
> Key: DRILL-3871
> URL: https://issues.apache.org/jira/browse/DRILL-3871
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Data Types
>Affects Versions: 1.2.0
>Reporter: Victoria Markman
>Assignee: Deneche A. Hakim
>Priority: Critical
> Labels: int96
> Fix For: 1.3.0
>
> Attachments: tables.tar
>
>
> Both tables in the join where created by impala, with column c_timestamp
> being parquet int96.
> {code}
> 0: jdbc:drill:schema=dfs> select
> . . . . . . . . . . . . > max(t1.c_timestamp),
> . . . . . . . . . . . . > min(t1.c_timestamp),
> . . . . . . . . . . . . > count(t1.c_timestamp)
> . . . . . . . . . . . . > from
> . . . . . . . . . . . . > imp_t1 t1
> . . . . . . . . . . . . > inner join
> . . . . . . . . . . . . > imp_t2 t2
> . . . . . . . . . . . . > on (t1.c_timestamp = t2.c_timestamp)
> . . . . . . . . . . . . > ;
> java.lang.RuntimeException: java.sql.SQLException: SYSTEM ERROR:
> TProtocolException: Required field 'uncompressed_page_size' was not found in
> serialized data! Struct: PageHeader(type:null, uncompressed_page_size:0,
> compressed_page_size:0)
> Fragment 0:0
> [Error Id: eb6a5df8-fc59-409b-957a-59cb1079b5b8 on atsqa4-133.qa.lab:31010]
> at sqlline.IncrementalRows.hasNext(IncrementalRows.java:73)
> at
> sqlline.TableOutputFormat$ResizingRowsProvider.next(TableOutputFormat.java:87)
> at sqlline.TableOutputFormat.print(TableOutputFormat.java:118)
> at sqlline.SqlLine.print(SqlLine.java:1583)
> at sqlline.Commands.execute(Commands.java:852)
> at sqlline.Commands.sql(Commands.java:751)
> at sqlline.SqlLine.dispatch(SqlLine.java:738)
> at sqlline.SqlLine.begin(SqlLine.java:612)
> at sqlline.SqlLine.start(SqlLine.java:366)
> at sqlline.SqlLine.main(SqlLine.java:259)
> {code}
> drillbit.log
> {code}
> 2015-09-30 21:15:45,710 [29f3aefe-3209-a6e6-0418-500dac60a339:foreman] INFO
> o.a.d.exec.store.parquet.Metadata - Took 0 ms to get file statuses
> 2015-09-30 21:15:45,712 [29f3aefe-3209-a6e6-0418-500dac60a339:foreman] INFO
> o.a.d.exec.store.parquet.Metadata - Fetch parquet metadata: Executed 1 out of
> 1 using 1 threads. Time: 1ms total, 1.645381ms avg, 1ms max.
> 2015-09-30 21:15:45,712 [29f3aefe-3209-a6e6-0418-500dac60a339:foreman] INFO
> o.a.d.exec.store.parquet.Metadata - Fetch parquet metadata: Executed 1 out of
> 1 using 1 threads. Earliest start: 1.332000 μs, Latest start: 1.332000 μs,
> Average start: 1.332000 μs .
> 2015-09-30 21:15:45,830 [29f3aefe-3209-a6e6-0418-500dac60a339:frag:0:0] INFO
> o.a.d.e.w.fragment.FragmentExecutor -
> 29f3aefe-3209-a6e6-0418-500dac60a339:0:0: State change requested
> AWAITING_ALLOCATION --> RUNNING
> 2015-09-30 21:15:45,830 [29f3aefe-3209-a6e6-0418-500dac60a339:frag:0:0] INFO
> o.a.d.e.w.f.FragmentStatusReporter -
> 29f3aefe-3209-a6e6-0418-500dac60a339:0:0: State to report: RUNNING
> 2015-09-30 21:15:45,925 [29f3aefe-3209-a6e6-0418-500dac60a339:frag:0:0] INFO
> o.a.d.e.w.fragment.FragmentExecutor -
> 29f3aefe-3209-a6e6-0418-500dac60a339:0:0: State change requested RUNNING -->
> FAILED
> 2015-09-30 21:15:45,930 [29f3aefe-3209-a6e6-0418-500dac60a339:frag:0:0] INFO
> o.a.d.e.w.fragment.FragmentExecutor -
> 29f3aefe-3209-a6e6-0418-500dac60a339:0:0: State change requested FAILED -->
> FINISHED
> 2015-09-30 21:15:45,931 [29f3aefe-3209-a6e6-0418-500dac60a339:frag:0:0] ERROR
> o.a.d.e.w.fragment.FragmentExecutor - SYSTEM ERROR: TProtocolException:
> Required field 'uncompressed_page_size' was not found in serialized data!
> Struct: PageHeader(type:null, uncompressed_page_size:0,
> compressed_page_size:0)
> Fragment 0:0
> [Error Id: eb6a5df8-fc59-409b-957a-59cb1079b5b8 on atsqa4-133.qa.lab:31010]
> org.apache.drill.common.exceptions.UserException: SYSTEM ERROR:
> TProtocolException: Required field 'uncompressed_page_size' was not found in
> serialized data! Struct: PageHeader(type:null, uncompressed_page_size:0,
> compressed_page_size:0)
> Fragment 0:0
> [Error Id: eb6a5df8-fc59-409b-957a-59cb1079b5b8 on atsqa4-133.qa.lab:31010]
> at
> org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:534)
> ~[drill-common-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
> at
> org.
[jira] [Commented] (DRILL-4009) Assert in a query with concat function when wrapped in another subquery
[
https://issues.apache.org/jira/browse/DRILL-4009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14986234#comment-14986234
]
Victoria Markman commented on DRILL-4009:
-
And this could be even more generic, I can reproduce the same error with substr
function:
{code}
0: jdbc:drill:schema=dfs> select * from ( select substr(b1, 1, 3) from t1 group
by substr(b1, 1,3)) as foo;
Error: SYSTEM ERROR: AssertionError: Internal error: Type 'RecordType(ANY $f0)'
has no field 'EXPR$0'
[Error Id: 3fd2e14e-5fa0-4971-b7aa-8eebf820b8e5 on atsqa4-133.qa.lab:31010]
(state=,code=0)
{code}
> Assert in a query with concat function when wrapped in another subquery
> ---
>
> Key: DRILL-4009
> URL: https://issues.apache.org/jira/browse/DRILL-4009
> Project: Apache Drill
> Issue Type: Bug
> Components: Query Planning & Optimization
>Affects Versions: 1.2.0
>Reporter: Victoria Markman
>
> {code}
> 0: jdbc:drill:schema=dfs> select * from ( select concat(b1, '***') from t1
> group by concat(b1, '***') ) limit 10;
> Error: SYSTEM ERROR: AssertionError: Internal error: Type 'RecordType(ANY
> $f0)' has no field 'EXPR$0'
> [Error Id: 3366119e-baec-4aa1-b4fe-a58947bba088 on atsqa4-133.qa.lab:31010]
> (state=,code=0)
> {code}
> Works:
> {code}
> 0: jdbc:drill:schema=dfs> select concat(b1, '***') from t1 group by
> concat(b1, '***') limit 10;
> +---+
> | EXPR$0 |
> +---+
> | *** |
> | a*** |
> | b*** |
> | c*** |
> | e*** |
> | f*** |
> | g*** |
> | h*** |
> | i*** |
> | j*** |
> +---+
> 10 rows selected (0.672 seconds)
> {code}
> Drill version:
> {code}
> #Sat Oct 31 01:10:06 UTC 2015
> git.commit.id.abbrev=e4b94a7
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (DRILL-4009) Assert in a query with concat function when wrapped in another subquery
[
https://issues.apache.org/jira/browse/DRILL-4009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Victoria Markman updated DRILL-4009:
Summary: Assert in a query with concat function when wrapped in another
subquery (was: Assert in a query with concat function in a group by and limit
in outer query)
> Assert in a query with concat function when wrapped in another subquery
> ---
>
> Key: DRILL-4009
> URL: https://issues.apache.org/jira/browse/DRILL-4009
> Project: Apache Drill
> Issue Type: Bug
> Components: Query Planning & Optimization
>Affects Versions: 1.2.0
>Reporter: Victoria Markman
>
> {code}
> 0: jdbc:drill:schema=dfs> select * from ( select concat(b1, '***') from t1
> group by concat(b1, '***') ) limit 10;
> Error: SYSTEM ERROR: AssertionError: Internal error: Type 'RecordType(ANY
> $f0)' has no field 'EXPR$0'
> [Error Id: 3366119e-baec-4aa1-b4fe-a58947bba088 on atsqa4-133.qa.lab:31010]
> (state=,code=0)
> {code}
> Works:
> {code}
> 0: jdbc:drill:schema=dfs> select concat(b1, '***') from t1 group by
> concat(b1, '***') limit 10;
> +---+
> | EXPR$0 |
> +---+
> | *** |
> | a*** |
> | b*** |
> | c*** |
> | e*** |
> | f*** |
> | g*** |
> | h*** |
> | i*** |
> | j*** |
> +---+
> 10 rows selected (0.672 seconds)
> {code}
> Drill version:
> {code}
> #Sat Oct 31 01:10:06 UTC 2015
> git.commit.id.abbrev=e4b94a7
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-4009) Assert in a query with concat function in a group by and limit in outer query
[
https://issues.apache.org/jira/browse/DRILL-4009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14986221#comment-14986221
]
Victoria Markman commented on DRILL-4009:
-
No limit is necessary, just wrap it in subquery:
{code}
0: jdbc:drill:schema=dfs> select * from ( select concat(b1, '***') from t1
group by concat(b1, '***') ) as foo ;
Error: SYSTEM ERROR: AssertionError: Internal error: Type 'RecordType(ANY $f0)'
has no field 'EXPR$0'
[Error Id: a276465d-87fa-4b94-899d-4003b6917bc7 on atsqa4-133.qa.lab:31010]
(state=,code=0)
{code}
> Assert in a query with concat function in a group by and limit in outer query
> -
>
> Key: DRILL-4009
> URL: https://issues.apache.org/jira/browse/DRILL-4009
> Project: Apache Drill
> Issue Type: Bug
> Components: Query Planning & Optimization
>Affects Versions: 1.2.0
>Reporter: Victoria Markman
>
> {code}
> 0: jdbc:drill:schema=dfs> select * from ( select concat(b1, '***') from t1
> group by concat(b1, '***') ) limit 10;
> Error: SYSTEM ERROR: AssertionError: Internal error: Type 'RecordType(ANY
> $f0)' has no field 'EXPR$0'
> [Error Id: 3366119e-baec-4aa1-b4fe-a58947bba088 on atsqa4-133.qa.lab:31010]
> (state=,code=0)
> {code}
> Works:
> {code}
> 0: jdbc:drill:schema=dfs> select concat(b1, '***') from t1 group by
> concat(b1, '***') limit 10;
> +---+
> | EXPR$0 |
> +---+
> | *** |
> | a*** |
> | b*** |
> | c*** |
> | e*** |
> | f*** |
> | g*** |
> | h*** |
> | i*** |
> | j*** |
> +---+
> 10 rows selected (0.672 seconds)
> {code}
> Drill version:
> {code}
> #Sat Oct 31 01:10:06 UTC 2015
> git.commit.id.abbrev=e4b94a7
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-4009) Assert in a query with concat function in a group by and limit in outer query
[
https://issues.apache.org/jira/browse/DRILL-4009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14986206#comment-14986206
]
Victoria Markman commented on DRILL-4009:
-
drillbit.log
{code}
2015-11-02 22:21:00,219 [29c81e34-602c-dd9e-54f6-0057bc6efc9e:foreman] ERROR
o.a.drill.exec.work.foreman.Foreman - SYSTEM ERROR: AssertionError: Internal
error: Type 'RecordType(ANY $f0)' has no field 'EXPR$0'
[Error Id: 3366119e-baec-4aa1-b4fe-a58947bba088 on atsqa4-133.qa.lab:31010]
org.apache.drill.common.exceptions.UserException: SYSTEM ERROR: AssertionError:
Internal error: Type 'RecordType(ANY $f0)' has no field 'EXPR$0'
[Error Id: 3366119e-baec-4aa1-b4fe-a58947bba088 on atsqa4-133.qa.lab:31010]
at
org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:534)
~[drill-common-1.3.0-SNAPSHOT.jar:1.3.0-SNAPSHOT]
at
org.apache.drill.exec.work.foreman.Foreman$ForemanResult.close(Foreman.java:742)
[drill-java-exec-1.3.0-SNAPSHOT.jar:1.3.0-SNAPSHOT]
at
org.apache.drill.exec.work.foreman.Foreman$StateSwitch.processEvent(Foreman.java:841)
[drill-java-exec-1.3.0-SNAPSHOT.jar:1.3.0-SNAPSHOT]
at
org.apache.drill.exec.work.foreman.Foreman$StateSwitch.processEvent(Foreman.java:786)
[drill-java-exec-1.3.0-SNAPSHOT.jar:1.3.0-SNAPSHOT]
at
org.apache.drill.common.EventProcessor.sendEvent(EventProcessor.java:73)
[drill-common-1.3.0-SNAPSHOT.jar:1.3.0-SNAPSHOT]
at
org.apache.drill.exec.work.foreman.Foreman$StateSwitch.moveToState(Foreman.java:788)
[drill-java-exec-1.3.0-SNAPSHOT.jar:1.3.0-SNAPSHOT]
at
org.apache.drill.exec.work.foreman.Foreman.moveToState(Foreman.java:894)
[drill-java-exec-1.3.0-SNAPSHOT.jar:1.3.0-SNAPSHOT]
at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:255)
[drill-java-exec-1.3.0-SNAPSHOT.jar:1.3.0-SNAPSHOT]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[na:1.7.0_71]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_71]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_71]
Caused by: org.apache.drill.exec.work.foreman.ForemanException: Unexpected
exception during fragment initialization: Internal error: Type 'RecordType(ANY
$f0)' has no field 'EXPR$0'
... 4 common frames omitted
Caused by: java.lang.AssertionError: Internal error: Type 'RecordType(ANY $f0)'
has no field 'EXPR$0'
at org.apache.calcite.util.Util.newInternal(Util.java:777)
~[calcite-core-1.4.0-drill-r6.jar:1.4.0-drill-r6]
at
org.apache.calcite.rex.RexBuilder.makeFieldAccess(RexBuilder.java:167)
~[calcite-core-1.4.0-drill-r6.jar:1.4.0-drill-r6]
at
org.apache.calcite.sql2rel.SqlToRelConverter.convertIdentifier(SqlToRelConverter.java:3217)
~[calcite-core-1.4.0-drill-r6.jar:1.4.0-drill-r6]
at
org.apache.calcite.sql2rel.SqlToRelConverter.access$1400(SqlToRelConverter.java:184)
~[calcite-core-1.4.0-drill-r6.jar:1.4.0-drill-r6]
at
org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:4172)
~[calcite-core-1.4.0-drill-r6.jar:1.4.0-drill-r6]
at
org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.visit(SqlToRelConverter.java:3595)
~[calcite-core-1.4.0-drill-r6.jar:1.4.0-drill-r6]
at org.apache.calcite.sql.SqlIdentifier.accept(SqlIdentifier.java:274)
~[calcite-core-1.4.0-drill-r6.jar:1.4.0-drill-r6]
at
org.apache.calcite.sql2rel.SqlToRelConverter$Blackboard.convertExpression(SqlToRelConverter.java:4054)
~[calcite-core-1.4.0-drill-r6.jar:1.4.0-drill-r6]
at
org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectList(SqlToRelConverter.java:3425)
~[calcite-core-1.4.0-drill-r6.jar:1.4.0-drill-r6]
at
org.apache.calcite.sql2rel.SqlToRelConverter.convertSelectImpl(SqlToRelConverter.java:626)
~[calcite-core-1.4.0-drill-r6.jar:1.4.0-drill-r6]
at
org.apache.calcite.sql2rel.SqlToRelConverter.convertSelect(SqlToRelConverter.java:582)
~[calcite-core-1.4.0-drill-r6.jar:1.4.0-drill-r6]
at
org.apache.calcite.sql2rel.SqlToRelConverter.convertQueryRecursive(SqlToRelConverter.java:2787)
~[calcite-core-1.4.0-drill-r6.jar:1.4.0-drill-r6]
at
org.apache.calcite.sql2rel.SqlToRelConverter.convertQuery(SqlToRelConverter.java:536)
~[calcite-core-1.4.0-drill-r6.jar:1.4.0-drill-r6]
at org.apache.calcite.prepare.PlannerImpl.convert(PlannerImpl.java:204)
~[calcite-core-1.4.0-drill-r6.jar:1.4.0-drill-r6]
at
org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToRel(DefaultSqlHandler.java:467)
~[drill-java-exec-1.3.0-SNAPSHOT.jar:1.3.0-SNAPSHOT]
at
org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.validateAndConvert(DefaultSqlHandler.java:193)
~[drill-java-exec-1.3.0-SNAPSHOT.jar:1.3.0-SNAPSHOT]
at
org.apache.drill.exec.planner.sql.handlers.D
[jira] [Created] (DRILL-4009) Assert in a query with concat function in a group by and limit in outer query
Victoria Markman created DRILL-4009:
---
Summary: Assert in a query with concat function in a group by and
limit in outer query
Key: DRILL-4009
URL: https://issues.apache.org/jira/browse/DRILL-4009
Project: Apache Drill
Issue Type: Bug
Components: Query Planning & Optimization
Affects Versions: 1.2.0
Reporter: Victoria Markman
{code}
0: jdbc:drill:schema=dfs> select * from ( select concat(b1, '***') from t1
group by concat(b1, '***') ) limit 10;
Error: SYSTEM ERROR: AssertionError: Internal error: Type 'RecordType(ANY $f0)'
has no field 'EXPR$0'
[Error Id: 3366119e-baec-4aa1-b4fe-a58947bba088 on atsqa4-133.qa.lab:31010]
(state=,code=0)
{code}
Works:
{code}
0: jdbc:drill:schema=dfs> select concat(b1, '***') from t1 group by concat(b1,
'***') limit 10;
+---+
| EXPR$0 |
+---+
| *** |
| a*** |
| b*** |
| c*** |
| e*** |
| f*** |
| g*** |
| h*** |
| i*** |
| j*** |
+---+
10 rows selected (0.672 seconds)
{code}
Drill version:
{code}
#Sat Oct 31 01:10:06 UTC 2015
git.commit.id.abbrev=e4b94a7
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Closed] (DRILL-2726) Display Drill version in sys.version
[ https://issues.apache.org/jira/browse/DRILL-2726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Victoria Markman closed DRILL-2726. --- > Display Drill version in sys.version > > > Key: DRILL-2726 > URL: https://issues.apache.org/jira/browse/DRILL-2726 > Project: Apache Drill > Issue Type: Improvement > Components: Storage - Other >Reporter: Andries Engelbrecht > Fix For: 1.3.0 > > > Include the Drill version information in sys.version, so it is easy to > determine the exact version of Drill being used for support purposes. > Adding a version column to sys.version to show the exact version i.e. > mapr-drill-0.8.0.31168-1 > or > apache-drill-0.8.0.31168-1 > Will make it easier for users to quickly identify the Drill version being > used, and provide that information. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-2726) Display Drill version in sys.version
[
https://issues.apache.org/jira/browse/DRILL-2726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14981506#comment-14981506
]
Victoria Markman commented on DRILL-2726:
-
{code}
0: jdbc:drill:schema=dfs> select * from sys.version;
+-+---+---++--++
| version | commit_id |
commit_message |
commit_time | build_email | build_time |
+-+---+---++--++
| 1.3.0-SNAPSHOT | 1d067d26b1ba510f4a51489d62d2a6d0480a473c | DRILL-2967:
Incompatible types error reported in a "not in" query with compatible data
types | 26.10.2015 @ 23:37:35 UTC | Unknown | 29.10.2015 @ 04:15:20 UTC
|
+-+---+---++--++
1 row selected (2.824 seconds)
{code}
> Display Drill version in sys.version
>
>
> Key: DRILL-2726
> URL: https://issues.apache.org/jira/browse/DRILL-2726
> Project: Apache Drill
> Issue Type: Improvement
> Components: Storage - Other
>Reporter: Andries Engelbrecht
> Fix For: 1.3.0
>
>
> Include the Drill version information in sys.version, so it is easy to
> determine the exact version of Drill being used for support purposes.
> Adding a version column to sys.version to show the exact version i.e.
> mapr-drill-0.8.0.31168-1
> or
> apache-drill-0.8.0.31168-1
> Will make it easier for users to quickly identify the Drill version being
> used, and provide that information.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (DRILL-3996) Project/filter pushdown logic is causing excessive planning time as number of joins in the query increases
Victoria Markman created DRILL-3996:
---
Summary: Project/filter pushdown logic is causing excessive
planning time as number of joins in the query increases
Key: DRILL-3996
URL: https://issues.apache.org/jira/browse/DRILL-3996
Project: Apache Drill
Issue Type: Bug
Components: Query Planning & Optimization
Affects Versions: 1.2.0
Reporter: Victoria Markman
{code}
CREATE OR REPLACE VIEW v1 as
SELECT
wh.w_warehouse_name as warehouse_name,
wh.w_warehouse_sq_ftas warehouse_sq_ft,
i.i_item_desc as item_desc,
i.i_current_price as current_price,
p.p_promo_name as promo_name,
p.p_channel_catalog as channel_catalog,
sm.sm_carrier as carrier,
sm.sm_contract as contract,
c.c_first_name as first_name,
c.c_last_name as last_name,
c.c_email_address as email_address,
ca.ca_city as city,
ca.ca_countyas county,
ca.ca_state as state,
ca.ca_zip as zip,
td.t_shift as shift,
ws.ws_sold_date_sk as sold_date,
ws.ws_sold_time_sk as sold_time,
ws.ws_ship_date_sk as ship_date,
ws.ws_bill_cdemo_sk as bill_cdemo,
ws.ws_bill_hdemo_sk as bill_hdemo,
ws.ws_bill_addr_sk as bill_addr,
ws.ws_ship_customer_sk as ship_customer,
ws.ws_ship_cdemo_sk as ship_cdemo,
ws.ws_ship_hdemo_sk as ship_hdemo,
ws.ws_ship_addr_sk as ship_addr,
ws.ws_web_page_sk as web_page,
ws.ws_order_number as order_number,
ws.ws_quantity as quanity,
ws.ws_wholesale_costas wholesale_cost,
ws.ws_list_priceas list_price,
ws.ws_sales_price as sales_price,
ws.ws_ext_discount_amt as ext_discount_amt,
ws.ws_ext_sales_price as ext_sales_price,
ws.ws_ext_wholesale_costas ext_wholesale_cost,
ws.ws_ext_list_priceas ext_list_price,
ws.ws_ext_tax as ext_tax ,
ws.ws_coupon_amtas coupon_amt,
ws.ws_ext_ship_cost as ext_ship_cost,
ws.ws_net_paid as net_paid,
ws.ws_net_paid_inc_tax as net_paid_inc_tax,
ws.ws_net_paid_inc_ship as net_paid_inc_ship,
ws.ws_net_paid_inc_ship_tax as net_paid_inc_ship_tax ,
ws.ws_net_profitas net_profit
FROM
web_sales ws
JOIN warehouse wh
ON ws.ws_warehouse_sk = wh.w_warehouse_sk
JOIN item i
ON ws.ws_item_sk = i.i_item_sk
JOIN promotion p
ON ws.ws_promo_sk = p.p_promo_sk
JOIN ship_mode sm
ON ws.ws_ship_mode_sk = sm.sm_ship_mode_sk
JOIN customer c
ON ws.ws_bill_customer_sk = c.c_customer_sk
JOIN customer_address ca
ON ws.ws_bill_addr_sk = ca.ca_address_sk
LEFT JOIN time_dim td
ON ws.ws_sold_time_sk = td.t_time_sk
WHERE
ws.ws_warehouse_sk = 5
;
{code}
Query below takes between 25 and 40 seconds to plan:
{code}
explain plan for select state from v1 where NOT (v1.contract IS NULL) AND
(v1.sold_date NOT IN()) AND (3=3) limit 0;
{code}
In our internal experiments we found out that if we disable projection pushdown
for this particular query, planning time reduces to below 1 second.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-3958) Improve error message when JDBC driver not found
[ https://issues.apache.org/jira/browse/DRILL-3958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14971443#comment-14971443 ] Victoria Markman commented on DRILL-3958: - Related if not the same: https://issues.apache.org/jira/browse/DRILL-2663 > Improve error message when JDBC driver not found > > > Key: DRILL-3958 > URL: https://issues.apache.org/jira/browse/DRILL-3958 > Project: Apache Drill > Issue Type: Improvement > Components: Client - HTTP >Affects Versions: 1.2.0 >Reporter: Uwe Geercken >Priority: Critical > > When setting up a storage definition for JDBC in the Drill web UI, the > appropriate driver has to be available in the 3rdparty folder before defining > the storage, otherwise an error is displayed. > The error message refers to a JSON mapping error which is completely > inappropriate in this case, because the error is the missing JDBC driver in > the 3rdparty folder and not the JSON mapping. > I request to change the error message to something appropriate that the > class/driver referred to could not be found (like for example: > com.mysql.jdbc.Driver) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-3958) Improve error message when JDBC driver not found
[ https://issues.apache.org/jira/browse/DRILL-3958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Victoria Markman updated DRILL-3958: Priority: Major (was: Minor) > Improve error message when JDBC driver not found > > > Key: DRILL-3958 > URL: https://issues.apache.org/jira/browse/DRILL-3958 > Project: Apache Drill > Issue Type: Improvement > Components: Client - HTTP >Affects Versions: 1.2.0 >Reporter: Uwe Geercken > > When setting up a storage definition for JDBC in the Drill web UI, the > appropriate driver has to be available in the 3rdparty folder before defining > the storage, otherwise an error is displayed. > The error message refers to a JSON mapping error which is completely > inappropriate in this case, because the error is the missing JDBC driver in > the 3rdparty folder and not the JSON mapping. > I request to change the error message to something appropriate that the > class/driver referred to could not be found (like for example: > com.mysql.jdbc.Driver) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-3543) Add stats for external sort to a query profile
[
https://issues.apache.org/jira/browse/DRILL-3543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Victoria Markman updated DRILL-3543:
Assignee: Sudheesh Katkam
> Add stats for external sort to a query profile
> --
>
> Key: DRILL-3543
> URL: https://issues.apache.org/jira/browse/DRILL-3543
> Project: Apache Drill
> Issue Type: Improvement
> Components: Execution - Relational Operators
>Affects Versions: 1.1.0
>Reporter: Victoria Markman
>Assignee: Sudheesh Katkam
> Fix For: Future
>
>
> The only indication if sort spilled to disk today is info from the
> drillbit.log.
> It would be great if this information was displayed in the query profile.
> {code}
> 015-07-22 23:47:29,907 [2a4fd46e-f8c3-6b96-b165-b665a41be311:frag:0:0] INFO
> o.a.d.e.p.i.xsort.ExternalSortBatch - Completed spilling to
> /tmp/drill/spill/2a4fd46e-f8c3-6b96-b165-b665a41be311/major_fragment_0/minor_fragment_0/operator_7/92
> 2015-07-22 23:47:29,919 [2a4fd46e-f8c3-6b96-b165-b665a41be311:frag:0:0] INFO
> o.a.d.e.p.i.xsort.ExternalSortBatch - Merging and spilling to
> /tmp/drill/spill/2a4fd46e-f8c3-6b96-b165-b665a41be311/major_fragment_0/minor_fragment_0/operator_7/93
> 2015-07-22 23:47:29,919 [2a4fd46e-f8c3-6b96-b165-b665a41be311:frag:0:0] INFO
> o.a.d.e.p.i.xsort.ExternalSortBatch - Completed spilling to
> /tmp/drill/spill/2a4fd46e-f8c3-6b96-b165-b665a41be311/major_fragment_0/minor_fragment_0/operator_7/93
> 2015-07-22 23:47:29,919 [2a4fd46e-f8c3-6b96-b165-b665a41be311:frag:0:0] WARN
> o.a.d.e.p.i.xsort.ExternalSortBatch - Starting to merge. 7 batch groups.
> Current allocated memory: 11566787
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (DRILL-3961) CONVERT_FROM should return INTEGER when converting from TINYINT/SMALLINT types
[
https://issues.apache.org/jira/browse/DRILL-3961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Victoria Markman updated DRILL-3961:
Priority: Critical (was: Major)
> CONVERT_FROM should return INTEGER when converting from TINYINT/SMALLINT types
> --
>
> Key: DRILL-3961
> URL: https://issues.apache.org/jira/browse/DRILL-3961
> Project: Apache Drill
> Issue Type: Bug
> Components: Functions - Drill
>Affects Versions: 1.1.0, 1.2.0
>Reporter: Victoria Markman
>Priority: Critical
>
> We've disabled SMALLINT, TINYINT SQL types because we had bugs that we could
> not fix before 1.0.0 Some things were left in sort of half implemented state
> and we should either fix up these things or re-implement TINYINT/SMALLINT
> types.
> * We can't cast to TINYINT/SMALLINT:
> {code}
> 0: jdbc:drill:schema=dfs> select cast(1 as tinyint) from sys.options limit 1;
> Error: UNSUPPORTED_OPERATION ERROR: TINYINT is not supported
> See Apache Drill JIRA: DRILL-1959
> [Error Id: c86bb4f5-4ea9-47cc-856e-ce1e96463477 on atsqa4-133.qa.lab:31010]
> (state=,code=0)
> 0: jdbc:drill:schema=dfs> select cast(1 as smallint) from sys.options limit 1;
> Error: UNSUPPORTED_OPERATION ERROR: SMALLINT is not supported
> See Apache Drill JIRA: DRILL-1959
> [Error Id: 14cc008a-532b-4f02-a9ad-6a54cf5dc6bc on atsqa4-133.qa.lab:31010]
> (state=,code=0)
> {code}
> * We hacked our implementation to read TINTINT/SMALLINT in hive as INTEGER:
> https://issues.apache.org/jira/browse/DRILL-3263
>Which works. Now, if you try querying hive table with TINYINT/SMALLINT
> attribute, you get INTEGER in Drill.
> Hive table, notice c9/c10 columns are TINYINT/SMALLINT respectively:
> {code}
> hive> describe alltypes;
> OK
> c1 int
> c2 boolean
> c3 double
> c4 string
> c5 array
> c6 map
> c7 map
> c8 struct
> c9 tinyint
> c10 smallint
> c11 float
> c12 bigint
> c13 array>
> c14 map>
> c15 struct>
> c16 array,n:int>>
> c17 timestamp
> c18 decimal(10,0)
> c19 binary
> c20 date
> Time taken: 0.094 seconds, Fetched: 20 row(s)
> {code}
> Same table accessed from Drill:
> {code}
> 0: jdbc:drill:schema=dfs> use hive;
> +---+---+
> | ok | summary |
> +---+---+
> | true | Default schema changed to [hive] |
> +---+---+
> 1 row selected (0.337 seconds)
> 0: jdbc:drill:schema=dfs> describe alltypes;
> +--++--+
> | COLUMN_NAME | DATA_TYPE | IS_NULLABLE |
> +--++--+
> | c1 | INTEGER| YES |
> | c2 | BOOLEAN| YES |
> | c3 | DOUBLE | YES |
> | c4 | CHARACTER VARYING | YES |
> | c5 | ARRAY | NO |
> | c6 | MAP| NO |
> | c7 | MAP| NO |
> | c8 | ROW| NO |
> | c9 | INTEGER| YES |
> | c10 | INTEGER| YES |
> | c11 | FLOAT | YES |
> | c12 | BIGINT | YES |
> | c13 | ARRAY | NO |
> | c14 | MAP| NO |
> | c15 | ROW| NO |
> | c16 | ARRAY | NO |
> | c17 | TIMESTAMP | YES |
> | c18 | DECIMAL| YES |
> | c19 | BINARY VARYING | YES |
> | c20 | DATE | YES |
> +--++--+
> 20 rows selected (1.379 seconds)
> {code}
> * However, if you try reading parquet file with TINYINT/SMALLINT types
> (generated by impala for instance), CONVERT_FROM function still returns
> SMALLINT/TINYINT types.
> I can successfully read SMALLINT field from impala generated parquet file:
> {code}
> 0: jdbc:drill:schema=dfs> select asset_id from `impala.parquet` limit 1;
> +--+
> | asset_id |
> +--+
> | [B@22a3aec1 |
> +--+
> 1 row selected (0.298 seconds)
> {code}
> Using CONVERT_FROM function:
> {code}
> 0: jdbc:drill:schema=dfs> select CONVERT_FROM(asset_id,
[jira] [Created] (DRILL-3961) CONVERT_FROM should return INTEGER when converting from TINYINT/SMALLINT types
Victoria Markman created DRILL-3961:
---
Summary: CONVERT_FROM should return INTEGER when converting from
TINYINT/SMALLINT types
Key: DRILL-3961
URL: https://issues.apache.org/jira/browse/DRILL-3961
Project: Apache Drill
Issue Type: Bug
Components: Functions - Drill
Affects Versions: 1.1.0, 1.2.0
Reporter: Victoria Markman
We've disabled SMALLINT, TINYINT SQL types because we had bugs that we could
not fix before 1.0.0 Some things were left in sort of half implemented state
and we should either fix up these things or re-implement TINYINT/SMALLINT types.
* We can't cast to TINYINT/SMALLINT:
{code}
0: jdbc:drill:schema=dfs> select cast(1 as tinyint) from sys.options limit 1;
Error: UNSUPPORTED_OPERATION ERROR: TINYINT is not supported
See Apache Drill JIRA: DRILL-1959
[Error Id: c86bb4f5-4ea9-47cc-856e-ce1e96463477 on atsqa4-133.qa.lab:31010]
(state=,code=0)
0: jdbc:drill:schema=dfs> select cast(1 as smallint) from sys.options limit 1;
Error: UNSUPPORTED_OPERATION ERROR: SMALLINT is not supported
See Apache Drill JIRA: DRILL-1959
[Error Id: 14cc008a-532b-4f02-a9ad-6a54cf5dc6bc on atsqa4-133.qa.lab:31010]
(state=,code=0)
{code}
* We hacked our implementation to read TINTINT/SMALLINT in hive as INTEGER:
https://issues.apache.org/jira/browse/DRILL-3263
Which works. Now, if you try querying hive table with TINYINT/SMALLINT
attribute, you get INTEGER in Drill.
Hive table, notice c9/c10 columns are TINYINT/SMALLINT respectively:
{code}
hive> describe alltypes;
OK
c1 int
c2 boolean
c3 double
c4 string
c5 array
c6 map
c7 map
c8 struct
c9 tinyint
c10 smallint
c11 float
c12 bigint
c13 array>
c14 map>
c15 struct>
c16 array,n:int>>
c17 timestamp
c18 decimal(10,0)
c19 binary
c20 date
Time taken: 0.094 seconds, Fetched: 20 row(s)
{code}
Same table accessed from Drill:
{code}
0: jdbc:drill:schema=dfs> use hive;
+---+---+
| ok | summary |
+---+---+
| true | Default schema changed to [hive] |
+---+---+
1 row selected (0.337 seconds)
0: jdbc:drill:schema=dfs> describe alltypes;
+--++--+
| COLUMN_NAME | DATA_TYPE | IS_NULLABLE |
+--++--+
| c1 | INTEGER| YES |
| c2 | BOOLEAN| YES |
| c3 | DOUBLE | YES |
| c4 | CHARACTER VARYING | YES |
| c5 | ARRAY | NO |
| c6 | MAP| NO |
| c7 | MAP| NO |
| c8 | ROW| NO |
| c9 | INTEGER| YES |
| c10 | INTEGER| YES |
| c11 | FLOAT | YES |
| c12 | BIGINT | YES |
| c13 | ARRAY | NO |
| c14 | MAP| NO |
| c15 | ROW| NO |
| c16 | ARRAY | NO |
| c17 | TIMESTAMP | YES |
| c18 | DECIMAL| YES |
| c19 | BINARY VARYING | YES |
| c20 | DATE | YES |
+--++--+
20 rows selected (1.379 seconds)
{code}
* However, if you try reading parquet file with TINYINT/SMALLINT types
(generated by impala for instance), CONVERT_FROM function still returns
SMALLINT/TINYINT types.
I can successfully read SMALLINT field from impala generated parquet file:
{code}
0: jdbc:drill:schema=dfs> select asset_id from `impala.parquet` limit 1;
+--+
| asset_id |
+--+
| [B@22a3aec1 |
+--+
1 row selected (0.298 seconds)
{code}
Using CONVERT_FROM function:
{code}
0: jdbc:drill:schema=dfs> select CONVERT_FROM(asset_id, 'SMALLINT') from
`impala.parquet` limit 1;
+-+
| EXPR$0 |
+-+
| 19535 |
+-+
1 row selected (0.283 seconds)
{code}
Use of this function in an arithmetic expression results in an error:
{code}
0: jdbc:drill:schema=dfs> select CONVERT_FROM(asset_id, 'SMALLINT') + 1 from
`impala.parquet` limit 1;
Error: SYSTEM ERROR: SchemaChangeException: Failure while trying to materialize
incoming schema. Errors:
Error in expression at
[jira] [Commented] (DRILL-2687) TINYINT/SMALLINT's clean not-implemented-yet messages are missing or late for views
[
https://issues.apache.org/jira/browse/DRILL-2687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14965853#comment-14965853
]
Victoria Markman commented on DRILL-2687:
-
This bug has been fixed in DRILL-2057 by disabling ability to cast to TINYINT:
In drill 1.2.0
{code}
0: jdbc:drill:schema=dfs> CREATE OR REPLACE VIEW TempView AS SELECT CAST('1' AS
SMALLINT) FROM INFORMATION_SCHEMA.CATALOGS;
Error: UNSUPPORTED_OPERATION ERROR: SMALLINT is not supported
See Apache Drill JIRA: DRILL-1959
[Error Id: 73c5560a-9e90-442b-aa04-af8bf7630a2a on atsqa4-133.qa.lab:31010]
(state=,code=0)
{code}
> TINYINT/SMALLINT's clean not-implemented-yet messages are missing or late for
> views
>
>
> Key: DRILL-2687
> URL: https://issues.apache.org/jira/browse/DRILL-2687
> Project: Apache Drill
> Issue Type: Bug
> Components: SQL Parser
>Reporter: Daniel Barclay (Drill)
> Fix For: 1.4.0
>
>
> The checks to detect uses of TINYINT and SMALLINT and directly report that
> they are not implemented yet doesn't seem to cover view creation:
> 1. Some cases report a very unclean error:
> {noformat}
> 0: jdbc:drill:zk=local> USE dfs.tmp;
> +++
> | ok | summary |
> +++
> | true | Default schema changed to 'dfs.tmp' |
> +++
> 1 row selected (0.02 seconds)
> 0: jdbc:drill:zk=local> CREATE OR REPLACE VIEW TempView AS SELECT CAST('1' AS
> SMALLINT) FROM INFORMATION_SCHEMA.CATALOGS;
> +++
> | ok | summary |
> +++
> | true | View 'TempView' replaced successfully in 'dfs.tmp' schema |
> +++
> 1 row selected (0.027 seconds)
> 0: jdbc:drill:zk=local> SELECT * FROM TempView;
> Exception in thread "2ae0be1b-df9a-a7f8-132e-917d4bd51eaa:frag:0:0"
> java.lang.RuntimeException: Error closing fragment context.
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.closeOutResources(FragmentExecutor.java:224)
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:187)
> at
> org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.RuntimeException: cast function for type SMALLINT is not
> defined
> at
> org.apache.drill.common.expression.fn.CastFunctions.getCastFunc(CastFunctions.java:101)
> at
> org.apache.drill.exec.expr.ExpressionTreeMaterializer$MaterializeVisitor.visitCastExpression(ExpressionTreeMaterializer.java:547)
> at
> org.apache.drill.exec.expr.ExpressionTreeMaterializer$MaterializeVisitor.visitCastExpression(ExpressionTreeMaterializer.java:187)
> at
> org.apache.drill.common.expression.CastExpression.accept(CastExpression.java:43)
> at
> org.apache.drill.exec.expr.ExpressionTreeMaterializer.materialize(ExpressionTreeMaterializer.java:103)
> at
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.setupNewSchema(ProjectRecordBatch.java:393)
> at
> org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:78)
> at
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:134)
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:142)
> at
> org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:118)
> at
> org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:68)
> at
> org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext(ScreenCreator.java:96)
> at
> org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:58)
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:163)
> ... 4 more
> Query failed: RemoteRpcException: Failure while running fragment., cast
> function for type SMALLINT is not defined [
> f2543aac-aaef-4090-89f7-32746efa4242 on dev-linux2:31013 ]
> [ f2543aac-aaef-4090-89f7-32746efa4242 on dev-linux2:31013 ]
> Error: exception while executing query: Failure while executing query.
> (state=,code=0)
> 0: jdbc:drill:zk=local>
> 0: jdbc:drill:zk=local>
> {noformat}
> 2. Other cases report a less messy but still non-specific error:
> {noformat}
> 0: jdbc:drill:zk=local>
> 0: jdbc:drill:zk=local> USE dfs.tmp;
> +++
> | ok | summary |
> +++
[jira] [Closed] (DRILL-2687) TINYINT/SMALLINT's clean not-implemented-yet messages are missing or late for views
[
https://issues.apache.org/jira/browse/DRILL-2687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Victoria Markman closed DRILL-2687.
---
> TINYINT/SMALLINT's clean not-implemented-yet messages are missing or late for
> views
>
>
> Key: DRILL-2687
> URL: https://issues.apache.org/jira/browse/DRILL-2687
> Project: Apache Drill
> Issue Type: Bug
> Components: SQL Parser
>Reporter: Daniel Barclay (Drill)
> Fix For: 1.4.0
>
>
> The checks to detect uses of TINYINT and SMALLINT and directly report that
> they are not implemented yet doesn't seem to cover view creation:
> 1. Some cases report a very unclean error:
> {noformat}
> 0: jdbc:drill:zk=local> USE dfs.tmp;
> +++
> | ok | summary |
> +++
> | true | Default schema changed to 'dfs.tmp' |
> +++
> 1 row selected (0.02 seconds)
> 0: jdbc:drill:zk=local> CREATE OR REPLACE VIEW TempView AS SELECT CAST('1' AS
> SMALLINT) FROM INFORMATION_SCHEMA.CATALOGS;
> +++
> | ok | summary |
> +++
> | true | View 'TempView' replaced successfully in 'dfs.tmp' schema |
> +++
> 1 row selected (0.027 seconds)
> 0: jdbc:drill:zk=local> SELECT * FROM TempView;
> Exception in thread "2ae0be1b-df9a-a7f8-132e-917d4bd51eaa:frag:0:0"
> java.lang.RuntimeException: Error closing fragment context.
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.closeOutResources(FragmentExecutor.java:224)
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:187)
> at
> org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.RuntimeException: cast function for type SMALLINT is not
> defined
> at
> org.apache.drill.common.expression.fn.CastFunctions.getCastFunc(CastFunctions.java:101)
> at
> org.apache.drill.exec.expr.ExpressionTreeMaterializer$MaterializeVisitor.visitCastExpression(ExpressionTreeMaterializer.java:547)
> at
> org.apache.drill.exec.expr.ExpressionTreeMaterializer$MaterializeVisitor.visitCastExpression(ExpressionTreeMaterializer.java:187)
> at
> org.apache.drill.common.expression.CastExpression.accept(CastExpression.java:43)
> at
> org.apache.drill.exec.expr.ExpressionTreeMaterializer.materialize(ExpressionTreeMaterializer.java:103)
> at
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.setupNewSchema(ProjectRecordBatch.java:393)
> at
> org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:78)
> at
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:134)
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:142)
> at
> org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:118)
> at
> org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:68)
> at
> org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext(ScreenCreator.java:96)
> at
> org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:58)
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:163)
> ... 4 more
> Query failed: RemoteRpcException: Failure while running fragment., cast
> function for type SMALLINT is not defined [
> f2543aac-aaef-4090-89f7-32746efa4242 on dev-linux2:31013 ]
> [ f2543aac-aaef-4090-89f7-32746efa4242 on dev-linux2:31013 ]
> Error: exception while executing query: Failure while executing query.
> (state=,code=0)
> 0: jdbc:drill:zk=local>
> 0: jdbc:drill:zk=local>
> {noformat}
> 2. Other cases report a less messy but still non-specific error:
> {noformat}
> 0: jdbc:drill:zk=local>
> 0: jdbc:drill:zk=local> USE dfs.tmp;
> +++
> | ok | summary |
> +++
> | true | Default schema changed to 'dfs.tmp' |
> +++
> 1 row selected (0.036 seconds)
> 0: jdbc:drill:zk=local> CREATE OR REPLACE VIEW TempView AS SELECT CAST(1 AS
> SMALLINT) FROM INFORMATION_SCHEMA.CATALOGS;
> +++
> | ok | summary |
> +++
> | true | View 'TempView' replaced successfully in 'dfs.tmp' schema |
> +++
> 1 row selected (0.043 seconds)
> 0: jdbc
[jira] [Resolved] (DRILL-2687) TINYINT/SMALLINT's clean not-implemented-yet messages are missing or late for views
[
https://issues.apache.org/jira/browse/DRILL-2687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Victoria Markman resolved DRILL-2687.
-
Resolution: Fixed
> TINYINT/SMALLINT's clean not-implemented-yet messages are missing or late for
> views
>
>
> Key: DRILL-2687
> URL: https://issues.apache.org/jira/browse/DRILL-2687
> Project: Apache Drill
> Issue Type: Bug
> Components: SQL Parser
>Reporter: Daniel Barclay (Drill)
> Fix For: 1.4.0
>
>
> The checks to detect uses of TINYINT and SMALLINT and directly report that
> they are not implemented yet doesn't seem to cover view creation:
> 1. Some cases report a very unclean error:
> {noformat}
> 0: jdbc:drill:zk=local> USE dfs.tmp;
> +++
> | ok | summary |
> +++
> | true | Default schema changed to 'dfs.tmp' |
> +++
> 1 row selected (0.02 seconds)
> 0: jdbc:drill:zk=local> CREATE OR REPLACE VIEW TempView AS SELECT CAST('1' AS
> SMALLINT) FROM INFORMATION_SCHEMA.CATALOGS;
> +++
> | ok | summary |
> +++
> | true | View 'TempView' replaced successfully in 'dfs.tmp' schema |
> +++
> 1 row selected (0.027 seconds)
> 0: jdbc:drill:zk=local> SELECT * FROM TempView;
> Exception in thread "2ae0be1b-df9a-a7f8-132e-917d4bd51eaa:frag:0:0"
> java.lang.RuntimeException: Error closing fragment context.
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.closeOutResources(FragmentExecutor.java:224)
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:187)
> at
> org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.RuntimeException: cast function for type SMALLINT is not
> defined
> at
> org.apache.drill.common.expression.fn.CastFunctions.getCastFunc(CastFunctions.java:101)
> at
> org.apache.drill.exec.expr.ExpressionTreeMaterializer$MaterializeVisitor.visitCastExpression(ExpressionTreeMaterializer.java:547)
> at
> org.apache.drill.exec.expr.ExpressionTreeMaterializer$MaterializeVisitor.visitCastExpression(ExpressionTreeMaterializer.java:187)
> at
> org.apache.drill.common.expression.CastExpression.accept(CastExpression.java:43)
> at
> org.apache.drill.exec.expr.ExpressionTreeMaterializer.materialize(ExpressionTreeMaterializer.java:103)
> at
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.setupNewSchema(ProjectRecordBatch.java:393)
> at
> org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:78)
> at
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:134)
> at
> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:142)
> at
> org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:118)
> at
> org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:68)
> at
> org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext(ScreenCreator.java:96)
> at
> org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:58)
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:163)
> ... 4 more
> Query failed: RemoteRpcException: Failure while running fragment., cast
> function for type SMALLINT is not defined [
> f2543aac-aaef-4090-89f7-32746efa4242 on dev-linux2:31013 ]
> [ f2543aac-aaef-4090-89f7-32746efa4242 on dev-linux2:31013 ]
> Error: exception while executing query: Failure while executing query.
> (state=,code=0)
> 0: jdbc:drill:zk=local>
> 0: jdbc:drill:zk=local>
> {noformat}
> 2. Other cases report a less messy but still non-specific error:
> {noformat}
> 0: jdbc:drill:zk=local>
> 0: jdbc:drill:zk=local> USE dfs.tmp;
> +++
> | ok | summary |
> +++
> | true | Default schema changed to 'dfs.tmp' |
> +++
> 1 row selected (0.036 seconds)
> 0: jdbc:drill:zk=local> CREATE OR REPLACE VIEW TempView AS SELECT CAST(1 AS
> SMALLINT) FROM INFORMATION_SCHEMA.CATALOGS;
> +++
> | ok | summary |
> +++
> | true | View 'TempView' replaced successfully in 'dfs.tmp' schema |
> +++
> 1 row selected
[jira] [Updated] (DRILL-3942) IS NOT NULL filter is not pushed pass aggregation
[
https://issues.apache.org/jira/browse/DRILL-3942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Victoria Markman updated DRILL-3942:
Description:
It seems to me that we should be able to do that, x is a grouping column:
{code}
0: jdbc:drill:schema=dfs> explain plan for select x, y, z from ( select
ss_sold_date_sk, ss_customer_sk, avg(ss_quantity) from store_sales group by
ss_sold_date_sk, ss_customer_sk ) as sq(x, y, z) where x is not null;
+--+--+
| text | json |
+--+--+
| 00-00Screen
00-01 Project(x=[$0], y=[$1], z=[$2])
00-02UnionExchange
01-01 Project(x=[$0], y=[$1], z=[$2])
01-02Project(ss_sold_date_sk=[$0], ss_customer_sk=[$1],
EXPR$2=[CAST(/(CastHigh(CASE(=($3, 0), null, $2)), $3)):ANY NOT NULL])
01-03 SelectionVectorRemover
01-04Filter(condition=[IS NOT NULL($0)])
01-05 HashAgg(group=[{0, 1}], agg#0=[$SUM0($2)],
agg#1=[$SUM0($3)])
01-06Project(ss_sold_date_sk=[$0], ss_customer_sk=[$1],
$f2=[$2], $f3=[$3])
01-07 HashToRandomExchange(dist0=[[$0]], dist1=[[$1]])
02-01UnorderedMuxExchange
03-01 Project(ss_sold_date_sk=[$0],
ss_customer_sk=[$1], $f2=[$2], $f3=[$3],
E_X_P_R_H_A_S_H_F_I_E_L_D=[castInt(hash64AsDouble($1, hash64AsDouble($0)))])
03-02HashAgg(group=[{0, 1}], agg#0=[$SUM0($2)],
agg#1=[COUNT($2)])
03-03 Project(ss_sold_date_sk=[$2],
ss_customer_sk=[$1], ss_quantity=[$0])
03-04Scan(groupscan=[ParquetGroupScan
[entries=[ReadEntryWithPath [path=maprfs:///tpcds1/parquet/store_sales]],
selectionRoot=maprfs:/tpcds1/parquet/store_sales, numFiles=1,
usedMetadataFile=false, columns=[`ss_sold_date_sk`, `ss_customer_sk`,
`ss_quantity`]]])
{code}
If I add another not null filter, it is pushed down:
{code}
0: jdbc:drill:schema=dfs> explain plan for select x, y, z from ( select
ss_sold_date_sk, ss_customer_sk, avg(ss_quantity) from store_sales group by
ss_sold_date_sk, ss_customer_sk ) as sq(x, y, z) where x is not null and y is
not null;
+--+--+
| text | json |
+--+--+
| 00-00Screen
00-01 Project(x=[$0], y=[$1], z=[$2])
00-02UnionExchange
01-01 Project(x=[$0], y=[$1], z=[$2])
01-02Project(ss_sold_date_sk=[$0], ss_customer_sk=[$1],
EXPR$2=[CAST(/(CastHigh(CASE(=($3, 0), null, $2)), $3)):ANY NOT NULL])
01-03 HashAgg(group=[{0, 1}], agg#0=[$SUM0($2)], agg#1=[$SUM0($3)])
01-04Project(ss_sold_date_sk=[$0], ss_customer_sk=[$1],
$f2=[$2], $f3=[$3])
01-05 HashToRandomExchange(dist0=[[$0]], dist1=[[$1]])
02-01UnorderedMuxExchange
03-01 Project(ss_sold_date_sk=[$0], ss_customer_sk=[$1],
$f2=[$2], $f3=[$3], E_X_P_R_H_A_S_H_F_I_E_L_D=[castInt(hash64AsDouble($1,
hash64AsDouble($0)))])
03-02HashAgg(group=[{0, 1}], agg#0=[$SUM0($2)],
agg#1=[COUNT($2)])
03-03 SelectionVectorRemover
03-04Filter(condition=[AND(IS NOT NULL($0), IS NOT
NULL($1))])
03-05 Project(ss_sold_date_sk=[$2],
ss_customer_sk=[$1], ss_quantity=[$0])
03-06Scan(groupscan=[ParquetGroupScan
[entries=[ReadEntryWithPath [path=maprfs:///tpcds1/parquet/store_sales]],
selectionRoot=maprfs:/tpcds1/parquet/store_sales, numFiles=1,
usedMetadataFile=false, columns=[`ss_sold_date_sk`, `ss_customer_sk`,
`ss_quantity`]]])
{code}
IS NULL filter is pushed down:
{code}
0: jdbc:drill:schema=dfs> explain plan for select x, y, z from ( select
ss_sold_date_sk, ss_customer_sk, avg(ss_quantity) from store_sales group by
ss_sold_date_sk, ss_customer_sk ) as sq(x, y, z) where x is null;
+--+--+
| text | json |
+--+--+
| 00-00Screen
00-01 Project(x=[$0], y=[$1], z=[$2])
00-02Project(x=[$0], y=[$1], z=[$2])
00-03 Project(ss_sold_date_sk=[$0], ss_customer_sk=[$1],
EXPR$2=[CAST(/(CastHigh(CASE(=($3, 0), null, $2)), $3)):ANY NOT NULL])
00-04HashAgg(group=[{0, 1}], agg#0=[$SUM0($2)], agg#1=[$SUM0($3)])
00-05 HashAgg(group=[{0, 1}], agg#0=[$SUM0($2)], agg#1=[COUNT($2)])
00-06SelectionVectorRemover
00-07 Filter(condition=[IS NULL($0)])
00-08Project(ss_sold_date_sk=[$2], ss_customer_sk=[$1],
ss_quantity=[$0])
00-09 Scan(groupscan=[ParquetGroupScan
[entries=[ReadEntryWithPath [path=maprfs:///tpcds1/parquet/store_sales]],
selectionRoot=maprfs:/tpcds1/parquet/store_sales, numFiles=1,
usedMetadataFile=false, columns=[`ss_sold_date_sk`, `ss_customer_sk`,
`ss_quantity`]]])
{code}
was:
It seems to me that we should be able to do that, x is a grouping column:
{code}
0: jdbc:drill:schema=dfs> explain
[jira] [Created] (DRILL-3942) IS NOT NULL filter is not pushed pass aggregation
Victoria Markman created DRILL-3942:
---
Summary: IS NOT NULL filter is not pushed pass aggregation
Key: DRILL-3942
URL: https://issues.apache.org/jira/browse/DRILL-3942
Project: Apache Drill
Issue Type: Bug
Components: Query Planning & Optimization
Affects Versions: 1.2.0
Reporter: Victoria Markman
It seems to me that we should be able to do that, x is a grouping column:
{code}
0: jdbc:drill:schema=dfs> explain plan for select x, y, z from ( select
ss_sold_date_sk, ss_customer_sk, avg(ss_quantity) from store_sales group by
ss_sold_date_sk, ss_customer_sk ) as sq(x, y, z) where x is not null;
+--+--+
| text | json |
+--+--+
| 00-00Screen
00-01 Project(x=[$0], y=[$1], z=[$2])
00-02UnionExchange
01-01 Project(x=[$0], y=[$1], z=[$2])
01-02Project(ss_sold_date_sk=[$0], ss_customer_sk=[$1],
EXPR$2=[CAST(/(CastHigh(CASE(=($3, 0), null, $2)), $3)):ANY NOT NULL])
01-03 SelectionVectorRemover
01-04Filter(condition=[IS NOT NULL($0)])
01-05 HashAgg(group=[{0, 1}], agg#0=[$SUM0($2)],
agg#1=[$SUM0($3)])
01-06Project(ss_sold_date_sk=[$0], ss_customer_sk=[$1],
$f2=[$2], $f3=[$3])
01-07 HashToRandomExchange(dist0=[[$0]], dist1=[[$1]])
02-01UnorderedMuxExchange
03-01 Project(ss_sold_date_sk=[$0],
ss_customer_sk=[$1], $f2=[$2], $f3=[$3],
E_X_P_R_H_A_S_H_F_I_E_L_D=[castInt(hash64AsDouble($1, hash64AsDouble($0)))])
03-02HashAgg(group=[{0, 1}], agg#0=[$SUM0($2)],
agg#1=[COUNT($2)])
03-03 Project(ss_sold_date_sk=[$2],
ss_customer_sk=[$1], ss_quantity=[$0])
03-04Scan(groupscan=[ParquetGroupScan
[entries=[ReadEntryWithPath [path=maprfs:///tpcds1/parquet/store_sales]],
selectionRoot=maprfs:/tpcds1/parquet/store_sales, numFiles=1,
usedMetadataFile=false, columns=[`ss_sold_date_sk`, `ss_customer_sk`,
`ss_quantity`]]])
{code}
If I add another not null filter, it is pushed down:
{code}
0: jdbc:drill:schema=dfs> explain plan for select x, y, z from ( select
ss_sold_date_sk, ss_customer_sk, avg(ss_quantity) from store_sales group by
ss_sold_date_sk, ss_customer_sk ) as sq(x, y, z) where x is not null and y is
not null;
+--+--+
| text | json |
+--+--+
| 00-00Screen
00-01 Project(x=[$0], y=[$1], z=[$2])
00-02UnionExchange
01-01 Project(x=[$0], y=[$1], z=[$2])
01-02Project(ss_sold_date_sk=[$0], ss_customer_sk=[$1],
EXPR$2=[CAST(/(CastHigh(CASE(=($3, 0), null, $2)), $3)):ANY NOT NULL])
01-03 HashAgg(group=[{0, 1}], agg#0=[$SUM0($2)], agg#1=[$SUM0($3)])
01-04Project(ss_sold_date_sk=[$0], ss_customer_sk=[$1],
$f2=[$2], $f3=[$3])
01-05 HashToRandomExchange(dist0=[[$0]], dist1=[[$1]])
02-01UnorderedMuxExchange
03-01 Project(ss_sold_date_sk=[$0], ss_customer_sk=[$1],
$f2=[$2], $f3=[$3], E_X_P_R_H_A_S_H_F_I_E_L_D=[castInt(hash64AsDouble($1,
hash64AsDouble($0)))])
03-02HashAgg(group=[{0, 1}], agg#0=[$SUM0($2)],
agg#1=[COUNT($2)])
03-03 SelectionVectorRemover
03-04Filter(condition=[AND(IS NOT NULL($0), IS NOT
NULL($1))])
03-05 Project(ss_sold_date_sk=[$2],
ss_customer_sk=[$1], ss_quantity=[$0])
03-06Scan(groupscan=[ParquetGroupScan
[entries=[ReadEntryWithPath [path=maprfs:///tpcds1/parquet/store_sales]],
selectionRoot=maprfs:/tpcds1/parquet/store_sales, numFiles=1,
usedMetadataFile=false, columns=[`ss_sold_date_sk`, `ss_customer_sk`,
`ss_quantity`]]])
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (DRILL-3936) We don't handle out of memory condition during build phase of hash join
Victoria Markman created DRILL-3936:
---
Summary: We don't handle out of memory condition during build
phase of hash join
Key: DRILL-3936
URL: https://issues.apache.org/jira/browse/DRILL-3936
Project: Apache Drill
Issue Type: Bug
Components: Execution - Relational Operators
Reporter: Victoria Markman
It looks like we just fall through ( see excerpt from HashJoinBatch.java below )
{code:java}
public void executeBuildPhase() throws SchemaChangeException,
ClassTransformationException, IOException {
//Setup the underlying hash table
// skip first batch if count is zero, as it may be an empty schema batch
if (right.getRecordCount() == 0) {
for (final VectorWrapper w : right) {
w.clear();
}
rightUpstream = next(right);
}
boolean moreData = true;
while (moreData) {
switch (rightUpstream) {
case OUT_OF_MEMORY:
case NONE:
case NOT_YET:
case STOP:
moreData = false;
continue;
...
{code}
We don't handle it later either:
{code:java}
public IterOutcome innerNext() {
try {
/* If we are here for the first time, execute the build phase of the
* hash join and setup the run time generated class for the probe side
*/
if (state == BatchState.FIRST) {
// Build the hash table, using the build side record batches.
executeBuildPhase();
//IterOutcome next = next(HashJoinHelper.LEFT_INPUT,
left);
hashJoinProbe.setupHashJoinProbe(context, hyperContainer, left,
left.getRecordCount(), this, hashTable,
hjHelper, joinType);
// Update the hash table related stats for the operator
updateStats(this.hashTable);
}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (DRILL-2759) Wrong result on count(*) from empty csv file
[
https://issues.apache.org/jira/browse/DRILL-2759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Victoria Markman updated DRILL-2759:
Summary: Wrong result on count(*) from empty csv file (was: Improve error
message when reading from an empty csv file)
> Wrong result on count(*) from empty csv file
>
>
> Key: DRILL-2759
> URL: https://issues.apache.org/jira/browse/DRILL-2759
> Project: Apache Drill
> Issue Type: Bug
> Components: Storage - Text & CSV
>Affects Versions: 0.9.0
>Reporter: Victoria Markman
>Priority: Critical
> Fix For: Future
>
>
> t1.csv is an empty file:
> {code}
> 0: jdbc:drill:schema=dfs> select count(*) from `bigtable/2015/01/t1.csv`;
> Query failed: IllegalArgumentException: Incoming endpoints 1 is greater than
> number of row groups 0
> Error: exception while executing query: Failure while executing query.
> (state=,code=0)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (DRILL-3714) Query runs out of memory and remains in CANCELLATION_REQUESTED state until drillbit is restarted
[
https://issues.apache.org/jira/browse/DRILL-3714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Victoria Markman updated DRILL-3714:
Priority: Critical (was: Major)
> Query runs out of memory and remains in CANCELLATION_REQUESTED state until
> drillbit is restarted
>
>
> Key: DRILL-3714
> URL: https://issues.apache.org/jira/browse/DRILL-3714
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Flow
>Affects Versions: 1.2.0
>Reporter: Victoria Markman
>Priority: Critical
> Fix For: Future
>
> Attachments: Screen Shot 2015-08-26 at 10.36.33 AM.png, drillbit.log,
> jstack.txt, query_profile_2a2210a7-7a78-c774-d54c-c863d0b77bb0.json
>
>
> This is a variation of DRILL-3705 with the difference of drill behavior when
> hitting OOM condition.
> Query runs out of memory during execution and remains in
> "CANCELLATION_REQUESTED" state until drillbit is bounced.
> Client (sqlline in this case) never gets a response from the server.
> Reproduction details:
> Single node drillbit installation.
> DRILL_MAX_DIRECT_MEMORY="8G"
> DRILL_HEAP="4G"
> Run this query on TPCDS SF100 data set
> {code}
> SELECT SUM(ss.ss_net_paid_inc_tax) OVER (PARTITION BY ss.ss_store_sk) AS
> TotalSpend FROM store_sales ss WHERE ss.ss_store_sk IS NOT NULL ORDER BY 1
> LIMIT 10;
> {code}
> drillbit.log
> {code}
> 2015-08-26 16:54:58,469 [2a2210a7-7a78-c774-d54c-c863d0b77bb0:frag:3:22] INFO
> o.a.d.e.w.f.FragmentStatusReporter -
> 2a2210a7-7a78-c774-d54c-c863d0b77bb0:3:22: State to report: RUNNING
> 2015-08-26 16:55:50,498 [BitServer-5] WARN
> o.a.drill.exec.rpc.data.DataServer - Message of mode REQUEST of rpc type 3
> took longer than 500ms. Actual duration was 2569ms.
> 2015-08-26 16:56:31,086 [BitServer-5] ERROR
> o.a.d.exec.rpc.RpcExceptionHandler - Exception in RPC communication.
> Connection: /10.10.88.133:31012 <--> /10.10.88.133:54554 (data server).
> Closing connection.
> io.netty.handler.codec.DecoderException: java.lang.OutOfMemoryError: Direct
> buffer memory
> at
> io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:233)
> ~[netty-codec-4.0.27.Final.jar:4.0.27.Final]
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
> [netty-transport-4.0.27.Final.jar:4.0.27.Final]
> at
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
> [netty-transport-4.0.27.Final.jar:4.0.27.Final]
> at
> io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86)
> [netty-transport-4.0.27.Final.jar:4.0.27.Final]
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
> [netty-transport-4.0.27.Final.jar:4.0.27.Final]
> at
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
> [netty-transport-4.0.27.Final.jar:4.0.27.Final]
> at
> io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:847)
> [netty-transport-4.0.27.Final.jar:4.0.27.Final]
> at
> io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:618)
> [netty-transport-native-epoll-4.0.27.Final-linux-x86_64.jar:na]
> at
> io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:329)
> [netty-transport-native-epoll-4.0.27.Final-linux-x86_64.jar:na]
> at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:250)
> [netty-transport-native-epoll-4.0.27.Final-linux-x86_64.jar:na]
> at
> io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
> [netty-common-4.0.27.Final.jar:4.0.27.Final]
> at java.lang.Thread.run(Thread.java:745) [na:1.7.0_71]
> Caused by: java.lang.OutOfMemoryError: Direct buffer memory
> at java.nio.Bits.reserveMemory(Bits.java:658) ~[na:1.7.0_71]
> at java.nio.DirectByteBuffer.(DirectByteBuffer.java:123)
> ~[na:1.7.0_71]
> at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:306)
> ~[na:1.7.0_71]
> at io.netty.buffer.PoolArena$DirectArena.newChunk(PoolArena.java:437)
> ~[netty-buffer-4.0.27.Final.jar:4.0.27.Final]
> at io.netty.buffer.PoolArena.allocateNormal(PoolArena.java:179)
> ~[netty-buffer-4.0.27.Final.jar:4.0.27.Final]
> at io.netty.buffer.PoolArena.allocate(PoolArena.java:168)
> ~[netty-buffer-4.0.27.Final.jar:4.0.27.Final]
> at io.netty.buffer.PoolArena.reallocate(PoolArena.java:280)
> ~[netty-buffer-4.0.27.Final.jar:4.0.27.Final]
> at io.net
[jira] [Updated] (DRILL-3705) Query runs out of memory, reported as FAILED and leaves thread running
[
https://issues.apache.org/jira/browse/DRILL-3705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Victoria Markman updated DRILL-3705:
Priority: Critical (was: Major)
> Query runs out of memory, reported as FAILED and leaves thread running
> ---
>
> Key: DRILL-3705
> URL: https://issues.apache.org/jira/browse/DRILL-3705
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Flow
>Affects Versions: 1.2.0
>Reporter: Victoria Markman
>Priority: Critical
> Fix For: Future
>
> Attachments: 2a2451ec-09d8-9f26-e856-5fd349ae72fd.sys.drill,
> drillbit.log, jstack.txt
>
>
> Single node drill installation
> DRILL_MAX_DIRECT_MEMORY="2G"
> DRILL_HEAP="1G"
> Execute tpcds query 15 SF100 (parquet) with the settings above. Reproduces 2
> out of 3 times.
> {code}
> SELECT ca.ca_zip,
>Sum(cs.cs_sales_price)
> FROM catalog_salescs,
>customer c,
>customer_address ca,
>date_dim dd
> WHERE cs.cs_bill_customer_sk = c.c_customer_sk
>AND c.c_current_addr_sk = ca.ca_address_sk
>AND ( Substr(ca.ca_zip, 1, 5) IN ( '85669', '86197', '88274', '83405',
>'86475', '85392', '85460', '80348',
>'81792' )
> OR ca.ca_state IN ( 'CA', 'WA', 'GA' )
> OR cs.cs_sales_price > 500 )
>AND cs.cs_sold_date_sk = dd.d_date_sk
>AND dd.d_qoy = 1
>AND dd.d_year = 1998
> GROUP BY ca.ca_zip
> ORDER BY ca.ca_zip
> LIMIT 100;
> {code}
> Query runs out of memory, but leaves thread behind even though it is reported
> as FAILED (expected result)
> Snippet from jstack:
> {code}
> "2a2451ec-09d8-9f26-e856-5fd349ae72fd:frag:4:0" daemon prio=10
> tid=0x7f507414 nid=0x3000 waiting on condition [0x7f5055b66000]
>java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0xc012b038> (a
> java.util.concurrent.Semaphore$NonfairSync)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:834)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:994)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1303)
> at java.util.concurrent.Semaphore.acquire(Semaphore.java:472)
> at
> org.apache.drill.exec.ops.SendingAccountor.waitForSendComplete(SendingAccountor.java:48)
> - locked <0xc012b068> (a
> org.apache.drill.exec.ops.SendingAccountor)
> at
> org.apache.drill.exec.ops.FragmentContext.waitForSendComplete(FragmentContext.java:436)
> at
> org.apache.drill.exec.physical.impl.BaseRootExec.close(BaseRootExec.java:112)
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.closeOutResources(FragmentExecutor.java:341)
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:173)
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:292)
> at
> org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> NPE in drillbit.log:
> {code}
> 2015-08-24 23:52:04,486 [BitServer-5] ERROR
> o.a.d.exec.rpc.RpcExceptionHandler - Exception in RPC communication.
> Connection: /10.10.88.133:31012 <--> /10.10.88.133:52417 (data server).
> Closing connection.
> io.netty.handler.codec.DecoderException: java.lang.NullPointerException
> at
> io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:99)
> [netty-codec-4.0.27.Final.jar:4.0.27.Final]
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
> [netty-transport-4.0.27.Final.jar:4.0.27.Final]
> at
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
> [netty-transport-4.0.27.Final.jar:4.0.27.Final]
> at
> io.netty.handler.timeout.ReadTimeoutHandler.channelRead(ReadTimeoutHandler.java:150)
> [netty-handler-4.0.27.Final.jar:4.0.27.Final]
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
> [netty-transport-4.0.27.Fin
[jira] [Updated] (DRILL-3241) Query with window function runs out of direct memory and does not report back to client that it did
[
https://issues.apache.org/jira/browse/DRILL-3241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Victoria Markman updated DRILL-3241:
Priority: Critical (was: Major)
> Query with window function runs out of direct memory and does not report back
> to client that it did
> ---
>
> Key: DRILL-3241
> URL: https://issues.apache.org/jira/browse/DRILL-3241
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Relational Operators
>Affects Versions: 1.0.0
>Reporter: Victoria Markman
>Priority: Critical
> Fix For: Future
>
>
> Even though query run out of memory and was cancelled on the server, client
> (sqlline) was never notified of the event and it appears to the user that
> query is hung.
> Configuration:
> Single drillbit configured with:
> DRILL_MAX_DIRECT_MEMORY="2G"
> DRILL_HEAP="1G"
> TPCDS100 parquet files
> Query:
> {code}
> select
> sum(ss_quantity) over(partition by ss_store_sk order by ss_sold_date_sk)
> from store_sales;
> {code}
> drillbit.log
> {code}
> 2015-06-01 21:42:29,514 [BitServer-5] ERROR
> o.a.d.exec.rpc.RpcExceptionHandler - Exception in RPC communication.
> Connection: /10.10.88.133:31012 <--> /10.10.88.133:38887 (data server).
> Closing connection.
> io.netty.handler.codec.DecoderException: java.lang.OutOfMemoryError: Direct
> buffer memory
> at
> io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:233)
> ~[netty-codec-4.0.27.Final.jar:4.0.27.Final]
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
> [netty-transport-4.0.27.Final.jar:4.0.27.Final]
> at
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
> [netty-transport-4.0.27.Final.jar:4.0.27.Final]
> at
> io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86)
> [netty-transport-4.0.27.Final.jar:4.0.27.Final]
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
> [netty-transport-4.0.27.Final.jar:4.0.27.Final]
> at
> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
> [netty-transport-4.0.27.Final.jar:4.0.27.Final]
> at
> io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:847)
> [netty-transport-4.0.27.Final.jar:4.0.27.Final]
> at
> io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:618)
> [netty-transport-native-epoll-4.0.27.Final-linux-x86_64.jar:na]
> at
> io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:329)
> [netty-transport-native-epoll-4.0.27.Final-linux-x86_64.jar:na]
> at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:250)
> [netty-transport-native-epoll-4.0.27.Final-linux-x86_64.jar:na]
> at
> io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
> [netty-common-4.0.27.Final.jar:4.0.27.Final]
> at java.lang.Thread.run(Thread.java:745) [na:1.7.0_71]
> Caused by: java.lang.OutOfMemoryError: Direct buffer memory
> at java.nio.Bits.reserveMemory(Bits.java:658) ~[na:1.7.0_71]
> at java.nio.DirectByteBuffer.(DirectByteBuffer.java:123)
> ~[na:1.7.0_71]
> at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:306)
> ~[na:1.7.0_71]
> at io.netty.buffer.PoolArena$DirectArena.newChunk(PoolArena.java:437)
> ~[netty-buffer-4.0.27.Final.jar:4.0.27.Final]
> at io.netty.buffer.PoolArena.allocateNormal(PoolArena.java:179)
> ~[netty-buffer-4.0.27.Final.jar:4.0.27.Final]
> at io.netty.buffer.PoolArena.allocate(PoolArena.java:168)
> ~[netty-buffer-4.0.27.Final.jar:4.0.27.Final]
> at io.netty.buffer.PoolArena.reallocate(PoolArena.java:280)
> ~[netty-buffer-4.0.27.Final.jar:4.0.27.Final]
> at io.netty.buffer.PooledByteBuf.capacity(PooledByteBuf.java:110)
> ~[netty-buffer-4.0.27.Final.jar:4.0.27.Final]
> at
> io.netty.buffer.AbstractByteBuf.ensureWritable(AbstractByteBuf.java:251)
> ~[netty-buffer-4.0.27.Final.jar:4.0.27.Final]
> at
> io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:849)
> ~[netty-buffer-4.0.27.Final.jar:4.0.27.Final]
> at
> io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:841)
> ~[netty-buffer-4.0.27.Final.jar:4.0.27.Final]
> at
> io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:831)
> ~[netty-buffer-4.0.27.Final.jar:4.0.27.Final]
> at io.netty.buffer.WrappedByteBuf.writeBytes(WrappedByteBuf.java:60
[jira] [Commented] (DRILL-2891) Allowing ROUND function on boolean type can cause all sorts of problems
[
https://issues.apache.org/jira/browse/DRILL-2891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14949561#comment-14949561
]
Victoria Markman commented on DRILL-2891:
-
Now that I think about it, first case is technically data corruption. Raising
priority to critical.
> Allowing ROUND function on boolean type can cause all sorts of problems
> ---
>
> Key: DRILL-2891
> URL: https://issues.apache.org/jira/browse/DRILL-2891
> Project: Apache Drill
> Issue Type: Bug
>Reporter: Victoria Markman
>Priority: Minor
> Fix For: 1.4.0
>
>
> Works, and I don't think it makes much sense:
> {code}
> 0: jdbc:drill:schema=dfs> select round(c_boolean) from alltypes_with_nulls
> limit 1;
> ++
> | EXPR$0 |
> ++
> | 1 |
> ++
> 1 row selected (0.19 seconds)
> {code}
> Fails later if used in other parts of the query.
> In order by:
> {code}
> 0: jdbc:drill:schema=dfs> select round(c_boolean) from alltypes_with_nulls
> order by 1;
> ++
> | EXPR$0 |
> ++
> Query failed: SYSTEM ERROR: java.lang.UnsupportedOperationException: Failure
> finding function that runtime code generation expected. Signature:
> compare_to_nulls_high( TINYINT:OPTIONAL, TINYINT:OPTIONAL ) returns
> INT:REQUIRED
> Fragment 0:0
> [7add2ed7-de6a-4c66-b511-ecad32413fcc on atsqa4-133.qa.lab:31010]
> java.lang.RuntimeException: java.sql.SQLException: Failure while executing
> query.
> at sqlline.SqlLine$IncrementalRows.hasNext(SqlLine.java:2514)
> at sqlline.SqlLine$TableOutputFormat.print(SqlLine.java:2148)
> at sqlline.SqlLine.print(SqlLine.java:1809)
> at sqlline.SqlLine$Commands.execute(SqlLine.java:3766)
> at sqlline.SqlLine$Commands.sql(SqlLine.java:3663)
> at sqlline.SqlLine.dispatch(SqlLine.java:889)
> at sqlline.SqlLine.begin(SqlLine.java:763)
> at sqlline.SqlLine.start(SqlLine.java:498)
> at sqlline.SqlLine.main(SqlLine.java:460)
> {code}
> In group by
> {code}
> 0: jdbc:drill:schema=dfs> select round(c_boolean) from alltypes group by
> round(c_boolean);
> Query failed: SYSTEM ERROR: Failure finding function that runtime code
> generation expected. Signature: compare_to_nulls_high( TINYINT:REQUIRED,
> TINYINT:REQUIRED ) returns INT:REQUIRED
> Fragment 0:0
> [286777b2-3395-4e44-94a2-d9dafa07f9dc on atsqa4-133.qa.lab:31010]
> Error: exception while executing query: Failure while executing query.
> (state=,code=0)
> {code}
> We should not allow that.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (DRILL-2891) Allowing ROUND function on boolean type can cause all sorts of problems
[
https://issues.apache.org/jira/browse/DRILL-2891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Victoria Markman updated DRILL-2891:
Priority: Critical (was: Minor)
> Allowing ROUND function on boolean type can cause all sorts of problems
> ---
>
> Key: DRILL-2891
> URL: https://issues.apache.org/jira/browse/DRILL-2891
> Project: Apache Drill
> Issue Type: Bug
>Reporter: Victoria Markman
>Priority: Critical
> Fix For: 1.4.0
>
>
> Works, and I don't think it makes much sense:
> {code}
> 0: jdbc:drill:schema=dfs> select round(c_boolean) from alltypes_with_nulls
> limit 1;
> ++
> | EXPR$0 |
> ++
> | 1 |
> ++
> 1 row selected (0.19 seconds)
> {code}
> Fails later if used in other parts of the query.
> In order by:
> {code}
> 0: jdbc:drill:schema=dfs> select round(c_boolean) from alltypes_with_nulls
> order by 1;
> ++
> | EXPR$0 |
> ++
> Query failed: SYSTEM ERROR: java.lang.UnsupportedOperationException: Failure
> finding function that runtime code generation expected. Signature:
> compare_to_nulls_high( TINYINT:OPTIONAL, TINYINT:OPTIONAL ) returns
> INT:REQUIRED
> Fragment 0:0
> [7add2ed7-de6a-4c66-b511-ecad32413fcc on atsqa4-133.qa.lab:31010]
> java.lang.RuntimeException: java.sql.SQLException: Failure while executing
> query.
> at sqlline.SqlLine$IncrementalRows.hasNext(SqlLine.java:2514)
> at sqlline.SqlLine$TableOutputFormat.print(SqlLine.java:2148)
> at sqlline.SqlLine.print(SqlLine.java:1809)
> at sqlline.SqlLine$Commands.execute(SqlLine.java:3766)
> at sqlline.SqlLine$Commands.sql(SqlLine.java:3663)
> at sqlline.SqlLine.dispatch(SqlLine.java:889)
> at sqlline.SqlLine.begin(SqlLine.java:763)
> at sqlline.SqlLine.start(SqlLine.java:498)
> at sqlline.SqlLine.main(SqlLine.java:460)
> {code}
> In group by
> {code}
> 0: jdbc:drill:schema=dfs> select round(c_boolean) from alltypes group by
> round(c_boolean);
> Query failed: SYSTEM ERROR: Failure finding function that runtime code
> generation expected. Signature: compare_to_nulls_high( TINYINT:REQUIRED,
> TINYINT:REQUIRED ) returns INT:REQUIRED
> Fragment 0:0
> [286777b2-3395-4e44-94a2-d9dafa07f9dc on atsqa4-133.qa.lab:31010]
> Error: exception while executing query: Failure while executing query.
> (state=,code=0)
> {code}
> We should not allow that.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (DRILL-2860) Unable to cast integer column from parquet file to interval day
[
https://issues.apache.org/jira/browse/DRILL-2860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Victoria Markman updated DRILL-2860:
Labels: interval (was: )
> Unable to cast integer column from parquet file to interval day
> ---
>
> Key: DRILL-2860
> URL: https://issues.apache.org/jira/browse/DRILL-2860
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Data Types
>Reporter: Victoria Markman
> Labels: interval
> Fix For: Future
>
> Attachments: t1.parquet
>
>
> I can cast numeric literal to "interval day":
> {code}
> 0: jdbc:drill:schema=dfs> select cast(1 as interval day) from t1;
> ++
> | EXPR$0 |
> ++
> | P1D|
> | P1D|
> | P1D|
> | P1D|
> | P1D|
> | P1D|
> | P1D|
> | P1D|
> | P1D|
> | P1D|
> ++
> 10 rows selected (0.122 seconds)
> {code}
> Get an error when I'm trying to do the same from parquet file:
> {code}
> 0: jdbc:drill:schema=dfs> select cast(a1 as interval day) from t1 where a1 =
> 1;
> Query failed: SYSTEM ERROR: Invalid format: "1"
> Fragment 0:0
> [6a4adf04-f3db-4feb-8010-ebc3bfced1e3 on atsqa4-134.qa.lab:31010]
> (java.lang.IllegalArgumentException) Invalid format: "1"
> org.joda.time.format.PeriodFormatter.parseMutablePeriod():326
> org.joda.time.format.PeriodFormatter.parsePeriod():304
> org.joda.time.Period.parse():92
> org.joda.time.Period.parse():81
> org.apache.drill.exec.test.generated.ProjectorGen180.doEval():77
> org.apache.drill.exec.test.generated.ProjectorGen180.projectRecords():62
>
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.doWork():170
> org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext():93
>
> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext():130
> org.apache.drill.exec.record.AbstractRecordBatch.next():144
>
> org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next():118
> org.apache.drill.exec.physical.impl.BaseRootExec.next():74
>
> org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext():80
> org.apache.drill.exec.physical.impl.BaseRootExec.next():64
> org.apache.drill.exec.work.fragment.FragmentExecutor$1.run():198
> org.apache.drill.exec.work.fragment.FragmentExecutor$1.run():192
> java.security.AccessController.doPrivileged():-2
> javax.security.auth.Subject.doAs():415
> org.apache.hadoop.security.UserGroupInformation.doAs():1469
> org.apache.drill.exec.work.fragment.FragmentExecutor.run():192
> org.apache.drill.common.SelfCleaningRunnable.run():38
> java.util.concurrent.ThreadPoolExecutor.runWorker():1145
> java.util.concurrent.ThreadPoolExecutor$Worker.run():615
> java.lang.Thread.run():745
> Error: exception while executing query: Failure while executing query.
> (state=,code=0)
> {code}
> If I try casting a1 to an integer I run into drill-2859
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (DRILL-2859) Unexpected exception in the query with an interval data type
[
https://issues.apache.org/jira/browse/DRILL-2859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Victoria Markman updated DRILL-2859:
Labels: interval (was: )
> Unexpected exception in the query with an interval data type
>
>
> Key: DRILL-2859
> URL: https://issues.apache.org/jira/browse/DRILL-2859
> Project: Apache Drill
> Issue Type: Bug
> Components: Query Planning & Optimization
>Affects Versions: 0.9.0
>Reporter: Victoria Markman
>Priority: Critical
> Labels: interval
> Fix For: Future
>
>
> {code}
> 0: jdbc:drill:schema=dfs> select cast(cast(a1 as int) as interval day) from
> t1 where a1 = 1;
> Query failed: SYSTEM ERROR: Unexpected exception during fragment
> initialization: todo: implement syntax
> SPECIAL(Reinterpret(*(Reinterpret(CAST(CAST($0):INTEGER):DECIMAL(2, 0)),
> 8640)))
> [5119315b-dd73-432f-ab93-49e76e9165f6 on atsqa4-134.qa.lab:31010]
> (org.apache.drill.exec.work.foreman.ForemanException) Unexpected exception
> during fragment initialization: todo: implement syntax
> SPECIAL(Reinterpret(*(Reinterpret(CAST(CAST($0):INTEGER):DECIMAL(2, 0)),
> 8640)))
> org.apache.drill.exec.work.foreman.Foreman.run():212
> java.util.concurrent.ThreadPoolExecutor.runWorker():1145
> java.util.concurrent.ThreadPoolExecutor$Worker.run():615
> java.lang.Thread.run():745
> Caused By (java.lang.AssertionError) todo: implement syntax
> SPECIAL(Reinterpret(*(Reinterpret(CAST(CAST($0):INTEGER):DECIMAL(2, 0)),
> 8640)))
>
> org.apache.drill.exec.planner.logical.DrillOptiq$RexToDrill.visitCall():182
> org.apache.drill.exec.planner.logical.DrillOptiq$RexToDrill.visitCall():73
> org.apache.calcite.rex.RexCall.accept():107
> org.apache.drill.exec.planner.logical.DrillOptiq.toDrill():70
>
> org.apache.drill.exec.planner.common.DrillProjectRelBase.getProjectExpressions():111
>
> org.apache.drill.exec.planner.physical.ProjectPrel.getPhysicalOperator():57
> org.apache.drill.exec.planner.physical.ScreenPrel.getPhysicalOperator():51
>
> org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToPop():376
> org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan():157
> org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan():167
> org.apache.drill.exec.work.foreman.Foreman.runSQL():773
> org.apache.drill.exec.work.foreman.Foreman.run():203
> java.util.concurrent.ThreadPoolExecutor.runWorker():1145
> java.util.concurrent.ThreadPoolExecutor$Worker.run():615
> java.lang.Thread.run():745
> Error: exception while executing query: Failure while executing query.
> (state=,code=0)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-2859) Unexpected exception in the query with an interval data type
[
https://issues.apache.org/jira/browse/DRILL-2859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14949551#comment-14949551
]
Victoria Markman commented on DRILL-2859:
-
I marked this bug as critical because this looks like it could be a very common
case ...
> Unexpected exception in the query with an interval data type
>
>
> Key: DRILL-2859
> URL: https://issues.apache.org/jira/browse/DRILL-2859
> Project: Apache Drill
> Issue Type: Bug
> Components: Query Planning & Optimization
>Affects Versions: 0.9.0
>Reporter: Victoria Markman
>Priority: Critical
> Fix For: Future
>
>
> {code}
> 0: jdbc:drill:schema=dfs> select cast(cast(a1 as int) as interval day) from
> t1 where a1 = 1;
> Query failed: SYSTEM ERROR: Unexpected exception during fragment
> initialization: todo: implement syntax
> SPECIAL(Reinterpret(*(Reinterpret(CAST(CAST($0):INTEGER):DECIMAL(2, 0)),
> 8640)))
> [5119315b-dd73-432f-ab93-49e76e9165f6 on atsqa4-134.qa.lab:31010]
> (org.apache.drill.exec.work.foreman.ForemanException) Unexpected exception
> during fragment initialization: todo: implement syntax
> SPECIAL(Reinterpret(*(Reinterpret(CAST(CAST($0):INTEGER):DECIMAL(2, 0)),
> 8640)))
> org.apache.drill.exec.work.foreman.Foreman.run():212
> java.util.concurrent.ThreadPoolExecutor.runWorker():1145
> java.util.concurrent.ThreadPoolExecutor$Worker.run():615
> java.lang.Thread.run():745
> Caused By (java.lang.AssertionError) todo: implement syntax
> SPECIAL(Reinterpret(*(Reinterpret(CAST(CAST($0):INTEGER):DECIMAL(2, 0)),
> 8640)))
>
> org.apache.drill.exec.planner.logical.DrillOptiq$RexToDrill.visitCall():182
> org.apache.drill.exec.planner.logical.DrillOptiq$RexToDrill.visitCall():73
> org.apache.calcite.rex.RexCall.accept():107
> org.apache.drill.exec.planner.logical.DrillOptiq.toDrill():70
>
> org.apache.drill.exec.planner.common.DrillProjectRelBase.getProjectExpressions():111
>
> org.apache.drill.exec.planner.physical.ProjectPrel.getPhysicalOperator():57
> org.apache.drill.exec.planner.physical.ScreenPrel.getPhysicalOperator():51
>
> org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToPop():376
> org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan():157
> org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan():167
> org.apache.drill.exec.work.foreman.Foreman.runSQL():773
> org.apache.drill.exec.work.foreman.Foreman.run():203
> java.util.concurrent.ThreadPoolExecutor.runWorker():1145
> java.util.concurrent.ThreadPoolExecutor$Worker.run():615
> java.lang.Thread.run():745
> Error: exception while executing query: Failure while executing query.
> (state=,code=0)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-2859) Unexpected exception in the query with an interval data type
[
https://issues.apache.org/jira/browse/DRILL-2859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14949550#comment-14949550
]
Victoria Markman commented on DRILL-2859:
-
{code}
0: jdbc:drill:schema=dfs> select cast(cast(a1 as int) as interval day) from
`t1.csv` where a1 = 1;
Error: SYSTEM ERROR: AssertionError: todo: implement syntax
SPECIAL(Reinterpret(*(Reinterpret(CAST(CAST($0):INTEGER):DECIMAL(2, 0)),
8640)))
[Error Id: 538e6136-8ef1-4a4b-ba29-3756617af561 on atsqa4-133.qa.lab:31010]
(state=,code=0)
{code}
> Unexpected exception in the query with an interval data type
>
>
> Key: DRILL-2859
> URL: https://issues.apache.org/jira/browse/DRILL-2859
> Project: Apache Drill
> Issue Type: Bug
> Components: Query Planning & Optimization
>Affects Versions: 0.9.0
>Reporter: Victoria Markman
> Fix For: Future
>
>
> {code}
> 0: jdbc:drill:schema=dfs> select cast(cast(a1 as int) as interval day) from
> t1 where a1 = 1;
> Query failed: SYSTEM ERROR: Unexpected exception during fragment
> initialization: todo: implement syntax
> SPECIAL(Reinterpret(*(Reinterpret(CAST(CAST($0):INTEGER):DECIMAL(2, 0)),
> 8640)))
> [5119315b-dd73-432f-ab93-49e76e9165f6 on atsqa4-134.qa.lab:31010]
> (org.apache.drill.exec.work.foreman.ForemanException) Unexpected exception
> during fragment initialization: todo: implement syntax
> SPECIAL(Reinterpret(*(Reinterpret(CAST(CAST($0):INTEGER):DECIMAL(2, 0)),
> 8640)))
> org.apache.drill.exec.work.foreman.Foreman.run():212
> java.util.concurrent.ThreadPoolExecutor.runWorker():1145
> java.util.concurrent.ThreadPoolExecutor$Worker.run():615
> java.lang.Thread.run():745
> Caused By (java.lang.AssertionError) todo: implement syntax
> SPECIAL(Reinterpret(*(Reinterpret(CAST(CAST($0):INTEGER):DECIMAL(2, 0)),
> 8640)))
>
> org.apache.drill.exec.planner.logical.DrillOptiq$RexToDrill.visitCall():182
> org.apache.drill.exec.planner.logical.DrillOptiq$RexToDrill.visitCall():73
> org.apache.calcite.rex.RexCall.accept():107
> org.apache.drill.exec.planner.logical.DrillOptiq.toDrill():70
>
> org.apache.drill.exec.planner.common.DrillProjectRelBase.getProjectExpressions():111
>
> org.apache.drill.exec.planner.physical.ProjectPrel.getPhysicalOperator():57
> org.apache.drill.exec.planner.physical.ScreenPrel.getPhysicalOperator():51
>
> org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToPop():376
> org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan():157
> org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan():167
> org.apache.drill.exec.work.foreman.Foreman.runSQL():773
> org.apache.drill.exec.work.foreman.Foreman.run():203
> java.util.concurrent.ThreadPoolExecutor.runWorker():1145
> java.util.concurrent.ThreadPoolExecutor$Worker.run():615
> java.lang.Thread.run():745
> Error: exception while executing query: Failure while executing query.
> (state=,code=0)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (DRILL-2859) Unexpected exception in the query with an interval data type
[
https://issues.apache.org/jira/browse/DRILL-2859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Victoria Markman updated DRILL-2859:
Priority: Critical (was: Major)
> Unexpected exception in the query with an interval data type
>
>
> Key: DRILL-2859
> URL: https://issues.apache.org/jira/browse/DRILL-2859
> Project: Apache Drill
> Issue Type: Bug
> Components: Query Planning & Optimization
>Affects Versions: 0.9.0
>Reporter: Victoria Markman
>Priority: Critical
> Fix For: Future
>
>
> {code}
> 0: jdbc:drill:schema=dfs> select cast(cast(a1 as int) as interval day) from
> t1 where a1 = 1;
> Query failed: SYSTEM ERROR: Unexpected exception during fragment
> initialization: todo: implement syntax
> SPECIAL(Reinterpret(*(Reinterpret(CAST(CAST($0):INTEGER):DECIMAL(2, 0)),
> 8640)))
> [5119315b-dd73-432f-ab93-49e76e9165f6 on atsqa4-134.qa.lab:31010]
> (org.apache.drill.exec.work.foreman.ForemanException) Unexpected exception
> during fragment initialization: todo: implement syntax
> SPECIAL(Reinterpret(*(Reinterpret(CAST(CAST($0):INTEGER):DECIMAL(2, 0)),
> 8640)))
> org.apache.drill.exec.work.foreman.Foreman.run():212
> java.util.concurrent.ThreadPoolExecutor.runWorker():1145
> java.util.concurrent.ThreadPoolExecutor$Worker.run():615
> java.lang.Thread.run():745
> Caused By (java.lang.AssertionError) todo: implement syntax
> SPECIAL(Reinterpret(*(Reinterpret(CAST(CAST($0):INTEGER):DECIMAL(2, 0)),
> 8640)))
>
> org.apache.drill.exec.planner.logical.DrillOptiq$RexToDrill.visitCall():182
> org.apache.drill.exec.planner.logical.DrillOptiq$RexToDrill.visitCall():73
> org.apache.calcite.rex.RexCall.accept():107
> org.apache.drill.exec.planner.logical.DrillOptiq.toDrill():70
>
> org.apache.drill.exec.planner.common.DrillProjectRelBase.getProjectExpressions():111
>
> org.apache.drill.exec.planner.physical.ProjectPrel.getPhysicalOperator():57
> org.apache.drill.exec.planner.physical.ScreenPrel.getPhysicalOperator():51
>
> org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToPop():376
> org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan():157
> org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan():167
> org.apache.drill.exec.work.foreman.Foreman.runSQL():773
> org.apache.drill.exec.work.foreman.Foreman.run():203
> java.util.concurrent.ThreadPoolExecutor.runWorker():1145
> java.util.concurrent.ThreadPoolExecutor$Worker.run():615
> java.lang.Thread.run():745
> Error: exception while executing query: Failure while executing query.
> (state=,code=0)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (DRILL-2859) Unexpected exception in the query with an interval data type
[
https://issues.apache.org/jira/browse/DRILL-2859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14949550#comment-14949550
]
Victoria Markman edited comment on DRILL-2859 at 10/8/15 10:43 PM:
---
As of 1.2.0:
{code}
0: jdbc:drill:schema=dfs> select cast(cast(a1 as int) as interval day) from
`t1.csv` where a1 = 1;
Error: SYSTEM ERROR: AssertionError: todo: implement syntax
SPECIAL(Reinterpret(*(Reinterpret(CAST(CAST($0):INTEGER):DECIMAL(2, 0)),
8640)))
[Error Id: 538e6136-8ef1-4a4b-ba29-3756617af561 on atsqa4-133.qa.lab:31010]
(state=,code=0)
{code}
was (Author: vicky):
{code}
0: jdbc:drill:schema=dfs> select cast(cast(a1 as int) as interval day) from
`t1.csv` where a1 = 1;
Error: SYSTEM ERROR: AssertionError: todo: implement syntax
SPECIAL(Reinterpret(*(Reinterpret(CAST(CAST($0):INTEGER):DECIMAL(2, 0)),
8640)))
[Error Id: 538e6136-8ef1-4a4b-ba29-3756617af561 on atsqa4-133.qa.lab:31010]
(state=,code=0)
{code}
> Unexpected exception in the query with an interval data type
>
>
> Key: DRILL-2859
> URL: https://issues.apache.org/jira/browse/DRILL-2859
> Project: Apache Drill
> Issue Type: Bug
> Components: Query Planning & Optimization
>Affects Versions: 0.9.0
>Reporter: Victoria Markman
> Fix For: Future
>
>
> {code}
> 0: jdbc:drill:schema=dfs> select cast(cast(a1 as int) as interval day) from
> t1 where a1 = 1;
> Query failed: SYSTEM ERROR: Unexpected exception during fragment
> initialization: todo: implement syntax
> SPECIAL(Reinterpret(*(Reinterpret(CAST(CAST($0):INTEGER):DECIMAL(2, 0)),
> 8640)))
> [5119315b-dd73-432f-ab93-49e76e9165f6 on atsqa4-134.qa.lab:31010]
> (org.apache.drill.exec.work.foreman.ForemanException) Unexpected exception
> during fragment initialization: todo: implement syntax
> SPECIAL(Reinterpret(*(Reinterpret(CAST(CAST($0):INTEGER):DECIMAL(2, 0)),
> 8640)))
> org.apache.drill.exec.work.foreman.Foreman.run():212
> java.util.concurrent.ThreadPoolExecutor.runWorker():1145
> java.util.concurrent.ThreadPoolExecutor$Worker.run():615
> java.lang.Thread.run():745
> Caused By (java.lang.AssertionError) todo: implement syntax
> SPECIAL(Reinterpret(*(Reinterpret(CAST(CAST($0):INTEGER):DECIMAL(2, 0)),
> 8640)))
>
> org.apache.drill.exec.planner.logical.DrillOptiq$RexToDrill.visitCall():182
> org.apache.drill.exec.planner.logical.DrillOptiq$RexToDrill.visitCall():73
> org.apache.calcite.rex.RexCall.accept():107
> org.apache.drill.exec.planner.logical.DrillOptiq.toDrill():70
>
> org.apache.drill.exec.planner.common.DrillProjectRelBase.getProjectExpressions():111
>
> org.apache.drill.exec.planner.physical.ProjectPrel.getPhysicalOperator():57
> org.apache.drill.exec.planner.physical.ScreenPrel.getPhysicalOperator():51
>
> org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.convertToPop():376
> org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan():157
> org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan():167
> org.apache.drill.exec.work.foreman.Foreman.runSQL():773
> org.apache.drill.exec.work.foreman.Foreman.run():203
> java.util.concurrent.ThreadPoolExecutor.runWorker():1145
> java.util.concurrent.ThreadPoolExecutor$Worker.run():615
> java.lang.Thread.run():745
> Error: exception while executing query: Failure while executing query.
> (state=,code=0)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (DRILL-2833) Exception on select * from INFORMATION_SCHEMA.COLUMNS caused by enabled HBase storage plug in where HBase is not installed
[
https://issues.apache.org/jira/browse/DRILL-2833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Victoria Markman updated DRILL-2833:
Summary: Exception on select * from INFORMATION_SCHEMA.COLUMNS caused by
enabled HBase storage plug in where HBase is not installed (was: Exception on
select * from INFORMATION_SCHEMA.COLUMNS)
> Exception on select * from INFORMATION_SCHEMA.COLUMNS caused by enabled HBase
> storage plug in where HBase is not installed
> --
>
> Key: DRILL-2833
> URL: https://issues.apache.org/jira/browse/DRILL-2833
> Project: Apache Drill
> Issue Type: Bug
> Components: Storage - HBase
>Affects Versions: 0.9.0
>Reporter: Victoria Markman
>Assignee: Aditya Kishore
>Priority: Critical
>
> #Sat Apr 18 21:26:53 EDT 2015
> git.commit.id.abbrev=9ec257e
> {code}
> 0: jdbc:drill:schema=dfs> select * from INFORMATION_SCHEMA.COLUMNS;
> Query failed: SYSTEM ERROR: null
> Fragment 0:0
> [bd2a0477-90ea-423b-ad77-ad9784f4116b on atsqa4-133.qa.lab:31010]
> (java.lang.NullPointerException) null
> org.eigenbase.sql.type.IntervalSqlType.():41
> org.eigenbase.sql.type.SqlTypeFactoryImpl.createSqlIntervalType():104
> org.apache.drill.exec.dotdrill.View.getRowType():236
> org.apache.drill.exec.planner.logical.DrillViewTable.getRowType():46
> org.apache.drill.exec.store.ischema.RecordGenerator.scanSchema():123
> org.apache.drill.exec.store.ischema.RecordGenerator.scanSchema():109
> org.apache.drill.exec.store.ischema.RecordGenerator.scanSchema():109
> org.apache.drill.exec.store.ischema.RecordGenerator.scanSchema():97
> org.apache.drill.exec.store.ischema.SelectedTable.getRecordReader():47
> org.apache.drill.exec.store.ischema.InfoSchemaBatchCreator.getBatch():35
> org.apache.drill.exec.store.ischema.InfoSchemaBatchCreator.getBatch():30
> org.apache.drill.exec.physical.impl.ImplCreator.visitOp():62
> org.apache.drill.exec.physical.impl.ImplCreator.visitOp():39
>
> org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitSubScan():127
> org.apache.drill.exec.physical.base.AbstractSubScan.accept():39
> org.apache.drill.exec.physical.impl.ImplCreator.getChildren():74
> org.apache.drill.exec.physical.impl.ImplCreator.visitOp():62
> org.apache.drill.exec.physical.impl.ImplCreator.visitOp():39
>
> org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitIteratorValidator():215
> org.apache.drill.exec.physical.config.IteratorValidator.accept():34
> org.apache.drill.exec.physical.impl.ImplCreator.getChildren():74
> org.apache.drill.exec.physical.impl.ImplCreator.visitOp():62
> org.apache.drill.exec.physical.impl.ImplCreator.visitOp():39
>
> org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitProject():77
> org.apache.drill.exec.physical.config.Project.accept():51
> org.apache.drill.exec.physical.impl.ImplCreator.getChildren():74
> org.apache.drill.exec.physical.impl.ImplCreator.visitOp():62
> org.apache.drill.exec.physical.impl.ImplCreator.visitOp():39
>
> org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitIteratorValidator():215
> org.apache.drill.exec.physical.config.IteratorValidator.accept():34
> org.apache.drill.exec.physical.impl.ImplCreator.getChildren():74
> org.apache.drill.exec.physical.impl.ImplCreator.visitOp():59
> org.apache.drill.exec.physical.impl.ImplCreator.visitOp():39
>
> org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitStore():132
>
> org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitScreen():195
> org.apache.drill.exec.physical.config.Screen.accept():97
> org.apache.drill.exec.physical.impl.ImplCreator.getExec():87
> org.apache.drill.exec.work.fragment.FragmentExecutor.run():148
> org.apache.drill.common.SelfCleaningRunnable.run():38
> java.util.concurrent.ThreadPoolExecutor.runWorker():1145
> java.util.concurrent.ThreadPoolExecutor$Worker.run():615
> java.lang.Thread.run():745
> Error: exception while executing query: Failure while executing query.
> (state=,code=0)
> {code}
> Only happens when Hbase plugin is enabled. Thanks Rahul for helping diagnose
> the problem !
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-2759) Improve error message when reading from an empty csv file
[
https://issues.apache.org/jira/browse/DRILL-2759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14949537#comment-14949537
]
Victoria Markman commented on DRILL-2759:
-
As of 1.2.0
I think expected result is 0 and not an empty set ... please correct me if I'm
wrong.
{code}
0: jdbc:drill:schema=dfs> select count(*) from `empty.csv`;
+--+
| |
+--+
+--+
No rows selected (0.245 seconds)
{code}
> Improve error message when reading from an empty csv file
> -
>
> Key: DRILL-2759
> URL: https://issues.apache.org/jira/browse/DRILL-2759
> Project: Apache Drill
> Issue Type: Bug
> Components: Storage - Text & CSV
>Affects Versions: 0.9.0
>Reporter: Victoria Markman
>Priority: Minor
> Fix For: Future
>
>
> t1.csv is an empty file:
> {code}
> 0: jdbc:drill:schema=dfs> select count(*) from `bigtable/2015/01/t1.csv`;
> Query failed: IllegalArgumentException: Incoming endpoints 1 is greater than
> number of row groups 0
> Error: exception while executing query: Failure while executing query.
> (state=,code=0)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (DRILL-2759) Improve error message when reading from an empty csv file
[
https://issues.apache.org/jira/browse/DRILL-2759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Victoria Markman updated DRILL-2759:
Priority: Critical (was: Minor)
> Improve error message when reading from an empty csv file
> -
>
> Key: DRILL-2759
> URL: https://issues.apache.org/jira/browse/DRILL-2759
> Project: Apache Drill
> Issue Type: Bug
> Components: Storage - Text & CSV
>Affects Versions: 0.9.0
>Reporter: Victoria Markman
>Priority: Critical
> Fix For: Future
>
>
> t1.csv is an empty file:
> {code}
> 0: jdbc:drill:schema=dfs> select count(*) from `bigtable/2015/01/t1.csv`;
> Query failed: IllegalArgumentException: Incoming endpoints 1 is greater than
> number of row groups 0
> Error: exception while executing query: Failure while executing query.
> (state=,code=0)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (DRILL-2543) Correlated subquery where outer table contains NULL values returns seemingly wrong result
[
https://issues.apache.org/jira/browse/DRILL-2543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Victoria Markman updated DRILL-2543:
Priority: Critical (was: Major)
> Correlated subquery where outer table contains NULL values returns seemingly
> wrong result
> --
>
> Key: DRILL-2543
> URL: https://issues.apache.org/jira/browse/DRILL-2543
> Project: Apache Drill
> Issue Type: Bug
> Components: Query Planning & Optimization
>Affects Versions: 0.8.0
>Reporter: Victoria Markman
>Priority: Critical
> Fix For: Future
>
>
> {code}
> 0: jdbc:drill:schema=dfs> select * from t1;
> ++++
> | a1 | b1 | c1 |
> ++++
> | 1 | 2015-03-01 | a |
> | 2 | 2015-03-02 | b |
> | null | null | null |
> ++++
> 3 rows selected (0.064 seconds)
> 0: jdbc:drill:schema=dfs> select * from t2;
> ++++
> | a2 | b2 | c2 |
> ++++
> | 5 | 2017-03-01 | a |
> ++++
> 1 row selected (0.07 seconds)
> 0: jdbc:drill:schema=dfs> select t1.c1, count(*) from t1 where t1.b1 not in
> (select b2 from t2 where t1.a1 = t2.a2) group by t1.c1 order by t1.c1;
> +++
> | c1 | EXPR$1 |
> +++
> | a | 1 |
> | b | 1 |
> +++
> 2 rows selected (0.32 seconds)
> {code}
> Postgres returns row from the outer table where a1 is null.
> This is part that I don't understand, because join condition in the subquery
> should have eliminated row where a1 IS NULL. To me Drill result looks
> correct. Unless there is something different in correlated comparison
> semantics that I'm not aware of.
> {code}
> postgres=# select * from t1;
> a1 | b1 | c1
> ++---
> 1 | 2015-03-01 | a
> 2 | 2015-03-02 | b
> ||
> (3 rows)
> {code}
> Explain plan for the query:
> {code}
> 00-01 Project(c1=[$0], EXPR$1=[$1])
> 00-02StreamAgg(group=[{0}], EXPR$1=[COUNT()])
> 00-03 Sort(sort0=[$0], dir0=[ASC])
> 00-04Project(c1=[$0])
> 00-05 SelectionVectorRemover
> 00-06Filter(condition=[NOT(IS TRUE($3))])
> 00-07 HashJoin(condition=[=($1, $2)], joinType=[left])
> 00-09Project($f1=[$0], $f3=[$2])
> 00-11 SelectionVectorRemover
> 00-13Filter(condition=[IS NOT NULL($1)])
> 00-15 Project(c1=[$1], b1=[$0], a1=[$2])
> 00-17Scan(groupscan=[ParquetGroupScan
> [entries=[ReadEntryWithPath [path=maprfs:/test/t1]], selectionRoot=/test/t1,
> numFiles=1, columns=[`c1`, `b1`, `a1`]]])
> 00-08Project($f02=[$1], $f2=[$2])
> 00-10 StreamAgg(group=[{0, 1}], agg#0=[MIN($2)])
> 00-12Sort(sort0=[$0], sort1=[$1], dir0=[ASC],
> dir1=[ASC])
> 00-14 Project($f0=[$1], $f02=[$2], $f1=[true])
> 00-16HashJoin(condition=[=($2, $0)],
> joinType=[inner])
> 00-18 StreamAgg(group=[{0}])
> 00-20Sort(sort0=[$0], dir0=[ASC])
> 00-22 Project($f0=[$1])
> 00-23SelectionVectorRemover
> 00-24 Filter(condition=[IS NOT NULL($0)])
> 00-25Scan(groupscan=[ParquetGroupScan
> [entries=[ReadEntryWithPath [path=maprfs:/test/t1]], selectionRoot=/test/t1,
> numFiles=1, columns=[`b1`, `a1`]]])
> 00-19 Project(a2=[$1], b2=[$0])
> 00-21Scan(groupscan=[ParquetGroupScan
> [entries=[ReadEntryWithPath [path=maprfs:/test/t2]], selectionRoot=/test/t2,
> numFiles=1, columns=[`a2`, `b2`]]])
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-2543) Correlated subquery where outer table contains NULL values returns seemingly wrong result
[
https://issues.apache.org/jira/browse/DRILL-2543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14949526#comment-14949526
]
Victoria Markman commented on DRILL-2543:
-
[~jni] it does make sense after you spelled it out with the standard. It is
technically wrong result, raising priority.
> Correlated subquery where outer table contains NULL values returns seemingly
> wrong result
> --
>
> Key: DRILL-2543
> URL: https://issues.apache.org/jira/browse/DRILL-2543
> Project: Apache Drill
> Issue Type: Bug
> Components: Query Planning & Optimization
>Affects Versions: 0.8.0
>Reporter: Victoria Markman
> Fix For: Future
>
>
> {code}
> 0: jdbc:drill:schema=dfs> select * from t1;
> ++++
> | a1 | b1 | c1 |
> ++++
> | 1 | 2015-03-01 | a |
> | 2 | 2015-03-02 | b |
> | null | null | null |
> ++++
> 3 rows selected (0.064 seconds)
> 0: jdbc:drill:schema=dfs> select * from t2;
> ++++
> | a2 | b2 | c2 |
> ++++
> | 5 | 2017-03-01 | a |
> ++++
> 1 row selected (0.07 seconds)
> 0: jdbc:drill:schema=dfs> select t1.c1, count(*) from t1 where t1.b1 not in
> (select b2 from t2 where t1.a1 = t2.a2) group by t1.c1 order by t1.c1;
> +++
> | c1 | EXPR$1 |
> +++
> | a | 1 |
> | b | 1 |
> +++
> 2 rows selected (0.32 seconds)
> {code}
> Postgres returns row from the outer table where a1 is null.
> This is part that I don't understand, because join condition in the subquery
> should have eliminated row where a1 IS NULL. To me Drill result looks
> correct. Unless there is something different in correlated comparison
> semantics that I'm not aware of.
> {code}
> postgres=# select * from t1;
> a1 | b1 | c1
> ++---
> 1 | 2015-03-01 | a
> 2 | 2015-03-02 | b
> ||
> (3 rows)
> {code}
> Explain plan for the query:
> {code}
> 00-01 Project(c1=[$0], EXPR$1=[$1])
> 00-02StreamAgg(group=[{0}], EXPR$1=[COUNT()])
> 00-03 Sort(sort0=[$0], dir0=[ASC])
> 00-04Project(c1=[$0])
> 00-05 SelectionVectorRemover
> 00-06Filter(condition=[NOT(IS TRUE($3))])
> 00-07 HashJoin(condition=[=($1, $2)], joinType=[left])
> 00-09Project($f1=[$0], $f3=[$2])
> 00-11 SelectionVectorRemover
> 00-13Filter(condition=[IS NOT NULL($1)])
> 00-15 Project(c1=[$1], b1=[$0], a1=[$2])
> 00-17Scan(groupscan=[ParquetGroupScan
> [entries=[ReadEntryWithPath [path=maprfs:/test/t1]], selectionRoot=/test/t1,
> numFiles=1, columns=[`c1`, `b1`, `a1`]]])
> 00-08Project($f02=[$1], $f2=[$2])
> 00-10 StreamAgg(group=[{0, 1}], agg#0=[MIN($2)])
> 00-12Sort(sort0=[$0], sort1=[$1], dir0=[ASC],
> dir1=[ASC])
> 00-14 Project($f0=[$1], $f02=[$2], $f1=[true])
> 00-16HashJoin(condition=[=($2, $0)],
> joinType=[inner])
> 00-18 StreamAgg(group=[{0}])
> 00-20Sort(sort0=[$0], dir0=[ASC])
> 00-22 Project($f0=[$1])
> 00-23SelectionVectorRemover
> 00-24 Filter(condition=[IS NOT NULL($0)])
> 00-25Scan(groupscan=[ParquetGroupScan
> [entries=[ReadEntryWithPath [path=maprfs:/test/t1]], selectionRoot=/test/t1,
> numFiles=1, columns=[`b1`, `a1`]]])
> 00-19 Project(a2=[$1], b2=[$0])
> 00-21Scan(groupscan=[ParquetGroupScan
> [entries=[ReadEntryWithPath [path=maprfs:/test/t2]], selectionRoot=/test/t2,
> numFiles=1, columns=[`a2`, `b2`]]])
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (DRILL-2355) TRUNC function returns incorrect result when second parameter is specified
[
https://issues.apache.org/jira/browse/DRILL-2355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Victoria Markman updated DRILL-2355:
Priority: Critical (was: Minor)
> TRUNC function returns incorrect result when second parameter is specified
> --
>
> Key: DRILL-2355
> URL: https://issues.apache.org/jira/browse/DRILL-2355
> Project: Apache Drill
> Issue Type: Bug
> Components: Functions - Drill
>Affects Versions: 0.8.0
>Reporter: Victoria Markman
>Priority: Critical
> Labels: document_if_not_fixed
> Fix For: Future
>
>
> I believe our TRUNC function is modeled on Postgres TRUNC function.
> Semandics of Postgres TRUNC function:
> {code}
> postgres=# select trunc(1234.1234);
> trunc
> ---
> 1234
> (1 row)
> postgres=# select trunc(1234.1234, 0);
> trunc
> ---
> 1234
> (1 row)
> postgres=# select trunc(1234.1234, 2);
> trunc
> -
> 1234.12
> (1 row)
> postgres=# select trunc(1234.1234, -1);
> trunc
> ---
> 1230
> (1 row)
> postgres=# select trunc(1234.1234, -3);
> trunc
> ---
> 1000
> (1 row)
> {code}
> This is incorrect, I can't truncate to zero decimal places:
> {code}
> 0: jdbc:drill:schema=dfs> select trunc(1234.1234) from sys.options limit 1;
> ++
> | EXPR$0 |
> ++
> | 1234.0 |
> ++
> 1 row selected (0.133 seconds)
> 0: jdbc:drill:schema=dfs> select trunc(1234.1234,0) from sys.options limit 1;
> ++
> | EXPR$0 |
> ++
> | 1234.0 |
> ++
> 1 row selected (0.065 seconds)
> {code}
> Second negative parameter, does not do what it is supposed to do as well:
> {code}
> 0: jdbc:drill:schema=dfs> select trunc(1234.1234, -1) from sys.options limit
> 1;
> ++
> | EXPR$0 |
> ++
> | 1230.0 |
> ++
> 1 row selected (0.068 seconds)
> 0: jdbc:drill:schema=dfs> select trunc(1234.1234, -3) from sys.options limit
> 1;
> ++
> | EXPR$0 |
> ++
> | 1000.0 |
> ++
> 1 row selected (0.072 seconds)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-2246) select from sys.options fails in a weird way when slice_target=1
[
https://issues.apache.org/jira/browse/DRILL-2246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14949421#comment-14949421
]
Victoria Markman commented on DRILL-2246:
-
Works in 1.2.0, no info on check in.
{code}
0: jdbc:drill:schema=dfs> alter system set `planner.slice_target` =1;
+---++
| ok |summary |
+---++
| true | planner.slice_target updated. |
+---++
1 row selected (0.244 seconds)
0: jdbc:drill:schema=dfs> select * from sys.options order by name;
++--+-+--+-+-+---++
|name| kind
| type | status | num_val | string_val | bool_val | float_val |
++--+-+--+-+-+---++
| drill.exec.functions.cast_empty_string_to_null | BOOLEAN
| SYSTEM | DEFAULT | null| null| false | null |
| drill.exec.storage.file.partition.column.label | STRING
| SYSTEM | DEFAULT | null| dir | null | null |
| exec.errors.verbose| BOOLEAN
| SYSTEM | DEFAULT | null| null| false | null |
| exec.java_compiler | STRING
| SYSTEM | DEFAULT | null| DEFAULT | null | null |
| exec.java_compiler_debug | BOOLEAN
| SYSTEM | DEFAULT | null| null| true | null |
| exec.java_compiler_janino_maxsize | LONG
| SYSTEM | DEFAULT | 262144 | null| null | null |
| exec.max_hash_table_size | LONG
| SYSTEM | DEFAULT | 1073741824 | null| null | null |
| exec.min_hash_table_size | LONG
| SYSTEM | DEFAULT | 65536 | null| null | null |
| exec.queue.enable | BOOLEAN
| SYSTEM | DEFAULT | null| null| false | null |
| exec.queue.large | LONG
| SYSTEM | DEFAULT | 10 | null| null | null |
| exec.queue.small | LONG
| SYSTEM | DEFAULT | 100 | null| null | null |
| exec.queue.threshold | LONG
| SYSTEM | DEFAULT | 3000| null| null | null |
| exec.queue.timeout_millis | LONG
| SYSTEM | DEFAULT | 30 | null| null | null |
| exec.schedule.assignment.old | BOOLEAN
| SYSTEM | DEFAULT | null| null| false | null |
| exec.storage.enable_new_text_reader| BOOLEAN
| SYSTEM | DEFAULT | null| null| true | null |
| new_view_default_permissions | STRING
| SYSTEM | DEFAULT | null| 700 | null | null |
| org.apache.drill.exec.compile.ClassTransformer.scalar_replacement | STRING
| SYSTEM | DEFAULT | null| try | null | null |
| planner.add_producer_consumer | BOOLEAN
| SYSTEM | DEFAULT | null| null| false | null |
| planner.affinity_factor| DOUBLE
| SYSTEM | DEFAULT | null| null| null | 1.2|
| planner.broadcast_factor | DOUBLE
| SYSTEM | DEFAULT | null| null| null | 1.0|
| planner.broadcast_threshold| LONG
| SYSTEM | DEFAULT | 1000| null| null | null |
| planner.disable_exchanges | BOOLEAN
| SYSTEM | DEFAULT | null| null| false | null |
| planner.enable_broadcast_join | BOOLEAN
| SYSTEM | DEFAULT | null| null| true | null |
| planner.enable_constant_folding| BOOLEAN
| SYSTEM | DEFAULT | null| null| true | null |
| planner.enable_decimal_data_type | BOO
[jira] [Resolved] (DRILL-2246) select from sys.options fails in a weird way when slice_target=1
[
https://issues.apache.org/jira/browse/DRILL-2246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Victoria Markman resolved DRILL-2246.
-
Resolution: Fixed
> select from sys.options fails in a weird way when slice_target=1
>
>
> Key: DRILL-2246
> URL: https://issues.apache.org/jira/browse/DRILL-2246
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Flow
>Reporter: Victoria Markman
>Priority: Minor
> Fix For: Future
>
>
> Since this is not supposed to be customer visible option, it's a minor issue.
> {code}
> alter system set `planner.slice_target` =1;
> select * from sys.options order by name;
> {code}
> {code}
> 0: jdbc:drill:schema=dfs> select * from sys.options order by name;
> Query failed: RemoteRpcException: Failure while trying to start remote
> fragment, No suitable constructor found for type [simple type, class
> org.apache.drill.exec.store.sys.SystemTablePlugin]: can not instantiate from
> JSON object (need to add/enable type information?)
> at [Source: {
> "pop" : "hash-partition-sender",
> "@id" : 0,
> "receiver-major-fragment" : 1,
> "child" : {
> "pop" : "sys",
> "@id" : 1,
> "table" : "OPTION",
> "plugin" : {
> "config" : {
> "type" : "SystemTablePluginConfig",
> "enabled" : true
> },
> "optimizerRules" : [ ]
> },
> "cost" : 20.0
> },
> "expr" : "hash(`name`) ",
> "destinations" : [ "ChFhdHNxYTQtMTMzLnFhLmxhYhCi8gEYo/IBIKTyAQ==",
> "ChFhdHNxYTQtMTM0LnFhLmxhYhCi8gEYo/IBIKTyAQ==",
> "ChFhdHNxYTQtMTMzLnFhLmxhYhCi8gEYo/IBIKTyAQ==",
> "ChFhdHNxYTQtMTM0LnFhLmxhYhCi8gEYo/IBIKTyAQ==",
> "ChFhdHNxYTQtMTMzLnFhLmxhYhCi8gEYo/IBIKTyAQ==",
> "ChFhdHNxYTQtMTM0LnFhLmxhYhCi8gEYo/IBIKTyAQ==",
> "ChFhdHNxYTQtMTMzLnFhLmxhYhCi8gEYo/IBIKTyAQ==",
> "ChFhdHNxYTQtMTM0LnFhLmxhYhCi8gEYo/IBIKTyAQ==",
> "ChFhdHNxYTQtMTMzLnFhLmxhYhCi8gEYo/IBIKTyAQ==",
> "ChFhdHNxYTQtMTM0LnFhLmxhYhCi8gEYo/IBIKTyAQ==",
> "ChFhdHNxYTQtMTMzLnFhLmxhYhCi8gEYo/IBIKTyAQ==",
> "ChFhdHNxYTQtMTM0LnFhLmxhYhCi8gEYo/IBIKTyAQ==",
> "ChFhdHNxYTQtMTMzLnFhLmxhYhCi8gEYo/IBIKTyAQ==",
> "ChFhdHNxYTQtMTM0LnFhLmxhYhCi8gEYo/IBIKTyAQ==",
> "ChFhdHNxYTQtMTMzLnFhLmxhYhCi8gEYo/IBIKTyAQ==",
> "ChFhdHNxYTQtMTM0LnFhLmxhYhCi8gEYo/IBIKTyAQ==",
> "ChFhdHNxYTQtMTMzLnFhLmxhYhCi8gEYo/IBIKTyAQ==",
> "ChFhdHNxYTQtMTM0LnFhLmxhYhCi8gEYo/IBIKTyAQ==",
> "ChFhdHNxYTQtMTMzLnFhLmxhYhCi8gEYo/IBIKTyAQ==",
> "ChFhdHNxYTQtMTM0LnFhLmxhYhCi8gEYo/IBIKTyAQ==" ],
> "initialAllocation" : 100,
> "maxAllocation" : 100,
> "cost" : 0.0
> }; line: 10, column: 7] [ 3082d365-3bc6-4aeb-bb29-ddd133832cff on
> atsqa4-134.qa.lab:31010 ]
> Error: exception while executing query: Failure while executing query.
> (state=,code=0)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-2235) Assert when NOT IN clause contains multiple columns
[
https://issues.apache.org/jira/browse/DRILL-2235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14949417#comment-14949417
]
Victoria Markman commented on DRILL-2235:
-
Here is the new runtime error without count(*):
{code}
0: jdbc:drill:schema=dfs> select * from t1 where (a1, b1) not in (select a2, b2
from t2);
Error: SYSTEM ERROR: SchemaChangeException: Failure while materializing
expression.
Error in expression at index -1. Error: Missing function implementation:
[count(INT-OPTIONAL, VARCHAR-OPTIONAL)]. Full expression: --UNKNOWN
EXPRESSION--.
Fragment 0:0
[Error Id: 10394832-b18f-4158-853a-23add6463318 on atsqa4-133.qa.lab:31010]
(state=,code=0)
{code}
> Assert when NOT IN clause contains multiple columns
> ---
>
> Key: DRILL-2235
> URL: https://issues.apache.org/jira/browse/DRILL-2235
> Project: Apache Drill
> Issue Type: Bug
> Components: Query Planning & Optimization
>Affects Versions: 0.8.0
>Reporter: Victoria Markman
> Fix For: Future
>
>
> {code}
> 0: jdbc:drill:schema=dfs> select * from t1;
> ++++
> | a1 | b1 | c1 |
> ++++
> | 1 | a | 2015-01-01 |
> | 2 | b | 2015-01-02 |
> | 3 | c | 2015-01-03 |
> | 4 | null | 2015-01-04 |
> | 5 | e | 2015-01-05 |
> | 6 | f | 2015-01-06 |
> | 7 | g | 2015-01-07 |
> | null | h | 2015-01-08 |
> | 9 | i | null |
> | 10 | j | 2015-01-10 |
> ++++
> 10 rows selected (0.056 seconds)
> 0: jdbc:drill:schema=dfs> select * from t2;
> ++++
> | a2 | b2 | c2 |
> ++++
> | 0 | zzz| 2014-12-31 |
> | 1 | a | 2015-01-01 |
> | 2 | b | 2015-01-02 |
> | 2 | b | 2015-01-02 |
> | 2 | b | 2015-01-02 |
> | 3 | c | 2015-01-03 |
> | 4 | d | 2015-01-04 |
> | 5 | e | 2015-01-05 |
> | 6 | f | 2015-01-06 |
> | 7 | g | 2015-01-07 |
> | 7 | g | 2015-01-07 |
> | 8 | h | 2015-01-08 |
> | 9 | i | 2015-01-09 |
> ++++
> 13 rows selected (0.069 seconds)
> {code}
> IN clause returns correct result:
> {code}
> 0: jdbc:drill:schema=dfs> select count(*) from t1 where (a1, b1) in (select
> a2, b2 from t2);
> ++
> | EXPR$0 |
> ++
> | 7 |
> ++
> 1 row selected (0.258 seconds)
> {code}
> NOT IN clause asserts:
> {code}
> 0: jdbc:drill:schema=dfs> select count(*) from t1 where (a1, b1) not in
> (select a2, b2 from t2);
> Query failed: AssertionError: AND(AND(NOT(IS TRUE($7)), IS NOT NULL($3)), IS
> NOT NULL($4))
> Error: exception while executing query: Failure while executing query.
> (state=,code=0)
> {code}
> {code}
> #Thu Feb 12 12:13:26 EST 2015
> git.commit.id.abbrev=de89f36
> {code}
> drillbit.log
> {code}
> 2015-02-12 22:47:11,730 [2b22d290-315e-4450-8b3f-9b3590eb20c3:foreman] INFO
> o.a.drill.exec.work.foreman.Foreman - State change requested. PENDING -->
> FAILED
> org.apache.drill.exec.work.foreman.ForemanException: Unexpected exception
> during fragment initialization: AND(AND(NOT(IS TRUE($7)), IS NOT NULL($3)),
> IS NOT NULL($4))
> at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:197)
> [drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.WorkManager$RunnableWrapper.run(WorkManager.java:303)
> [drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> [na:1.7.0_71]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> [na:1.7.0_71]
> at java.lang.Thread.run(Thread.java:745) [na:1.7.0_71]
> Caused by: java.lang.AssertionError: AND(AND(NOT(IS TRUE($7)), IS NOT
> NULL($3)), IS NOT NULL($4))
> at org.eigenbase.rel.FilterRelBase.(FilterRelBase.java:56)
> ~[optiq-core-0.9-drill-r18.jar:na]
> at org.eigenbase.rel.FilterRel.(FilterRel.java:50)
> ~[optiq-core-0.9-drill-r18.jar:na]
> at org.eigenbase.rel.CalcRel.createFilter(CalcRel.java:212)
> ~[optiq-core-0.9-drill-r18.jar:na]
> at
> org.eigenbase.sql2rel.SqlToRelConverter.convertWhere(SqlToRelConverter.java:840)
> ~[optiq-core-0.9-drill-r18.jar:na]
> at
> org.eigenbase.sql2rel.SqlToRelConverter.convertSelectImpl(SqlToRelConver
[jira] [Updated] (DRILL-2139) Star is not expanded correctly in "select distinct" query
[
https://issues.apache.org/jira/browse/DRILL-2139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Victoria Markman updated DRILL-2139:
Priority: Critical (was: Major)
> Star is not expanded correctly in "select distinct" query
> -
>
> Key: DRILL-2139
> URL: https://issues.apache.org/jira/browse/DRILL-2139
> Project: Apache Drill
> Issue Type: Bug
> Components: Query Planning & Optimization
>Affects Versions: 0.8.0
>Reporter: Victoria Markman
>Assignee: Sean Hsuan-Yi Chu
>Priority: Critical
> Fix For: 1.3.0
>
>
> {code}
> 0: jdbc:drill:schema=dfs> select distinct * from t1;
> ++
> | * |
> ++
> | null |
> ++
> 1 row selected (0.14 seconds)
> 0: jdbc:drill:schema=dfs> select distinct * from `test.json`;
> ++
> | * |
> ++
> | null |
> ++
> 1 row selected (0.163 seconds)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-2123) Order of columns in the Web UI is wrong when columns are explicitly specified in projection list
[
https://issues.apache.org/jira/browse/DRILL-2123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14949406#comment-14949406
]
Victoria Markman commented on DRILL-2123:
-
[~sudheeshkatkam] I just ran this query and it looks correct to me, but I don't
remember any check ins related to this area. Can you please check it out ?
> Order of columns in the Web UI is wrong when columns are explicitly specified
> in projection list
>
>
> Key: DRILL-2123
> URL: https://issues.apache.org/jira/browse/DRILL-2123
> Project: Apache Drill
> Issue Type: Bug
> Components: Client - HTTP
>Affects Versions: 0.8.0
>Reporter: Victoria Markman
> Fix For: Future
>
> Attachments: Screen Shot 2015-01-29 at 4.08.06 PM.png
>
>
> I'm running query:
> {code}
> select c_integer,
>c_bigint,
>nullif(c_integer, c_bigint)
> from `dfs.aggregation`.t1
> order by c_integer
> {code}
> In sqlline I get correct order of columns:
> {code}
> 0: jdbc:drill:schema=dfs> select c_integer, c_bigint, nullif(c_integer,
> c_bigint) from `dfs.aggregation`.t1;
> ++++
> | c_integer | c_bigint | EXPR$2 |
> ++++
> | 451237400 | -3477884857818808320 | 451237400 |
> {code}
> In Web UI - columns are sorted in alphabetical order.
> Screenshot is attached.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-1951) Can't cast numeric value with decimal point read from CSV file into integer data type
[
https://issues.apache.org/jira/browse/DRILL-1951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14949380#comment-14949380
]
Victoria Markman commented on DRILL-1951:
-
In 1.2.0 it throws a new error:
{code}
0: jdbc:drill:schema=dfs> select cast(columns[3] as bigint) from `sales.csv`;
Error: SYSTEM ERROR: NumberFormatException: 3000.00
Fragment 0:0
[Error Id: 2fdbf4a5-7d58-4473-a984-aee9e2c81a76 on atsqa4-133.qa.lab:31010]
(state=,code=0)
{code}
> Can't cast numeric value with decimal point read from CSV file into integer
> data type
> -
>
> Key: DRILL-1951
> URL: https://issues.apache.org/jira/browse/DRILL-1951
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Data Types
>Affects Versions: 0.8.0
>Reporter: Victoria Markman
> Fix For: Future
>
>
> sales.csv file:
> {code}
> 997,Ford,ME350,3000.00, comment#1
> 1999,Chevy,Venture,4900.00, comment#2
> 1999,Chevy,Venture,5000.00, comment#3
> 1996,Jeep,Cherokee,1.01, comment#4
> 0: jdbc:drill:schema=dfs> select cast(columns[3] as decimal(18,2)) from
> `sales.csv`;
> ++
> | EXPR$0 |
> ++
> | 3000.00|
> | 4900.00|
> | 5000.00|
> | 1.01 |
> ++
> 4 rows selected (0.093 seconds)
> {code}
> -- Can cast to decimal
> {code}
> 0: jdbc:drill:schema=dfs> select cast(columns[3] as decimal(18,2)) from
> `sales.csv`;
> ++
> | EXPR$0 |
> ++
> | 3000.00|
> | 4900.00|
> | 5000.00|
> | 1.01 |
> ++
> 4 rows selected (0.095 seconds)
> {code}
> -- Can cast to float
> {code}
> 0: jdbc:drill:schema=dfs> select cast(columns[3] as float) from `sales.csv`;
> ++
> | EXPR$0 |
> ++
> | 3000.0 |
> | 4900.0 |
> | 5000.0 |
> | 1.01 |
> ++
> 4 rows selected (0.112 seconds)
> {code}-- Can't cast to INT/BIGINT
> {code}
> 0: jdbc:drill:schema=dfs> select cast(columns[3] as bigint) from `sales.csv`;
> Query failed: Query failed: Failure while running fragment., 3000.00 [
> 4818451a-c731-48a9-9992-1e81ab1d520d on atsqa4-134.qa.lab:31010 ]
> [ 4818451a-c731-48a9-9992-1e81ab1d520d on atsqa4-134.qa.lab:31010 ]
> Error: exception while executing query: Failure while executing query.
> (state=,code=0)
> {code}
> -- Same works with json/parquet files
> {code}
> 0: jdbc:drill:schema=dfs> select a1 from `t1.json`;
> ++
> | a1 |
> ++
> | 10.01 |
> ++
> 1 row selected (0.077 seconds)
> 0: jdbc:drill:schema=dfs> select cast(a1 as int) from `t1.json`;
> ++
> | EXPR$0 |
> ++
> | 10 |
> ++
> 0: jdbc:drill:schema=dfs> select * from test_cast;
> ++
> | a1 |
> ++
> | 10.0100|
> ++
> 1 row selected (0.06 seconds)
> 0: jdbc:drill:schema=dfs> select cast(a1 as int) from test_cast;
> ++
> | EXPR$0 |
> ++
> | 10 |
> ++
> 1 row selected (0.094 seconds)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (DRILL-2101) Decimal literals are treated as double
[
https://issues.apache.org/jira/browse/DRILL-2101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Victoria Markman updated DRILL-2101:
Labels: decimal (was: )
> Decimal literals are treated as double
> --
>
> Key: DRILL-2101
> URL: https://issues.apache.org/jira/browse/DRILL-2101
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Data Types
>Affects Versions: 0.8.0
>Reporter: Victoria Markman
> Labels: decimal
> Fix For: Future
>
> Attachments: DRILL-2101-PARTIAL-PATCH-enable-decimal-literals.patch,
> DRILL-2101.patch
>
>
> {code}
> create table t1(c1) as
> select
> cast(null as decimal(28,4))
> from `t1.csv`;
> message root {
> optional double c1; <-- Wrong, should be decimal
> }
> {code}
> This is very commonly used construct to convert csv files to parquet files,
> that's why I'm marking this bug as critical.
> {code}
> create table t2 as
> select
> case when columns[3] = '' then cast(null as decimal(28,4)) else
> cast(columns[3] as decimal(28, 4)) end
> from `t1.csv`;
> {code}
> Correct - cast string literal to decimal
> {code}
> create table t3(c1) as
> select
> cast('12345678901234567890.1234' as decimal(28,4))
> from `t1.csv`;
> message root {
> required fixed_len_byte_array(12) c1 (DECIMAL(28,4));
> }
> {code}
> Correct - cast literal from csv file as decimal
> {code}
> create table t4(c1) as
> select
> cast(columns[3] as decimal(28,4))
> from `t1.csv`;
> message root {
> optional fixed_len_byte_array(12) c1 (DECIMAL(28,4));
> }
> {code}
> Correct - case statement (no null involved)
> {code}
> create table t5(c1) as
> select
> case when columns[3] = '' then cast('' as decimal(28,4)) else
> cast(columns[3] as decimal(28,4)) end
> from `t1.csv`;
> message root {
> optional fixed_len_byte_array(12) c1 (DECIMAL(28,4));
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (DRILL-2051) NPE when querying view with where clause and derived table
[
https://issues.apache.org/jira/browse/DRILL-2051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Victoria Markman resolved DRILL-2051.
-
Resolution: Fixed
> NPE when querying view with where clause and derived table
> --
>
> Key: DRILL-2051
> URL: https://issues.apache.org/jira/browse/DRILL-2051
> Project: Apache Drill
> Issue Type: Bug
> Components: Query Planning & Optimization
>Affects Versions: 0.8.0
>Reporter: Victoria Markman
> Fix For: Future
>
> Attachments: drill-2051.log
>
>
> {code}
> #Wed Jan 21 12:38:45 EST 2015
> git.commit.id.abbrev=8d1e1af
> {code}
> `customer.json`
> {code}
> { "CustomerId": "100", "cityId": 10 }
> { "CustomerId": "101", "cityId": 10 }
> { "CustomerId": "102", "cityId": 10 }
> { "CustomerId": "103", "cityId": 20 }
> { "CustomerId": "104", "cityId": 30 }
> { "CustomerId": "105", "cityId": null }
> { "CustomerId": null, "cityId": 50 }
> {code}
> {code}
> 0: jdbc:drill:schema=dfs> create view v3 as select * from ( select * from
> `customer.json` ) where customerid >= 100;
> +++
> | ok | summary |
> +++
> | true | View 'v3' created successfully in 'dfs.identifiers' schema |
> +++
> 1 row selected (0.063 seconds)
> 0: jdbc:drill:schema=dfs> select * from v3;
> Query failed: NullPointerException:
> Error: exception while executing query: Failure while executing query.
> (state=,code=0)
> {code}
> Query by itself works fine:
> {code}
> 0: jdbc:drill:schema=dfs> select * from ( select * from `customer.json` )
> where CustomerId >= 100;
> +++
> | CustomerId | cityId |
> +++
> | 100| 10 |
> | 101| 10 |
> | 102| 10 |
> | 103| 20 |
> | 104| 30 |
> | 105| null |
> +++
> 6 rows selected (0.117 seconds)
> {code}
> If you remove where clause and leave just derived table in the view creation,
> you can query the view as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-2051) NPE when querying view with where clause and derived table
[
https://issues.apache.org/jira/browse/DRILL-2051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14949371#comment-14949371
]
Victoria Markman commented on DRILL-2051:
-
This bug is fixed, can't point at check in the fixed it. Resolving.
Tested with 1.2.0:
{code}
0: jdbc:drill:schema=dfs> create or replace view v3 as select * from ( select *
from `customer.json` ) where customerid >= 100;
+---+--+
| ok | summary|
+---+--+
| true | View 'v3' created successfully in 'dfs.test' schema |
+---+--+
1 row selected (0.31 seconds)
0: jdbc:drill:schema=dfs> select * from v3;
+-+-+
| CustomerId | cityId |
+-+-+
| 100 | 10 |
| 101 | 10 |
| 102 | 10 |
| 103 | 20 |
| 104 | 30 |
| 105 | null|
+-+-+
6 rows selected (0.287 seconds)
{code}
> NPE when querying view with where clause and derived table
> --
>
> Key: DRILL-2051
> URL: https://issues.apache.org/jira/browse/DRILL-2051
> Project: Apache Drill
> Issue Type: Bug
> Components: Query Planning & Optimization
>Affects Versions: 0.8.0
>Reporter: Victoria Markman
> Fix For: Future
>
> Attachments: drill-2051.log
>
>
> {code}
> #Wed Jan 21 12:38:45 EST 2015
> git.commit.id.abbrev=8d1e1af
> {code}
> `customer.json`
> {code}
> { "CustomerId": "100", "cityId": 10 }
> { "CustomerId": "101", "cityId": 10 }
> { "CustomerId": "102", "cityId": 10 }
> { "CustomerId": "103", "cityId": 20 }
> { "CustomerId": "104", "cityId": 30 }
> { "CustomerId": "105", "cityId": null }
> { "CustomerId": null, "cityId": 50 }
> {code}
> {code}
> 0: jdbc:drill:schema=dfs> create view v3 as select * from ( select * from
> `customer.json` ) where customerid >= 100;
> +++
> | ok | summary |
> +++
> | true | View 'v3' created successfully in 'dfs.identifiers' schema |
> +++
> 1 row selected (0.063 seconds)
> 0: jdbc:drill:schema=dfs> select * from v3;
> Query failed: NullPointerException:
> Error: exception while executing query: Failure while executing query.
> (state=,code=0)
> {code}
> Query by itself works fine:
> {code}
> 0: jdbc:drill:schema=dfs> select * from ( select * from `customer.json` )
> where CustomerId >= 100;
> +++
> | CustomerId | cityId |
> +++
> | 100| 10 |
> | 101| 10 |
> | 102| 10 |
> | 103| 20 |
> | 104| 30 |
> | 105| null |
> +++
> 6 rows selected (0.117 seconds)
> {code}
> If you remove where clause and leave just derived table in the view creation,
> you can query the view as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-2015) Casting numeric value that does not fit in integer data type produces incorrect result
[
https://issues.apache.org/jira/browse/DRILL-2015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14949351#comment-14949351
]
Victoria Markman commented on DRILL-2015:
-
Changed priority to critical, since it is incorrect result.
> Casting numeric value that does not fit in integer data type produces
> incorrect result
> --
>
> Key: DRILL-2015
> URL: https://issues.apache.org/jira/browse/DRILL-2015
> Project: Apache Drill
> Issue Type: Improvement
> Components: Execution - Data Types
>Affects Versions: 0.8.0, 1.2.0
>Reporter: Victoria Markman
>Priority: Critical
> Labels: document_if_not_fixed
> Fix For: Future
>
>
> t1.json
> {code}
> { "a1": 1 ,"b1" : 1}
> { "a1": 2 ,"b1" : 1}
> { "a1": 2 ,"b1" : 2}
> { "a1": 3 ,"b1" : 2}
> { "a1": 5000147483647 , "b1" : 3}
> {code}
> We should throw an error, this is technically data corruption.
> {code}
> 0: jdbc:drill:schema=dfs> select cast(a1 as integer) from `t1.json`;
> ++
> | EXPR$0 |
> ++
> | 1 |
> | 2 |
> | 2 |
> | 3 |
> | 805551103 |
> ++
> 5 rows selected (0.074 seconds)
> {code}
> {code}
> 0: jdbc:drill:schema=dfs> select cast(2147483648 as integer) from `t1.json`;
> ++
> | EXPR$0 |
> ++
> | -2147483648 |
> | -2147483648 |
> | -2147483648 |
> | -2147483648 |
> | -2147483648 |
> ++
> 5 rows selected (0.076 seconds)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (DRILL-2015) Casting numeric value that does not fit in integer data type produces incorrect result
[
https://issues.apache.org/jira/browse/DRILL-2015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Victoria Markman updated DRILL-2015:
Affects Version/s: 1.2.0
> Casting numeric value that does not fit in integer data type produces
> incorrect result
> --
>
> Key: DRILL-2015
> URL: https://issues.apache.org/jira/browse/DRILL-2015
> Project: Apache Drill
> Issue Type: Improvement
> Components: Execution - Data Types
>Affects Versions: 0.8.0, 1.2.0
>Reporter: Victoria Markman
>Priority: Critical
> Labels: document_if_not_fixed
> Fix For: Future
>
>
> t1.json
> {code}
> { "a1": 1 ,"b1" : 1}
> { "a1": 2 ,"b1" : 1}
> { "a1": 2 ,"b1" : 2}
> { "a1": 3 ,"b1" : 2}
> { "a1": 5000147483647 , "b1" : 3}
> {code}
> We should throw an error, this is technically data corruption.
> {code}
> 0: jdbc:drill:schema=dfs> select cast(a1 as integer) from `t1.json`;
> ++
> | EXPR$0 |
> ++
> | 1 |
> | 2 |
> | 2 |
> | 3 |
> | 805551103 |
> ++
> 5 rows selected (0.074 seconds)
> {code}
> {code}
> 0: jdbc:drill:schema=dfs> select cast(2147483648 as integer) from `t1.json`;
> ++
> | EXPR$0 |
> ++
> | -2147483648 |
> | -2147483648 |
> | -2147483648 |
> | -2147483648 |
> | -2147483648 |
> ++
> 5 rows selected (0.076 seconds)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (DRILL-2015) Casting numeric value that does not fit in integer data type produces incorrect result
[
https://issues.apache.org/jira/browse/DRILL-2015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Victoria Markman updated DRILL-2015:
Priority: Critical (was: Major)
> Casting numeric value that does not fit in integer data type produces
> incorrect result
> --
>
> Key: DRILL-2015
> URL: https://issues.apache.org/jira/browse/DRILL-2015
> Project: Apache Drill
> Issue Type: Improvement
> Components: Execution - Data Types
>Affects Versions: 0.8.0
>Reporter: Victoria Markman
>Priority: Critical
> Labels: document_if_not_fixed
> Fix For: Future
>
>
> t1.json
> {code}
> { "a1": 1 ,"b1" : 1}
> { "a1": 2 ,"b1" : 1}
> { "a1": 2 ,"b1" : 2}
> { "a1": 3 ,"b1" : 2}
> { "a1": 5000147483647 , "b1" : 3}
> {code}
> We should throw an error, this is technically data corruption.
> {code}
> 0: jdbc:drill:schema=dfs> select cast(a1 as integer) from `t1.json`;
> ++
> | EXPR$0 |
> ++
> | 1 |
> | 2 |
> | 2 |
> | 3 |
> | 805551103 |
> ++
> 5 rows selected (0.074 seconds)
> {code}
> {code}
> 0: jdbc:drill:schema=dfs> select cast(2147483648 as integer) from `t1.json`;
> ++
> | EXPR$0 |
> ++
> | -2147483648 |
> | -2147483648 |
> | -2147483648 |
> | -2147483648 |
> | -2147483648 |
> ++
> 5 rows selected (0.076 seconds)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-2001) Poor error message when arithmetic expression with MIN/MAX functions on a string type
[
https://issues.apache.org/jira/browse/DRILL-2001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14949346#comment-14949346
]
Victoria Markman commented on DRILL-2001:
-
in 1.2.0 we return a better error, however it should be a USER error and not a
SYSTEM error:
{code}
0: jdbc:drill:schema=dfs> select min(a1)+1 from `t1.json`;
Error: SYSTEM ERROR: NumberFormatException: aaa
Fragment 0:0
[Error Id: 4623ffca-3b83-4d92-9c55-585fedb27789 on atsqa4-133.qa.lab:31010]
(state=,code=0)
{code}
> Poor error message when arithmetic expression with MIN/MAX functions on a
> string type
> -
>
> Key: DRILL-2001
> URL: https://issues.apache.org/jira/browse/DRILL-2001
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Data Types
>Affects Versions: 0.8.0
>Reporter: Victoria Markman
>Priority: Minor
> Fix For: 1.4.0
>
>
> We do support MIN/MAX().
> However, if I inadvertently were to use this in arithmetic expression, I
> would get an exception:
> {code}
> { "a1": "aaa" }
> { "a1": "bbb" }
> { "a1": "bbb" }
> { "a1": "eee" }
> {code}
> Works correctly:
> {code}
> 0: jdbc:drill:schema=dfs> select max(a1) from `t.json`;
> ++
> | EXPR$0 |
> ++
> | eee|
> ++
> 1 row selected (0.085 seconds)
> 0: jdbc:drill:schema=dfs> select min(a1) from `t.json`;
> ++
> | EXPR$0 |
> ++
> | aaa|
> ++
> 1 row selected (0.104 seconds)
> {code}
> Throws an exception:
> {code}
> 0: jdbc:drill:schema=dfs> select min(a1)+1 from `t.json`;
> ++
> | EXPR$0 |
> ++
> Query failed: Query failed: Failure while running fragment., aaa [
> 9194a73d-676d-4e63-8a49-a3c0ff0e63e0 on atsqa4-133.qa.lab:31010 ]
> [ 9194a73d-676d-4e63-8a49-a3c0ff0e63e0 on atsqa4-133.qa.lab:31010 ]
> java.lang.RuntimeException: java.sql.SQLException: Failure while executing
> query.
> at sqlline.SqlLine$IncrementalRows.hasNext(SqlLine.java:2514)
> at sqlline.SqlLine$TableOutputFormat.print(SqlLine.java:2148)
> at sqlline.SqlLine.print(SqlLine.java:1809)
> at sqlline.SqlLine$Commands.execute(SqlLine.java:3766)
> at sqlline.SqlLine$Commands.sql(SqlLine.java:3663)
> at sqlline.SqlLine.dispatch(SqlLine.java:889)
> at sqlline.SqlLine.begin(SqlLine.java:763)
> at sqlline.SqlLine.start(SqlLine.java:498)
> at sqlline.SqlLine.main(SqlLine.java:460)
> 0: jdbc:drill:schema=dfs> select max(a1)+1 from `t.json`;
> ++
> | EXPR$0 |
> ++
> Query failed: Query failed: Failure while running fragment., eee [
> 23d7d06d-b93b-4cb0-a35b-c0fa0faf3369 on atsqa4-133.qa.lab:31010 ]
> [ 23d7d06d-b93b-4cb0-a35b-c0fa0faf3369 on atsqa4-133.qa.lab:31010 ]
> java.lang.RuntimeException: java.sql.SQLException: Failure while executing
> query.
> at sqlline.SqlLine$IncrementalRows.hasNext(SqlLine.java:2514)
> at sqlline.SqlLine$TableOutputFormat.print(SqlLine.java:2148)
> at sqlline.SqlLine.print(SqlLine.java:1809)
> at sqlline.SqlLine$Commands.execute(SqlLine.java:3766)
> at sqlline.SqlLine$Commands.sql(SqlLine.java:3663)
> at sqlline.SqlLine.dispatch(SqlLine.java:889)
> at sqlline.SqlLine.begin(SqlLine.java:763)
> at sqlline.SqlLine.start(SqlLine.java:498)
> at sqlline.SqlLine.main(SqlLine.java:460)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (DRILL-1998) Confusing error message when you pass character data type to SUM aggregate function
[
https://issues.apache.org/jira/browse/DRILL-1998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Victoria Markman resolved DRILL-1998.
-
Resolution: Fixed
> Confusing error message when you pass character data type to SUM aggregate
> function
> ---
>
> Key: DRILL-1998
> URL: https://issues.apache.org/jira/browse/DRILL-1998
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Data Types
>Affects Versions: 0.8.0
>Reporter: Victoria Markman
>Priority: Minor
> Fix For: Future
>
>
> t.json
> {code}
> { "a2": 1, "b2": 1,"c2": 1, "d2": "aaa", e1: "2015-01-01"}
> { "a2": 2, "b2": 2,"c2": 2, "d2": "bbb", e1: "2015-01-02"}
> { "a2": 2, "b2": 2,"c2": 2, "d2": "bbb", e1: "2015-01-02"}
> { "a2": 5, "b2": 5,"c2": 5, "d2": "eee", e1: "2015-01-05"}
> {code}
> {code}
> 0: jdbc:drill:schema=dfs> select sum(d2) from `t2.json`;
> Query failed: Query failed: Failure while running fragment., Only COUNT
> aggregate function supported for Boolean type [
> 003f011e-ae98-47ac-99c8-1674b1edb74b on atsqa4-133.qa.lab:31010 ]
> [ 003f011e-ae98-47ac-99c8-1674b1edb74b on atsqa4-133.qa.lab:31010 ]
> Error: exception while executing query: Failure while executing query.
> (state=,code=0)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-1998) Confusing error message when you pass character data type to SUM aggregate function
[
https://issues.apache.org/jira/browse/DRILL-1998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14949337#comment-14949337
]
Victoria Markman commented on DRILL-1998:
-
This bug is fixed in version 1.1.0, see comments in:
https://issues.apache.org/jira/browse/DRILL-3245
In 1.2.0
{code}
0: jdbc:drill:schema=dfs> select sum(d2) from `t1.json`;
Error: UNSUPPORTED_OPERATION ERROR: Only COUNT, MIN and MAX aggregate functions
supported for VarChar type
Fragment 0:0
[Error Id: 405da0ff-f3c9-4617-a463-76cade51aefe on atsqa4-133.qa.lab:31010]
(state=,code=0)
{code}
> Confusing error message when you pass character data type to SUM aggregate
> function
> ---
>
> Key: DRILL-1998
> URL: https://issues.apache.org/jira/browse/DRILL-1998
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Data Types
>Affects Versions: 0.8.0
>Reporter: Victoria Markman
>Priority: Minor
> Fix For: Future
>
>
> t.json
> {code}
> { "a2": 1, "b2": 1,"c2": 1, "d2": "aaa", e1: "2015-01-01"}
> { "a2": 2, "b2": 2,"c2": 2, "d2": "bbb", e1: "2015-01-02"}
> { "a2": 2, "b2": 2,"c2": 2, "d2": "bbb", e1: "2015-01-02"}
> { "a2": 5, "b2": 5,"c2": 5, "d2": "eee", e1: "2015-01-05"}
> {code}
> {code}
> 0: jdbc:drill:schema=dfs> select sum(d2) from `t2.json`;
> Query failed: Query failed: Failure while running fragment., Only COUNT
> aggregate function supported for Boolean type [
> 003f011e-ae98-47ac-99c8-1674b1edb74b on atsqa4-133.qa.lab:31010 ]
> [ 003f011e-ae98-47ac-99c8-1674b1edb74b on atsqa4-133.qa.lab:31010 ]
> Error: exception while executing query: Failure while executing query.
> (state=,code=0)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (DRILL-1868) Provide options to handle nonexistent columns in schema-less query
[
https://issues.apache.org/jira/browse/DRILL-1868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Victoria Markman updated DRILL-1868:
Summary: Provide options to handle nonexistent columns in schema-less query
(was: Filtering on an alias should return an error, user gets wrong result
instead)
> Provide options to handle nonexistent columns in schema-less query
> --
>
> Key: DRILL-1868
> URL: https://issues.apache.org/jira/browse/DRILL-1868
> Project: Apache Drill
> Issue Type: Improvement
> Components: SQL Parser
>Reporter: Victoria Markman
> Labels: document_if_not_fixed
> Fix For: Future
>
>
> git.commit.id.abbrev=c65928f
> {code}
> 0: jdbc:drill:schema=dfs> select * from `test.json`;
> +++
> | eventdate |sold|
> +++
> | 2014-01-01 | 100|
> | 2014-01-01 | 100|
> | 2014-02-01 | 200|
> +++
> 3 rows selected (0.099 seconds)
> {code}
>
> {code}
> 0: jdbc:drill:schema=dfs> -- Correct result
> 0: jdbc:drill:schema=dfs> SELECT
> . . . . . . . . . . . . > extract( month from eventdate ) as
> `month`,
> . . . . . . . . . . . . > extract( year from eventdate ) as
> `year`
> . . . . . . . . . . . . >
> . . . . . . . . . . . . > FROM`test.json`
> . . . . . . . . . . . . > WHERE extract( month from eventdate ) IS
> NOT NULL;
> +++
> | month|year|
> +++
> | 1 | 2014 |
> | 1 | 2014 |
> | 2 | 2014 |
> +++
> 3 rows selected (0.074 seconds)
> {code}
> {code}
> 0: jdbc:drill:schema=dfs> -- Wrong result, should throw an error
> 0: jdbc:drill:schema=dfs> SELECT
> . . . . . . . . . . . . > extract( month from eventdate ) as
> `month`,
> . . . . . . . . . . . . > extract( year from eventdate ) as
> `year`
> . . . . . . . . . . . . >
> . . . . . . . . . . . . > FROM`test.json`
> . . . . . . . . . . . . > WHERE `month` IS NOT NULL;
> +++
> | month|year|
> +++
> +++
> No rows selected (0.079 seconds)
> {code}
> {code}
> 0: jdbc:drill:schema=dfs> -- Wrong result, should throw an error
> 0: jdbc:drill:schema=dfs> SELECT
> . . . . . . . . . . . . > extract( month from eventdate ) as
> xyz,
> . . . . . . . . . . . . > extract( year from eventdate ) as
> `year`
> . . . . . . . . . . . . >
> . . . . . . . . . . . . > FROM`test.json`
> . . . . . . . . . . . . > WHERE xyz IS NOT NULL;
> +++
> |xyz |year|
> +++
> +++
> No rows selected (0.073 seconds)
> {code}
> {code}
> 0: jdbc:drill:schema=dfs> -- Correct result
> 0: jdbc:drill:schema=dfs> SELECT *
> . . . . . . . . . . . . > FROM
> . . . . . . . . . . . . > (
> . . . . . . . . . . . . > SELECT
> . . . . . . . . . . . . > extract( month from eventdate ) as
> `month`,
> . . . . . . . . . . . . > extract( year from eventdate ) as
> `year`
> . . . . . . . . . . . . >
> . . . . . . . . . . . . > FROM`test.json`
> . . . . . . . . . . . . > WHERE `month` IS NULL
> . . . . . . . . . . . . > )
> . . . . . . . . . . . . > WHERE `month` IS NOT NULL;
> +++
> | month|year|
> +++
> | 1 | 2014 |
> | 1 | 2014 |
> | 2 | 2014 |
> +++
> 3 rows selected (0.099 seconds)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-1895) Improve message where implicit cast fails (e.g. with IN clause)
[
https://issues.apache.org/jira/browse/DRILL-1895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14949304#comment-14949304
]
Victoria Markman commented on DRILL-1895:
-
Tried with 1.2.0, error is much more descriptive now. However I don't think it
should be SYSTEM error, but rather USER error:
{code}
0: jdbc:drill:schema=dfs> select * from cp.`tpch/nation.parquet` where
n_regionkey in ('abc');
Error: SYSTEM ERROR: NumberFormatException: abc
Fragment 0:0
[Error Id: 8f70a9f3-ab15-4c72-8887-14b05ad57752 on atsqa4-133.qa.lab:31010]
(state=,code=0)
{code}
> Improve message where implicit cast fails (e.g. with IN clause)
> ---
>
> Key: DRILL-1895
> URL: https://issues.apache.org/jira/browse/DRILL-1895
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Data Types
>Affects Versions: 0.7.0
>Reporter: Victoria Markman
>Priority: Minor
> Fix For: Future
>
>
> -- Works, because value in "IN CLAUSE" is a compatible numeric type
> {code}
> 0: jdbc:drill:schema=dfs> select * from cp.`tpch/nation.parquet` where
> n_regionkey in (4);
> +-++-++
> | n_nationkey | n_name | n_regionkey | n_comment |
> +-++-++
> | 4 | EGYPT | 4 | y above the carefully unusual
> theodolites. final dugouts are quickly across the furiously regular d |
> | 10 | IRAN | 4 | efully alongside of the slyly
> final dependencies. |
> | 11 | IRAQ | 4 | nic deposits boost atop the
> quickly final requests? quickly regula |
> | 13 | JORDAN | 4 | ic deposits are blithely about the
> carefully regular pa |
> | 20 | SAUDI ARABIA | 4 | ts. silent requests haggle.
> closely express packages sleep across the blithely |
> +-++-++
> 5 rows selected (0.092 seconds)
> {code}
> -- WORKS (trying to convert literal string to numeric and succeeds, because
> it can be implicitly converted)
> {code}
> 0: jdbc:drill:schema=dfs> select * from cp.`tpch/nation.parquet` where
> n_regionkey in ('4');
> +-++-++
> | n_nationkey | n_name | n_regionkey | n_comment |
> +-++-++
> | 4 | EGYPT | 4 | y above the carefully unusual
> theodolites. final dugouts are quickly across the furiously regular d |
> | 10 | IRAN | 4 | efully alongside of the slyly
> final dependencies. |
> | 11 | IRAQ | 4 | nic deposits boost atop the
> quickly final requests? quickly regula |
> | 13 | JORDAN | 4 | ic deposits are blithely about the
> carefully regular pa |
> | 20 | SAUDI ARABIA | 4 | ts. silent requests haggle.
> closely express packages sleep across the blithely |
> +-++-++
> 5 rows selected (0.073 seconds)
> {code}
> -- FAILS (can't be converted to numeric type)
> {code}
> 0: jdbc:drill:schema=dfs> select * from cp.`tpch/nation.parquet` where
> n_regionkey in ('abc');
> Query failed: Query failed: Failure while running fragment., abc [
> 4578a64c-75c5-4acf-be8c-28ce0db8623d on atsqa4-133.qa.lab:31010 ]
> [ 4578a64c-75c5-4acf-be8c-28ce0db8623d on atsqa4-133.qa.lab:31010 ]
> Error: exception while executing query: Failure while executing query.
> (state=,code=0)
> 0: jdbc:drill:schema=dfs> select * from cp.`tpch/nation.parquet` where
> n_regionkey in ('4');
> +-++-++
> | n_nationkey | n_name | n_regionkey | n_comment |
> +-++-++
> | 4 | EGYPT | 4 | y above the carefully unusual
> theodolites. final dugouts are quickly across the furiously regular d |
> | 10 | IRAN | 4 | efully alongside of the slyly
> final dependencies. |
> | 11 | IRAQ | 4 | nic deposits boost atop the
> quickly final requests? quickly regula |
> | 13 | JORDAN | 4 | ic deposits are blithely about the
> carefully regular pa |
> | 20 | SAUDI ARABIA | 4 | ts. silent requests haggle.
> closely express packages sleep across the blithely |
> +-++-++
> 5 rows selected (0.073 seconds)
> {code}
> It would be really neat to get a descriptive error message.
> Postges example:
> {code}
> postgres=# select * from t1 where c1 in ('abc');
> ERROR: invalid input syntax for integer: "abc"
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (DRILL-3913) Possible memory leak during CTAS using 30 TB TPC-H dataset
[ https://issues.apache.org/jira/browse/DRILL-3913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Victoria Markman updated DRILL-3913: Priority: Critical (was: Major) > Possible memory leak during CTAS using 30 TB TPC-H dataset > -- > > Key: DRILL-3913 > URL: https://issues.apache.org/jira/browse/DRILL-3913 > Project: Apache Drill > Issue Type: Bug > Components: Execution - Flow >Affects Versions: 1.2.0 > Environment: 47 nodes configured with 32 GB Drill Direct memory >Reporter: Abhishek Girish >Priority: Critical > Attachments: create_table_sf3.txt, query_profile.json, > sys.memory.txt > > > 8 CTAS queries were executed sequentially to write TPC-H text data into > Parquet. After successfully writing a few tables, CTAS failed with OOM. > Restarting Drillbits fixed the problem and re-run of pending CTAS queries > completed. This process was done twice in-order to complete all 8 tables to > be written. Overall source was 30TB in size. > Queries attached. Query profile for one of the CTAS which failed is attached. > Logs indicated that the Drillbit was out of Direct Memory. > Can share more details as required. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-3913) Possible memory leak during CTAS using 30 TB TPC-H dataset
[ https://issues.apache.org/jira/browse/DRILL-3913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14947825#comment-14947825 ] Victoria Markman commented on DRILL-3913: - [~agirish] 1. Can you please run one single CTAS query in a loop and see if it fails with OOM at some point ? Simple script is attached in: https://issues.apache.org/jira/browse/DRILL-2865 2. I believe reproduction can be scaled down: there is no need to run with 30TB of data 3. It is also possible, that you are running into an issue with memory fragmentation that [~adeneche] and I were trying to get to the bottom of couple of weeks ago, but got distracted by 1.2 release and never finished. I think this is very important to understand what the source of the problem is in this case, raising priority to critical. > Possible memory leak during CTAS using 30 TB TPC-H dataset > -- > > Key: DRILL-3913 > URL: https://issues.apache.org/jira/browse/DRILL-3913 > Project: Apache Drill > Issue Type: Bug > Components: Execution - Flow >Affects Versions: 1.2.0 > Environment: 47 nodes configured with 32 GB Drill Direct memory >Reporter: Abhishek Girish > Attachments: create_table_sf3.txt, query_profile.json, > sys.memory.txt > > > 8 CTAS queries were executed sequentially to write TPC-H text data into > Parquet. After successfully writing a few tables, CTAS failed with OOM. > Restarting Drillbits fixed the problem and re-run of pending CTAS queries > completed. This process was done twice in-order to complete all 8 tables to > be written. Overall source was 30TB in size. > Queries attached. Query profile for one of the CTAS which failed is attached. > Logs indicated that the Drillbit was out of Direct Memory. > Can share more details as required. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (DRILL-3732) Drill leaks memory if external sort hits out of disk space exception
[
https://issues.apache.org/jira/browse/DRILL-3732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Victoria Markman closed DRILL-3732.
---
> Drill leaks memory if external sort hits out of disk space exception
>
>
> Key: DRILL-3732
> URL: https://issues.apache.org/jira/browse/DRILL-3732
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Relational Operators
>Affects Versions: 1.2.0
>Reporter: Victoria Markman
>Assignee: Victoria Markman
>Priority: Critical
> Fix For: 1.2.0
>
> Attachments: drillbit.log
>
>
> Ran into it when running CTAS with partition by.
> Here is what reproduction looks like:
> {code}
> 0: jdbc:drill:schema=dfs> create table store_sales_4(ss_item_sk,
> ss_customer_sk, ss_cdemo_sk, ss_hdemo_sk, s_sold_date_sk, ss_promo_sk)
> partition by (ss_promo_sk) as
> . . . . . . . . . . . . > select
> . . . . . . . . . . . . > case when columns[2] = '' then cast(null as
> varchar(100)) else cast(columns[2] as varchar(100)) end,
> . . . . . . . . . . . . > case when columns[3] = '' then cast(null as
> varchar(100)) else cast(columns[3] as varchar(100)) end,
> . . . . . . . . . . . . > case when columns[4] = '' then cast(null as
> varchar(100)) else cast(columns[4] as varchar(100)) end,
> . . . . . . . . . . . . > case when columns[5] = '' then cast(null as
> varchar(100)) else cast(columns[5] as varchar(100)) end,
> . . . . . . . . . . . . > case when columns[0] = '' then cast(null as
> varchar(100)) else cast(columns[0] as varchar(100)) end,
> . . . . . . . . . . . . > case when columns[8] = '' then cast(null as
> varchar(100)) else cast(columns[8] as varchar(100)) end
> . . . . . . . . . . . . > from
> . . . . . . . . . . . . > `store_sales.dat` ss
> . . . . . . . . . . . . > ;
> Error: SYSTEM ERROR: IllegalStateException: Failure while closing accountor.
> Expected private and shared pools to be set to initial values. However, one
> or more were not. Stats are
> zoneinitallocated delta
> private 10009680512 319488
> shared 100010000.
> Fragment 1:21
> [Error Id: bd0d7d59-8693-476b-8671-70f0b2e7a176 on atsqa4-133.qa.lab:31010]
> (state=,code=0)
> {code}
> Setup:
> single node
> 8GB direct memory
> 4GB heap memory
>store_sales.dat is a file from TPCDS SF100
> drillbit.log attached
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-3732) Drill leaks memory if external sort hits out of disk space exception
[
https://issues.apache.org/jira/browse/DRILL-3732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14945442#comment-14945442
]
Victoria Markman commented on DRILL-3732:
-
Memory leak is gone now, something still not right with handling of out of
disk space error though. I get NPE.
After query failed with NPE, memory usage was OK.
NPE bug is filed here: https://issues.apache.org/jira/browse/DRILL-3898
#Mon Oct 05 19:59:48 UTC 2015
git.commit.id.abbrev=27a5abc
> Drill leaks memory if external sort hits out of disk space exception
>
>
> Key: DRILL-3732
> URL: https://issues.apache.org/jira/browse/DRILL-3732
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Relational Operators
>Affects Versions: 1.2.0
>Reporter: Victoria Markman
>Assignee: Victoria Markman
>Priority: Critical
> Fix For: 1.2.0
>
> Attachments: drillbit.log
>
>
> Ran into it when running CTAS with partition by.
> Here is what reproduction looks like:
> {code}
> 0: jdbc:drill:schema=dfs> create table store_sales_4(ss_item_sk,
> ss_customer_sk, ss_cdemo_sk, ss_hdemo_sk, s_sold_date_sk, ss_promo_sk)
> partition by (ss_promo_sk) as
> . . . . . . . . . . . . > select
> . . . . . . . . . . . . > case when columns[2] = '' then cast(null as
> varchar(100)) else cast(columns[2] as varchar(100)) end,
> . . . . . . . . . . . . > case when columns[3] = '' then cast(null as
> varchar(100)) else cast(columns[3] as varchar(100)) end,
> . . . . . . . . . . . . > case when columns[4] = '' then cast(null as
> varchar(100)) else cast(columns[4] as varchar(100)) end,
> . . . . . . . . . . . . > case when columns[5] = '' then cast(null as
> varchar(100)) else cast(columns[5] as varchar(100)) end,
> . . . . . . . . . . . . > case when columns[0] = '' then cast(null as
> varchar(100)) else cast(columns[0] as varchar(100)) end,
> . . . . . . . . . . . . > case when columns[8] = '' then cast(null as
> varchar(100)) else cast(columns[8] as varchar(100)) end
> . . . . . . . . . . . . > from
> . . . . . . . . . . . . > `store_sales.dat` ss
> . . . . . . . . . . . . > ;
> Error: SYSTEM ERROR: IllegalStateException: Failure while closing accountor.
> Expected private and shared pools to be set to initial values. However, one
> or more were not. Stats are
> zoneinitallocated delta
> private 10009680512 319488
> shared 100010000.
> Fragment 1:21
> [Error Id: bd0d7d59-8693-476b-8671-70f0b2e7a176 on atsqa4-133.qa.lab:31010]
> (state=,code=0)
> {code}
> Setup:
> single node
> 8GB direct memory
> 4GB heap memory
>store_sales.dat is a file from TPCDS SF100
> drillbit.log attached
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
