[ 
https://issues.apache.org/jira/browse/CALCITE-6429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated CALCITE-6429:
------------------------------------
    Labels: pull-request-available  (was: )

> Arrow adapter should default to the Enumerable convention for unsupported 
> filters
> ---------------------------------------------------------------------------------
>
>                 Key: CALCITE-6429
>                 URL: https://issues.apache.org/jira/browse/CALCITE-6429
>             Project: Calcite
>          Issue Type: Sub-task
>          Components: arrow-adapter
>    Affects Versions: 1.37.0
>            Reporter: Alessandro Solimando
>            Assignee: Alessandro Solimando
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 1.38.0
>
>
> Currently, the adapter fails with different errors when one of the known 
> unsupported features is used, while it might simply catch the 
> "UnsupportedOperationException" and bail out, thus allowing to generate a 
> plan where the unsupported operation is implemented in the Enumerable 
> convention.
>  
> For instance, currently several tests are disabled for this reason, like 
> [testArrowProjectFieldsWithDisjunctiveFilter|https://github.com/apache/calcite/blob/be9b860ebf3143dcbdbd0ee1777e604f0ace7b3c/arrow/src/test/java/org/apache/calcite/adapter/arrow/ArrowAdapterTest.java#L242],
>  because it would fail as follows:
> {code:java}
> Error while executing SQL "select "intField", "stringField"
> from arrowdata
> where "intField"=12 or "stringField"='12'": Error while applying rule 
> ArrowFilterRule, args 
> [rel#31:LogicalFilter.NONE.[](input=RelSubset#15,condition=OR(=($0, 12), 
> =($1, '12'))), rel#1:ArrowTableScan.ARROW.[](table=[ARROW, 
> ARROWDATA],fields=[0, 1, 2, 3])]
> java.sql.SQLException: Error while executing SQL "select "intField", 
> "stringField"
> from arrowdata
> where "intField"=12 or "stringField"='12'": Error while applying rule 
> ArrowFilterRule, args 
> [rel#31:LogicalFilter.NONE.[](input=RelSubset#15,condition=OR(=($0, 12), 
> =($1, '12'))), rel#1:ArrowTableScan.ARROW.[](table=[ARROW, 
> ARROWDATA],fields=[0, 1, 2, 3])]
>     at org.apache.calcite.avatica.Helper.createException(Helper.java:56)
>     at org.apache.calcite.avatica.Helper.createException(Helper.java:41)
>     at 
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:164)
>     at 
> org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:228)
>     at 
> org.apache.calcite.test.CalciteAssert.assertQuery(CalciteAssert.java:566)
>     at 
> org.apache.calcite.test.CalciteAssert$AssertQuery.lambda$returns$1(CalciteAssert.java:1495)
>     at 
> org.apache.calcite.test.CalciteAssert$AssertQuery.withConnection(CalciteAssert.java:1434)
>     at 
> org.apache.calcite.test.CalciteAssert$AssertQuery.returns(CalciteAssert.java:1493)
>     at 
> org.apache.calcite.test.CalciteAssert$AssertQuery.returns(CalciteAssert.java:1483)
>     at 
> org.apache.calcite.test.CalciteAssert$AssertQuery.returns(CalciteAssert.java:1446)
>     at 
> org.apache.calcite.adapter.arrow.ArrowAdapterTest.testArrowProjectFieldsWithDisjunctiveFilter(ArrowAdapterTest.java:260)
>     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>     at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>     at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>     at java.lang.reflect.Method.invoke(Method.java:498)
>     at 
> org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:727)
>     at 
> org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
>     at 
> org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
>     at 
> org.junit.jupiter.engine.extension.SameThreadTimeoutInvocation.proceed(SameThreadTimeoutInvocation.java:45)
>     at 
> org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:156)
>     at 
> org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:147)
>     at 
> org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:86)
>     at 
> org.junit.jupiter.engine.execution.InterceptingExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(InterceptingExecutableInvoker.java:103)
>     at 
> org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.lambda$invoke$0(InterceptingExecutableInvoker.java:93)
>     at 
> org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
>     at 
> org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
>     at 
> org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
>     at 
> org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
>     at 
> org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:92)
>     at 
> org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:86)
>     at 
> org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$7(TestMethodTestDescriptor.java:217)
>     at 
> org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
>     at 
> org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:213)
>     at 
> org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:138)
>     at 
> org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:68)
>     at 
> org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:151)
>     at 
> org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
>     at 
> org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
>     at 
> org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
>     at 
> org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
>     at 
> org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
>     at 
> org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
>     at 
> org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
>     at 
> org.junit.platform.engine.support.hierarchical.ForkJoinPoolHierarchicalTestExecutorService$ExclusiveTask.compute(ForkJoinPoolHierarchicalTestExecutorService.java:185)
>     at 
> org.junit.platform.engine.support.hierarchical.ForkJoinPoolHierarchicalTestExecutorService.invokeAll(ForkJoinPoolHierarchicalTestExecutorService.java:129)
>     at 
> org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155)
>     at 
> org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
>     at 
> org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
>     at 
> org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
>     at 
> org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
>     at 
> org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
>     at 
> org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
>     at 
> org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
>     at 
> org.junit.platform.engine.support.hierarchical.ForkJoinPoolHierarchicalTestExecutorService$ExclusiveTask.compute(ForkJoinPoolHierarchicalTestExecutorService.java:185)
>     at 
> org.junit.platform.engine.support.hierarchical.ForkJoinPoolHierarchicalTestExecutorService.invokeAll(ForkJoinPoolHierarchicalTestExecutorService.java:129)
>     at 
> org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155)
>     at 
> org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
>     at 
> org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
>     at 
> org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
>     at 
> org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
>     at 
> org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
>     at 
> org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
>     at 
> org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
>     at 
> org.junit.platform.engine.support.hierarchical.ForkJoinPoolHierarchicalTestExecutorService$ExclusiveTask.compute(ForkJoinPoolHierarchicalTestExecutorService.java:185)
>     at java.util.concurrent.RecursiveAction.exec(RecursiveAction.java:189)
>     at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
>     at 
> java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
>     at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
>     at 
> java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175)
>     Suppressed: org.apache.calcite.util.TestUtil$ExtraInformation: With 
> materializationsEnabled=false, limit=-1
>         at org.apache.calcite.util.TestUtil.rethrow(TestUtil.java:389)
>         at 
> org.apache.calcite.test.CalciteAssert.assertQuery(CalciteAssert.java:598)
>         ... 64 more
> Caused by: java.lang.RuntimeException: Error while applying rule 
> ArrowFilterRule, args 
> [rel#31:LogicalFilter.NONE.[](input=RelSubset#15,condition=OR(=($0, 12), 
> =($1, '12'))), rel#1:ArrowTableScan.ARROW.[](table=[ARROW, 
> ARROWDATA],fields=[0, 1, 2, 3])]
>     at 
> org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:250)
>     at 
> org.apache.calcite.plan.volcano.IterativeRuleDriver.drive(IterativeRuleDriver.java:59)
>     at 
> org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:524)
>     at org.apache.calcite.tools.Programs.lambda$standard$3(Programs.java:282)
>     at 
> org.apache.calcite.tools.Programs$SequenceProgram.run(Programs.java:348)
>     at org.apache.calcite.prepare.Prepare.optimize(Prepare.java:177)
>     at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:314)
>     at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:220)
>     at 
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:666)
>     at 
> org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:519)
>     at 
> org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:487)
>     at 
> org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:237)
>     at 
> org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:702)
>     at 
> org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:677)
>     at 
> org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:157)
>     ... 66 more
> Caused by: java.lang.UnsupportedOperationException: Unsupported disjunctive 
> condition OR(=($0, 12), =($1, '12'))
>     at 
> org.apache.calcite.adapter.arrow.ArrowTranslator.translateMatch(ArrowTranslator.java:70)
>     at 
> org.apache.calcite.adapter.arrow.ArrowFilter.<init>(ArrowFilter.java:43)
>     at 
> org.apache.calcite.adapter.arrow.ArrowRules$ArrowFilterRule.convert(ArrowRules.java:103)
>     at 
> org.apache.calcite.adapter.arrow.ArrowRules$ArrowFilterRule.onMatch(ArrowRules.java:88)
>     at 
> org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:223)
>     ... 80 more{code}
> The ideal plan the test should generate is:
> {code:java}
> "PLAN=ArrowToEnumerableConverter\n"
> + " ArrowProject(intField=[$0], stringField=[$1])\n"
> + " ArrowFilter(condition=[OR(=($0, 12), =($1, '12'))])\n"
> + " ArrowTableScan(table=[[ARROW, ARROWDATA]], fields=[[0, 1, 2, 
> 3]])\n\n";{code}
> The test query, however, after the proposed workaround, could produce this 
> valid plan in the meantime:
> {code:java}
> PLAN=EnumerableCalc(expr#0..1=[{inputs}], expr#2=[12], "
> + "expr#3=[=($t0, $t2)], expr#4=['12':VARCHAR], expr#5=[=($t1, $t4)], "
> + "expr#6=[OR($t3, $t5)], proj#0..1=[{exprs}], $condition=[$t6])\n"
> + " ArrowToEnumerableConverter\n"
> + " ArrowProject(intField=[$0], stringField=[$1])\n"
> + " ArrowTableScan(table=[[ARROW, ARROWDATA]], fields=[[0, 1, 2, 3]]){code}
> The advantage would be that more queries would be supported right away, while 
> waiting for the missing features to be contributed.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Reply via email to