[jira] [Created] (CALCITE-4200) ExceptionInInitializerError when initializing DruidRules
Stamatis Zampetakis created CALCITE-4200:
Summary: ExceptionInInitializerError when initializing DruidRules
Key: CALCITE-4200
URL: https://issues.apache.org/jira/browse/CALCITE-4200
Project: Calcite
Issue Type: Bug
Components: druid-adapter
Affects Versions: 1.25.0
Environment: Dockerized Druid containers created via
[https://github.com/zabetak/calcite-druid-dataset|calcite-druid-dataset].
Reporter: Stamatis Zampetakis
Assignee: Stamatis Zampetakis
Fix For: 1.26.0
Any attempt to run Druid integration tests ({{DruidAdapterIT}} or
{{DruidAdapter2IT}}) leads to an exception (see stacktrace below) due to class
initialization problems in {{DruidRules}}.
{noformat}
java.lang.ExceptionInInitializerError
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at java.lang.reflect.Proxy.newProxyInstance(Proxy.java:739)
at
org.apache.calcite.util.ImmutableBeans$BeanImpl.asBean(ImmutableBeans.java:458)
at
org.apache.calcite.util.ImmutableBeans$Def.makeBean(ImmutableBeans.java:476)
at
org.apache.calcite.util.ImmutableBeans$Def.access$200(ImmutableBeans.java:466)
at
org.apache.calcite.util.ImmutableBeans.create_(ImmutableBeans.java:90)
at org.apache.calcite.util.ImmutableBeans.copy(ImmutableBeans.java:78)
at org.apache.calcite.plan.RelRule$Config.as(RelRule.java:133)
at
java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:627)
at
org.apache.calcite.util.ImmutableBeans.lambda$makeDef$3(ImmutableBeans.java:275)
at
org.apache.calcite.util.ImmutableBeans$BeanImpl.invoke(ImmutableBeans.java:446)
at com.sun.proxy.$Proxy23.as(Unknown Source)
at
org.apache.calcite.adapter.druid.DruidRules.(DruidRules.java:169)
at
org.apache.calcite.adapter.druid.DruidQuery.register(DruidQuery.java:619)
at
org.apache.calcite.plan.AbstractRelOptPlanner.onNewClass(AbstractRelOptPlanner.java:239)
at
org.apache.calcite.plan.volcano.VolcanoPlanner.onNewClass(VolcanoPlanner.java:464)
at
org.apache.calcite.plan.AbstractRelOptPlanner.registerClass(AbstractRelOptPlanner.java:230)
at
org.apache.calcite.plan.volcano.VolcanoPlanner.registerImpl(VolcanoPlanner.java:1224)
at
org.apache.calcite.plan.volcano.VolcanoPlanner.register(VolcanoPlanner.java:589)
at
org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered(VolcanoPlanner.java:604)
at
org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered(VolcanoPlanner.java:84)
at
org.apache.calcite.rel.AbstractRelNode.onRegister(AbstractRelNode.java:268)
at
org.apache.calcite.plan.volcano.VolcanoPlanner.registerImpl(VolcanoPlanner.java:1132)
at
org.apache.calcite.plan.volcano.VolcanoPlanner.register(VolcanoPlanner.java:589)
at
org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered(VolcanoPlanner.java:604)
at
org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered(VolcanoPlanner.java:84)
at
org.apache.calcite.rel.AbstractRelNode.onRegister(AbstractRelNode.java:268)
at
org.apache.calcite.plan.volcano.VolcanoPlanner.registerImpl(VolcanoPlanner.java:1132)
at
org.apache.calcite.plan.volcano.VolcanoPlanner.setRoot(VolcanoPlanner.java:265)
at
org.apache.calcite.tools.Programs.lambda$standard$3(Programs.java:262)
at
org.apache.calcite.tools.Programs$SequenceProgram.run(Programs.java:331)
at org.apache.calcite.prepare.Prepare.optimize(Prepare.java:166)
at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:297)
at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:208)
at
org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:632)
at
org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:498)
at
org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:468)
at
org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:231)
at
org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:552)
at
org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:675)
at
org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:156)
at
org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:227)
at
org.apache.calcite.test.Calci
[jira] [Created] (CALCITE-4201) AssertionError when registering Druid rules due to conflict in description
Stamatis Zampetakis created CALCITE-4201:
Summary: AssertionError when registering Druid rules due to
conflict in description
Key: CALCITE-4201
URL: https://issues.apache.org/jira/browse/CALCITE-4201
Project: Calcite
Issue Type: Bug
Components: druid-adapter
Affects Versions: 1.25.0
Reporter: Stamatis Zampetakis
Assignee: Stamatis Zampetakis
Fix For: 1.26.0
Any attempt to run Druid integration tests ({{DruidAdapterIT}} or
{{DruidAdapter2IT}}) raises assertion errors (similar to the one below) when
Druid rules are registered to the planner.
A few Druids rules use the same description with rules registered by default in
the planner violating the uniqueness assertion.
{noformat}
java.lang.AssertionError: Rule's description should be unique; existing
rule=ProjectFilterTransposeRule; new rule=ProjectFilterTransposeRule
at
org.apache.calcite.plan.AbstractRelOptPlanner.addRule(AbstractRelOptPlanner.java:158)
at
org.apache.calcite.plan.volcano.VolcanoPlanner.addRule(VolcanoPlanner.java:406)
at
org.apache.calcite.adapter.druid.DruidQuery.register(DruidQuery.java:620)
at
org.apache.calcite.plan.AbstractRelOptPlanner.onNewClass(AbstractRelOptPlanner.java:239)
at
org.apache.calcite.plan.volcano.VolcanoPlanner.onNewClass(VolcanoPlanner.java:464)
at
org.apache.calcite.plan.AbstractRelOptPlanner.registerClass(AbstractRelOptPlanner.java:230)
at
org.apache.calcite.plan.volcano.VolcanoPlanner.registerImpl(VolcanoPlanner.java:1224)
at
org.apache.calcite.plan.volcano.VolcanoPlanner.register(VolcanoPlanner.java:589)
at
org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered(VolcanoPlanner.java:604)
at
org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered(VolcanoPlanner.java:84)
at
org.apache.calcite.rel.AbstractRelNode.onRegister(AbstractRelNode.java:268)
at
org.apache.calcite.plan.volcano.VolcanoPlanner.registerImpl(VolcanoPlanner.java:1132)
at
org.apache.calcite.plan.volcano.VolcanoPlanner.register(VolcanoPlanner.java:589)
at
org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered(VolcanoPlanner.java:604)
at
org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered(VolcanoPlanner.java:84)
at
org.apache.calcite.rel.AbstractRelNode.onRegister(AbstractRelNode.java:268)
at
org.apache.calcite.plan.volcano.VolcanoPlanner.registerImpl(VolcanoPlanner.java:1132)
at
org.apache.calcite.plan.volcano.VolcanoPlanner.setRoot(VolcanoPlanner.java:265)
at
org.apache.calcite.tools.Programs.lambda$standard$3(Programs.java:262)
at
org.apache.calcite.tools.Programs$SequenceProgram.run(Programs.java:331)
at org.apache.calcite.prepare.Prepare.optimize(Prepare.java:166)
at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:297)
at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:208)
at
org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:632)
at
org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:498)
at
org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:468)
at
org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:231)
at
org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:552)
at
org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:675)
at
org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:156)
at
org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:227)
at
org.apache.calcite.test.CalciteAssert.assertQuery(CalciteAssert.java:534)
at
org.apache.calcite.test.CalciteAssert$AssertQuery.lambda$queryContains$10(CalciteAssert.java:1788)
at
org.apache.calcite.test.CalciteAssert$AssertQuery.withConnection(CalciteAssert.java:1474)
at
org.apache.calcite.test.CalciteAssert$AssertQuery.queryContains(CalciteAssert.java:1787)
at
org.apache.calcite.test.DruidAdapterIT.testIsNotNull(DruidAdapterIT.java:3155)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:675)
at
org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
at
org.junit.jupiter.engine.execution.InvocationInterceptorChain$Vali
[jira] [Created] (CALCITE-4202) Refine Druid cost-model to capture differences in intermediate projections
Stamatis Zampetakis created CALCITE-4202:
Summary: Refine Druid cost-model to capture differences in
intermediate projections
Key: CALCITE-4202
URL: https://issues.apache.org/jira/browse/CALCITE-4202
Project: Calcite
Issue Type: Improvement
Components: druid-adapter
Reporter: Stamatis Zampetakis
The planner generates equivalent DruidQuery expressions with exactly the same
cost. Most of the time the expressions differ only in the number of
intermediate projections
For example, running the following query
{code:sql}
select distinct "countryName"
from "wiki"
where "page" = 'Jeremy Corbyn'
{code}
via {{DruidAdapterIT#testSelectDistinctWiki}} generates among others the
following alternatives during optimization.
+Choice 1+
{noformat}
rel#184:DruidQuery.BINDABLE.[](table=[wiki,
wiki],intervals=[1900-01-09T00:00:00.000Z/2992-01-10T00:00:00.000Z],filter==($13,
'Jeremy Corbyn'),projects=[$5, $13],groups={0},aggs=[])
{noformat}
+Choice 2+
{noformat}
rel#108:DruidQuery.BINDABLE.[](table=[wiki,
wiki],intervals=[1900-01-09T00:00:00.000Z/2992-01-10T00:00:00.000Z],filter==($13,
'Jeremy Corbyn'),projects=[$5],groups={0},aggs=[])
{noformat}
Using the debugger we can see that the cost of the two plans is exactly the
same (although they are different) which means that the one that was generated
first will dominate the other. Clearly in this case the second choice is a
better plan.
Performance wise the difference may not be that big but refining the cost is
beneficial at least for plan stability. Currently the final plan is dependent
on the order that the rules are applied.
The goal of this jira is to refine Druid's cost model so that choice 2 becomes
cheaper than choice 1 outlined above.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (CALCITE-4203) RelMdUniqueKeys should not return empty when meeting Intersect and Minus if its input has unique keys
Chunwei Lei created CALCITE-4203:
Summary: RelMdUniqueKeys should not return empty when meeting
Intersect and Minus if its input has unique keys
Key: CALCITE-4203
URL: https://issues.apache.org/jira/browse/CALCITE-4203
Project: Calcite
Issue Type: Improvement
Reporter: Chunwei Lei
Assignee: Chunwei Lei
Currently, we get an empty unique key for Intersect and Minus if its all is
true.
{code:java}
// code placeholder
public Set getUniqueKeys(SetOp rel, RelMetadataQuery mq,
boolean ignoreNulls) {
if (!rel.all) {
return ImmutableSet.of(
ImmutableBitSet.range(rel.getRowType().getFieldCount()));
}
return ImmutableSet.of();
}
{code}
However, from the semantic of Intersect and Minus, we can get their unique keys
from its input even if its all is true.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
