[
https://issues.apache.org/jira/browse/CALCITE-7687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Etienne Pelissier updated CALCITE-7687:
---------------------------------------
Description:
{{[RelMdSelectivity#getSelectivity|https://github.com/apache/calcite/blob/0211999427e294114db8f6b58cad95870cc13ccb/core/src/main/java/org/apache/calcite/rel/metadata/RelMdSelectivity.java#L178](Aggregate
rel, RelMetadataQuery mq, @Nullable RexNode predicate)}} method:
{code:java}
public @Nullable Double getSelectivity(Aggregate rel, RelMetadataQuery mq,
@Nullable RexNode predicate) {
...
RelOptUtil.splitFilters(
rel.getGroupSet(),
predicate,
pushable,
notPushable);
final RexBuilder rexBuilder = rel.getCluster().getRexBuilder();
RexNode childPred =
RexUtil.composeConjunction(rexBuilder, pushable, true);
Double selectivity = mq.getSelectivity(rel.getInput(), childPred);
...
}
{code}
{{[RelMdDistinctRowCount#getDistinctRowCount|https://github.com/apache/calcite/blob/0211999427e294114db8f6b58cad95870cc13ccb/core/src/main/java/org/apache/calcite/rel/metadata/RelMdDistinctRowCount.java#L168](Aggregate
rel, RelMetadataQuery mq, ImmutableBitSet groupKey, @Nullable RexNode
predicate)}} method:
{code:java}
public @Nullable Double getDistinctRowCount(Aggregate rel, RelMetadataQuery
mq,
ImmutableBitSet groupKey, @Nullable RexNode predicate) {
...
RelOptUtil.splitFilters(
ImmutableBitSet.range(rel.getGroupCount()),
predicate,
pushable,
notPushable);
final RexBuilder rexBuilder = rel.getCluster().getRexBuilder();
RexNode childPreds =
RexUtil.composeConjunction(rexBuilder, pushable, true);
// set the bits as they correspond to the child input
ImmutableBitSet.Builder childKey = ImmutableBitSet.builder();
RelMdUtil.setAggChildKeys(groupKey, rel, childKey);
Double distinctRowCount =
mq.getDistinctRowCount(rel.getInput(), childKey.build(), childPreds);
...
}
{code}
[CALCITE-4414|https://issues.apache.org/jira/browse/CALCITE-4414] fixed the
analogous problem in RelMdSelectivity#getSelectivity for Calc: it pushes a
predicate to its input, and RelOptUtil.pushPastCalc converts it to input-column
references first, which is possible because Calc output fields are themselves
expressions over the input.
Aggregate's group-key output fields are also references to specific input
fields, so predicates that reference only group keys can be translated the same
way, via an analogous RelOptUtil.pushPastAggregate, which currently doesn't
exist. Aggregate call outputs (e.g., COUNT(x\)) have no equivalent input
expression, so a predicate referencing one must not be pushed at all.
FilterAggregateTransposeRule already computes the same group-key mapping to
push a Filter past an Aggregate, and can be used as a basis to implement the
missing RelOptUtil.pushPastAggregate.
The two examples below show the problem for
RelMdSelectivity#getSelectivity(Aggregate) and
RelMdDistinctRowCount#getDistinctRowCount(Aggregate), respectively.
In both cases, the predicates reach the input with the wrong field references,
silently returning statistics for the wrong column rather than raising an
exception.
was:
{{[RelMdSelectivity#getSelectivity|https://github.com/apache/calcite/blob/0211999427e294114db8f6b58cad95870cc13ccb/core/src/main/java/org/apache/calcite/rel/metadata/RelMdSelectivity.java#L178](Aggregate
rel, RelMetadataQuery mq, @Nullable RexNode predicate)}} method:
{code:java}
public @Nullable Double getSelectivity(Aggregate rel, RelMetadataQuery mq,
@Nullable RexNode predicate) {
...
RelOptUtil.splitFilters(
rel.getGroupSet(),
predicate,
pushable,
notPushable);
final RexBuilder rexBuilder = rel.getCluster().getRexBuilder();
RexNode childPred =
RexUtil.composeConjunction(rexBuilder, pushable, true);
Double selectivity = mq.getSelectivity(rel.getInput(), childPred);
...
}
{code}
{{[RelMdDistinctRowCount#getDistinctRowCount|https://github.com/apache/calcite/blob/0211999427e294114db8f6b58cad95870cc13ccb/core/src/main/java/org/apache/calcite/rel/metadata/RelMdDistinctRowCount.java#L168](Aggregate
rel, RelMetadataQuery mq, ImmutableBitSet groupKey, @Nullable RexNode
predicate)}} method:
{code:java}
public @Nullable Double getDistinctRowCount(Aggregate rel, RelMetadataQuery
mq,
ImmutableBitSet groupKey, @Nullable RexNode predicate) {
...
RelOptUtil.splitFilters(
ImmutableBitSet.range(rel.getGroupCount()),
predicate,
pushable,
notPushable);
final RexBuilder rexBuilder = rel.getCluster().getRexBuilder();
RexNode childPreds =
RexUtil.composeConjunction(rexBuilder, pushable, true);
// set the bits as they correspond to the child input
ImmutableBitSet.Builder childKey = ImmutableBitSet.builder();
RelMdUtil.setAggChildKeys(groupKey, rel, childKey);
Double distinctRowCount =
mq.getDistinctRowCount(rel.getInput(), childKey.build(), childPreds);
...
}
{code}
[CALCITE-4414|https://issues.apache.org/jira/browse/CALCITE-4414] fixed the
analogous problem in RelMdSelectivity#getSelectivity for Calc: it pushes a
predicate to its input, and RelOptUtil.pushPastCalc converts it to input-column
references first, which is possible because Calc output fields are themselves
expressions over the input.
Aggregate's group-key output fields are also references to specific input
fields, so predicates that reference only group keys can be translated the same
way, via an analogous RelOptUtil.pushPastAggregate, which currently doesn't
exist. Aggregate call outputs (e.g., COUNT(x)) have no equivalent input
expression, so a predicate referencing one must not be pushed at all.
FilterAggregateTransposeRule already computes the same group-key mapping to
push a Filter past an Aggregate, and can be used as a basis to implement the
missing RelOptUtil.pushPastAggregate.
The two examples below show the problem for
RelMdSelectivity#getSelectivity(Aggregate) and
RelMdDistinctRowCount#getDistinctRowCount(Aggregate), respectively.
In both cases, the predicates reach the input with the wrong field references,
silently returning statistics for the wrong column rather than raising an
exception.
> RelMdSelectivity and RelMdDistinctRowCount for Aggregate can propagate a
> predicate with wrong references
> --------------------------------------------------------------------------------------------------------
>
> Key: CALCITE-7687
> URL: https://issues.apache.org/jira/browse/CALCITE-7687
> Project: Calcite
> Issue Type: Bug
> Components: core
> Affects Versions: 1.42.0
> Reporter: Etienne Pelissier
> Assignee: Etienne Pelissier
> Priority: Minor
> Labels: in-progress, pull-request-available
>
> {{[RelMdSelectivity#getSelectivity|https://github.com/apache/calcite/blob/0211999427e294114db8f6b58cad95870cc13ccb/core/src/main/java/org/apache/calcite/rel/metadata/RelMdSelectivity.java#L178](Aggregate
> rel, RelMetadataQuery mq, @Nullable RexNode predicate)}} method:
> {code:java}
> public @Nullable Double getSelectivity(Aggregate rel, RelMetadataQuery mq,
> @Nullable RexNode predicate) {
> ...
> RelOptUtil.splitFilters(
> rel.getGroupSet(),
> predicate,
> pushable,
> notPushable);
> final RexBuilder rexBuilder = rel.getCluster().getRexBuilder();
> RexNode childPred =
> RexUtil.composeConjunction(rexBuilder, pushable, true);
> Double selectivity = mq.getSelectivity(rel.getInput(), childPred);
> ...
> }
> {code}
> {{[RelMdDistinctRowCount#getDistinctRowCount|https://github.com/apache/calcite/blob/0211999427e294114db8f6b58cad95870cc13ccb/core/src/main/java/org/apache/calcite/rel/metadata/RelMdDistinctRowCount.java#L168](Aggregate
> rel, RelMetadataQuery mq, ImmutableBitSet groupKey, @Nullable RexNode
> predicate)}} method:
> {code:java}
> public @Nullable Double getDistinctRowCount(Aggregate rel, RelMetadataQuery
> mq,
> ImmutableBitSet groupKey, @Nullable RexNode predicate) {
> ...
> RelOptUtil.splitFilters(
> ImmutableBitSet.range(rel.getGroupCount()),
> predicate,
> pushable,
> notPushable);
> final RexBuilder rexBuilder = rel.getCluster().getRexBuilder();
> RexNode childPreds =
> RexUtil.composeConjunction(rexBuilder, pushable, true);
> // set the bits as they correspond to the child input
> ImmutableBitSet.Builder childKey = ImmutableBitSet.builder();
> RelMdUtil.setAggChildKeys(groupKey, rel, childKey);
> Double distinctRowCount =
> mq.getDistinctRowCount(rel.getInput(), childKey.build(), childPreds);
> ...
> }
> {code}
> [CALCITE-4414|https://issues.apache.org/jira/browse/CALCITE-4414] fixed the
> analogous problem in RelMdSelectivity#getSelectivity for Calc: it pushes a
> predicate to its input, and RelOptUtil.pushPastCalc converts it to
> input-column references first, which is possible because Calc output fields
> are themselves expressions over the input.
> Aggregate's group-key output fields are also references to specific input
> fields, so predicates that reference only group keys can be translated the
> same way, via an analogous RelOptUtil.pushPastAggregate, which currently
> doesn't exist. Aggregate call outputs (e.g., COUNT(x\)) have no equivalent
> input expression, so a predicate referencing one must not be pushed at all.
> FilterAggregateTransposeRule already computes the same group-key mapping to
> push a Filter past an Aggregate, and can be used as a basis to implement the
> missing RelOptUtil.pushPastAggregate.
> The two examples below show the problem for
> RelMdSelectivity#getSelectivity(Aggregate) and
> RelMdDistinctRowCount#getDistinctRowCount(Aggregate), respectively.
> In both cases, the predicates reach the input with the wrong field
> references, silently returning statistics for the wrong column rather than
> raising an exception.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
