[ https://issues.apache.org/jira/browse/CALCITE-7687 ]
Etienne Pelissier deleted comment on CALCITE-7687:
--------------------------------------------
was (Author: JIRAUSER313108):
Test:
{code:java}
// Per-column null fractions, distinct so the value returned names the column
// that the predicate resolved to.
private static final double[] NULL_FRACTION = {0.13, 0.42, 0.77}; // a, b, c
// Selectivity implied by the null fraction of the column the predicate names,
// else Calcite's generic guess -- what a real column-statistics handler does.
private static @Nullable Double selectivityOf(@Nullable RexNode predicate) {
if (predicate instanceof RexCall
&& (predicate.getKind() == SqlKind.IS_NULL
|| predicate.getKind() == SqlKind.IS_NOT_NULL)
&& ((RexCall) predicate).getOperands().get(0) instanceof RexInputRef) {
final RexInputRef ref =
(RexInputRef) ((RexCall) predicate).getOperands().get(0);
final double nullFraction = NULL_FRACTION[ref.getIndex()];
return predicate.getKind() == SqlKind.IS_NULL
? nullFraction
: 1.0 - nullFraction;
}
return RelMdUtil.guessSelectivity(predicate);
}
private static RelDataType abcRowType(RelDataTypeFactory typeFactory) {
final RelDataType varchar =
typeFactory.createTypeWithNullability(
typeFactory.createSqlType(SqlTypeName.VARCHAR), true);
return typeFactory.builder()
.add("a", varchar).add("b", varchar).add("c", varchar).build();
}
// Found by stock getSelectivity(TableScan, ...) through RelOptTable#unwrap,
// the hook added by CALCITE-4223. No custom RelMdSelectivity is involved.
private static class SelectivityByColumnTable extends AbstractTable
implements BuiltInMetadata.Selectivity.Handler {
@Nullable RexNode received;
@Override public RelDataType getRowType(RelDataTypeFactory typeFactory) {
return abcRowType(typeFactory);
}
@Override public @Nullable Double getSelectivity(RelNode r,
RelMetadataQuery mq, @Nullable RexNode predicate) {
received = predicate;
return selectivityOf(predicate);
}
}
// Aggregate(group={1, 2}, COUNT($0)) over a scan of (a, b, c). RelBuilder
// yields the non-prefix group set directly, so no rule is needed; the same
// shape arises from "SELECT count(a) FROM t GROUP BY b, c HAVING c IS NULL"
// once AggregateProjectMergeRule has fired.
private static RelNode aggregateGroupingOnFields1And2(AbstractTable table) {
final SchemaPlus root = Frameworks.createRootSchema(true);
root.add("T", table);
final FrameworkConfig config =
Frameworks.newConfigBuilder().defaultSchema(root).build();
final RelBuilder b = RelBuilder.create(config);
return b.scan("T")
.aggregate(b.groupKey(1, 2), b.count(false, "cnt", b.field(0)))
.build();
}
private static RexNode isNullOn(RelNode rel, int i) {
final RexBuilder rexBuilder = rel.getCluster().getRexBuilder();
return rexBuilder.makeCall(SqlStdOperatorTable.IS_NULL,
rexBuilder.makeInputRef(rel.getRowType().getFieldList().get(i).getType(), i));
}
// Aggregate output field i is input field groupSet.nth(i), so a predicate on
// a group key must be converted before it is pushed to the input.
@Test void testSelectivityAggregateConvertsPredicateToInputFields() {
final SelectivityByColumnTable table = new SelectivityByColumnTable();
final RelNode agg = aggregateGroupingOnFields1And2(table);
final RelMetadataQuery mq = agg.getCluster().getMetadataQuery();
// Aggregate output $1 is column "c", input $2.
assertThat(mq.getSelectivity(agg, isNullOn(agg, 1)),
isAlmost(NULL_FRACTION[2]));
assertThat(table.received, hasToString("IS NULL($2)"));
// Aggregate output $0 is column "b", input $1.
assertThat(mq.getSelectivity(agg, isNullOn(agg, 0)),
isAlmost(NULL_FRACTION[1]));
assertThat(table.received, hasToString("IS NULL($1)"));
}
// An aggregate call has no equivalent expression on the input, so a predicate
// on one must not be pushed. Pushability is tested against groupSet = {1, 2},
// which holds input indices, so output $2 slips through;
range(getGroupCount())
// = {0, 1} would refuse it.
@Test void testSelectivityAggregateDoesNotPushAggregateCallPredicate() {
final SelectivityByColumnTable table = new SelectivityByColumnTable();
final RelNode agg = aggregateGroupingOnFields1And2(table);
final RelMetadataQuery mq = agg.getCluster().getMetadataQuery();
// Aggregate output $2 is COUNT($0), not a group key.
assertThat(mq.getSelectivity(agg, isNullOn(agg, 2)),
isAlmost(DEFAULT_SELECTIVITY));
assertThat(table.received, nullValue());
}
{code}
Fails with:
{noformat}
java.lang.AssertionError:
Expected: a numeric value within <1.0E-5> of <0.77>
but: <0.42> differed by <0.34999> more than delta <1.0E-5>
at
org.apache.calcite.test.RelMetadataTest.testSelectivityAggregateConvertsPredicateToInputFields(RelMetadataTest.java:1857)
java.lang.AssertionError:
Expected: a numeric value within <1.0E-5> of <0.25>
but: <0.77> differed by <0.5199900000000001> more than delta <1.0E-5>
at
org.apache.calcite.test.RelMetadataTest.testSelectivityAggregateDoesNotPushAggregateCallPredicate(RelMetadataTest.java:1879)
{noformat}
> RelMdSelectivity and RelMdDistinctRowCount for Aggregate can propagate a
> predicate with wrong references
> --------------------------------------------------------------------------------------------------------
>
> Key: CALCITE-7687
> URL: https://issues.apache.org/jira/browse/CALCITE-7687
> Project: Calcite
> Issue Type: Bug
> Components: core
> Reporter: Etienne Pelissier
> Assignee: Etienne Pelissier
> Priority: Minor
> Labels: in-progress
>
> Predicate pushability is assessed without taking into account column
> remapping from the aggregate.
> Additionally, predicates are pushed without being rewritten as need be.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
