>From Preetham Poluparthi <[email protected]>: Preetham Poluparthi has submitted this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/21052?usp=email )
Change subject: [ASTERIXDB-3709][COMP] Propagate skip-index annotation across disjunctive predicates ...................................................................... [ASTERIXDB-3709][COMP] Propagate skip-index annotation across disjunctive predicates - user model changes: no - storage format changes: no - interface changes: no Details: Previously, skip-index annotations were applied only to the predicates where they were explicitly specified within disjunctions. This could lead to inconsistent evaluation and incorrect results, as some branches of the disjunction would bypass index usage while others would not. Ext-ref: MB-71123 Change-Id: Ie864d43060d688a52dd0a0f9621163bbab84623c Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/21052 Reviewed-by: Ali Alsuliman <[email protected]> Tested-by: Jenkins <[email protected]> Integration-Tests: Jenkins <[email protected]> Reviewed-by: Preetham Poluparthi <[email protected]> --- M asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AbstractIntroduceAccessMethodRule.java A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/index-selection/hints-skip-index/hints-skip-index.14.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/index-selection/hints-skip-index/hints-skip-index.15.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/index-selection/hints-use-index/hints-use-index.20.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/index-selection/hints-use-index/hints-use-index.21.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/index-selection/hints-use-index/hints-use-index.22.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/index-selection/hints-skip-index/hints-skip-index.14.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/index-selection/hints-skip-index/hints-skip-index.15.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/index-selection/hints-use-index/hints-use-index.20.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/index-selection/hints-use-index/hints-use-index.21.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/index-selection/hints-use-index/hints-use-index.22.adm 11 files changed, 302 insertions(+), 5 deletions(-) Approvals: Preetham Poluparthi: Looks good to me, but someone else must approve Jenkins: Verified; Verified Ali Alsuliman: Looks good to me, approved Objections: Anon. E. Moose #1000171: Violations found diff --git a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AbstractIntroduceAccessMethodRule.java b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AbstractIntroduceAccessMethodRule.java index 20387fd..7a66c6a 100644 --- a/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AbstractIntroduceAccessMethodRule.java +++ b/asterixdb/asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/am/AbstractIntroduceAccessMethodRule.java @@ -30,6 +30,7 @@ import java.util.stream.Collectors; import org.apache.asterix.common.annotations.AbstractExpressionAnnotationWithIndexNames; +import org.apache.asterix.common.annotations.SkipSecondaryIndexSearchExpressionAnnotation; import org.apache.asterix.common.config.DatasetConfig; import org.apache.asterix.common.config.DatasetConfig.DatasetType; import org.apache.asterix.common.config.DatasetConfig.IndexType; @@ -740,12 +741,8 @@ found = false; break; } - if (!found) { - break; - } - boolean matchFound = analyzeSelectOrJoinOpConditionAndUpdateAnalyzedAM(argFuncExpr, assignsAndUnnests, + found = analyzeSelectOrJoinOpConditionAndUpdateAnalyzedAM(argFuncExpr, assignsAndUnnests, disjuncAnalyzedAMs, context, typeEnvironment); - found = found && matchFound; if (!found) { break; } @@ -762,6 +759,50 @@ for (IOptimizableFuncExpr optFuncExpr : disjuncAnalysisCtx.getMatchedFuncExprs()) { analysisCtx.addMatchedFuncExpr(optFuncExpr); } + + /* + * If any predicate within a disjunction is marked with skip-index, + * propagate the annotation to all predicates in the disjunction. + * var /skip-index/ = 0 or var = 1 should entirely skip index search for var = 0 and var = 1 + * */ + + boolean skipAnyIndex = false; + Collection<String> indexNames = new ArrayList<>(); + for (Mutable<ILogicalExpression> arg : funcExpr.getArguments()) { + AbstractFunctionCallExpression argExpr = (AbstractFunctionCallExpression) arg.get(); + SkipSecondaryIndexSearchExpressionAnnotation anno = + argExpr.getAnnotation(SkipSecondaryIndexSearchExpressionAnnotation.class); + if (anno == SkipSecondaryIndexSearchExpressionAnnotation.INSTANCE_ANY_INDEX) { + skipAnyIndex = true; + break; + } else if (anno != null) { + indexNames.addAll(anno.getIndexNames()); + } + } + + if (skipAnyIndex) { + funcExpr.getArguments().forEach(arg -> { + AbstractFunctionCallExpression argExpr = (AbstractFunctionCallExpression) arg.get(); + SkipSecondaryIndexSearchExpressionAnnotation anno = + argExpr.getAnnotation(SkipSecondaryIndexSearchExpressionAnnotation.class); + if (anno != SkipSecondaryIndexSearchExpressionAnnotation.INSTANCE_ANY_INDEX) { + argExpr.removeAnnotation(SkipSecondaryIndexSearchExpressionAnnotation.class); + argExpr.putAnnotation(SkipSecondaryIndexSearchExpressionAnnotation.INSTANCE_ANY_INDEX); + } + }); + } else if (!indexNames.isEmpty()) { + SkipSecondaryIndexSearchExpressionAnnotation annotation = + SkipSecondaryIndexSearchExpressionAnnotation.newInstance(indexNames); + funcExpr.getArguments().forEach(arg -> { + AbstractFunctionCallExpression argExpr = (AbstractFunctionCallExpression) arg.get(); + SkipSecondaryIndexSearchExpressionAnnotation argExprAnnotation = + argExpr.getAnnotation(SkipSecondaryIndexSearchExpressionAnnotation.class); + if (argExprAnnotation != null) { + argExpr.removeAnnotation(SkipSecondaryIndexSearchExpressionAnnotation.class); + } + argExpr.putAnnotation(annotation); + }); + } } return found; } diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/index-selection/hints-skip-index/hints-skip-index.14.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/index-selection/hints-skip-index/hints-skip-index.14.query.sqlpp new file mode 100644 index 0000000..5021e56 --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/index-selection/hints-skip-index/hints-skip-index.14.query.sqlpp @@ -0,0 +1,31 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +use test; + +/* + * Skip index hint on one of the conditions should be applied to all conditions in the OR expression. +*/ + +SET `rewrite_or_as_join` "false"; + +select value unique1 +from tenk +where thousand /* +skip-index */ = 0 or thousand = 1 +order by unique1; \ No newline at end of file diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/index-selection/hints-skip-index/hints-skip-index.15.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/index-selection/hints-skip-index/hints-skip-index.15.query.sqlpp new file mode 100644 index 0000000..50c0147 --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/index-selection/hints-skip-index/hints-skip-index.15.query.sqlpp @@ -0,0 +1,32 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +use test; + +/* + * disjunctive predicate is translated into an index lookup, use skip-index to disable idx_1k for index lookup. + * explain plan -> expect index lookup using idx_1k_2k + */ + +SET `rewrite_or_as_join` "false"; + +select value unique1 +from tenk +where thousand /* +skip-index(idx_1k) */ = 0 or thousand = 1 +order by unique1; \ No newline at end of file diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/index-selection/hints-use-index/hints-use-index.20.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/index-selection/hints-use-index/hints-use-index.20.query.sqlpp new file mode 100644 index 0000000..2183deb --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/index-selection/hints-use-index/hints-use-index.20.query.sqlpp @@ -0,0 +1,32 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +use test; + +/* + * Use-index forcing an index usage in an OR condition. + */ + +SET `rewrite_or_as_join` "false"; + + +select value unique1 +from tenk +where thousand /* +use-index(idx_1k_2k) */ = 0 or thousand = 1 +order by unique1; \ No newline at end of file diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/index-selection/hints-use-index/hints-use-index.21.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/index-selection/hints-use-index/hints-use-index.21.query.sqlpp new file mode 100644 index 0000000..0ee5b70 --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/index-selection/hints-use-index/hints-use-index.21.query.sqlpp @@ -0,0 +1,29 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +use test; + + + +SET `rewrite_or_as_join` "false"; + +select value unique1 +from tenk +where thousand /* +use-index(idx_1k_2k) */ = 0 or thousand /* +skip-index */ = 1 +order by unique1; \ No newline at end of file diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/index-selection/hints-use-index/hints-use-index.22.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/index-selection/hints-use-index/hints-use-index.22.query.sqlpp new file mode 100644 index 0000000..c6c91ab --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/index-selection/hints-use-index/hints-use-index.22.query.sqlpp @@ -0,0 +1,32 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +use test; + +/* + * Use-index forcing an index usage in an OR condition. + */ + +SET `rewrite_or_as_join` "false"; + + +select value unique1 +from tenk +where thousand /* +use-index(idx_1k) */ = 0 or thousand = 1 +order by unique1; \ No newline at end of file diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/index-selection/hints-skip-index/hints-skip-index.14.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/index-selection/hints-skip-index/hints-skip-index.14.adm new file mode 100644 index 0000000..5d5162b --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/index-selection/hints-skip-index/hints-skip-index.14.adm @@ -0,0 +1,20 @@ +0 +1 +1000 +1001 +2000 +2001 +3000 +3001 +4000 +4001 +5000 +5001 +6000 +6001 +7000 +7001 +8000 +8001 +9000 +9001 \ No newline at end of file diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/index-selection/hints-skip-index/hints-skip-index.15.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/index-selection/hints-skip-index/hints-skip-index.15.adm new file mode 100644 index 0000000..5d5162b --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/index-selection/hints-skip-index/hints-skip-index.15.adm @@ -0,0 +1,20 @@ +0 +1 +1000 +1001 +2000 +2001 +3000 +3001 +4000 +4001 +5000 +5001 +6000 +6001 +7000 +7001 +8000 +8001 +9000 +9001 \ No newline at end of file diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/index-selection/hints-use-index/hints-use-index.20.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/index-selection/hints-use-index/hints-use-index.20.adm new file mode 100644 index 0000000..5d5162b --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/index-selection/hints-use-index/hints-use-index.20.adm @@ -0,0 +1,20 @@ +0 +1 +1000 +1001 +2000 +2001 +3000 +3001 +4000 +4001 +5000 +5001 +6000 +6001 +7000 +7001 +8000 +8001 +9000 +9001 \ No newline at end of file diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/index-selection/hints-use-index/hints-use-index.21.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/index-selection/hints-use-index/hints-use-index.21.adm new file mode 100644 index 0000000..5d5162b --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/index-selection/hints-use-index/hints-use-index.21.adm @@ -0,0 +1,20 @@ +0 +1 +1000 +1001 +2000 +2001 +3000 +3001 +4000 +4001 +5000 +5001 +6000 +6001 +7000 +7001 +8000 +8001 +9000 +9001 \ No newline at end of file diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/index-selection/hints-use-index/hints-use-index.22.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/index-selection/hints-use-index/hints-use-index.22.adm new file mode 100644 index 0000000..5d5162b --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/index-selection/hints-use-index/hints-use-index.22.adm @@ -0,0 +1,20 @@ +0 +1 +1000 +1001 +2000 +2001 +3000 +3001 +4000 +4001 +5000 +5001 +6000 +6001 +7000 +7001 +8000 +8001 +9000 +9001 \ No newline at end of file -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/21052?usp=email To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings?usp=email Gerrit-MessageType: merged Gerrit-Project: asterixdb Gerrit-Branch: phoenix Gerrit-Change-Id: Ie864d43060d688a52dd0a0f9621163bbab84623c Gerrit-Change-Number: 21052 Gerrit-PatchSet: 6 Gerrit-Owner: Preetham Poluparthi <[email protected]> Gerrit-Reviewer: Ali Alsuliman <[email protected]> Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins <[email protected]> Gerrit-Reviewer: Preetham Poluparthi <[email protected]>
