>From Vijay Sarathy <vijay.sara...@couchbase.com>: Vijay Sarathy has submitted this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18238 )
Change subject: [ASTERIXDB-3380][COMP] Internal error on a syntax error with productivity hint ...................................................................... [ASTERIXDB-3380][COMP] Internal error on a syntax error with productivity hint Change-Id: I42399559414f94204b22897d0fae43ea8c0f54d4 Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18238 Reviewed-by: Vijay Sarathy <vijay.sara...@couchbase.com> Reviewed-by: Ali Alsuliman <ali.al.solai...@gmail.com> Integration-Tests: Jenkins <jenk...@fulliautomatix.ics.uci.edu> Tested-by: Jenkins <jenk...@fulliautomatix.ics.uci.edu> --- A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/warnings/multiple-hints-warning/multiple-hints-warning.06.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/warnings/multiple-hints-warning/multiple-hints-warning.07.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/warnings/multiple-hints-warning/multiple-hints-warning.08.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/multiple-hints-warning/multiple-hints-warning.06.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/multiple-hints-warning/multiple-hints-warning.08.adm A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/warnings/multiple-hints-warning/multiple-hints-warning.05.query.sqlpp M asterixdb/asterix-app/src/test/resources/runtimets/sqlpp_queries.xml A asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/multiple-hints-warning/multiple-hints-warning.05.adm A asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/multiple-hints-warning/multiple-hints-warning.07.adm M asterixdb/asterix-lang-sqlpp/src/main/javacc/SQLPP.jj A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/warnings/multiple-hints-warning/multiple-hints-warning.09.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/multiple-hints-warning/multiple-hints-warning.09.adm 12 files changed, 210 insertions(+), 13 deletions(-) Approvals: Ali Alsuliman: Looks good to me, approved Vijay Sarathy: Looks good to me, but someone else must approve Jenkins: Verified; Verified Objections: Anon. E. Moose #1000171: Violations found diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/warnings/multiple-hints-warning/multiple-hints-warning.05.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/warnings/multiple-hints-warning/multiple-hints-warning.05.query.sqlpp new file mode 100644 index 0000000..64d8d90 --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/warnings/multiple-hints-warning/multiple-hints-warning.05.query.sqlpp @@ -0,0 +1,30 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +/* +* Description : Test warnings for multiple hints +* Expected Res : Warning, ignore hint +* Date : 11/20/2022 +*/ +// requesttype=application/json +// param max-warnings:json=10 + +use tpch; + +SELECT COUNT(*) FROM customer c JOIN nation n +ON c.c_nationkey /*+ hashjoin build (n) */ /*+ productivit y 100.0 */ = n.n_nationkey; diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/warnings/multiple-hints-warning/multiple-hints-warning.06.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/warnings/multiple-hints-warning/multiple-hints-warning.06.query.sqlpp new file mode 100644 index 0000000..2cdd902 --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/warnings/multiple-hints-warning/multiple-hints-warning.06.query.sqlpp @@ -0,0 +1,30 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +/* +* Description : Test warnings for multiple hints +* Expected Res : Warning, ignore hint +* Date : 11/20/2022 +*/ +// requesttype=application/json +// param max-warnings:json=10 + +use tpch; + +SELECT COUNT(*) FROM customer c JOIN nation n +ON c.c_nationkey /*+ productivit y 100.0 */ /*+ hashjoin build (n) */ = n.n_nationkey; diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/warnings/multiple-hints-warning/multiple-hints-warning.07.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/warnings/multiple-hints-warning/multiple-hints-warning.07.query.sqlpp new file mode 100644 index 0000000..7549340 --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/warnings/multiple-hints-warning/multiple-hints-warning.07.query.sqlpp @@ -0,0 +1,30 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +/* +* Description : Test warnings for multiple hints +* Expected Res : Warning, ignore hint +* Date : 11/20/2022 +*/ +// requesttype=application/json +// param max-warnings:json=10 + +use tpch; + +SELECT COUNT(*) FROM customer c JOIN nation n +ON c.c_nationkey /*+ hashjoin build (n) */ /*+ productivit y 100.0 */ /*+ 100.0 */ = n.n_nationkey; diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/warnings/multiple-hints-warning/multiple-hints-warning.08.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/warnings/multiple-hints-warning/multiple-hints-warning.08.query.sqlpp new file mode 100644 index 0000000..d580eaa --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/warnings/multiple-hints-warning/multiple-hints-warning.08.query.sqlpp @@ -0,0 +1,30 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +/* +* Description : Test warnings for multiple hints +* Expected Res : Warning, ignore hint +* Date : 11/20/2022 +*/ +// requesttype=application/json +// param max-warnings:json=10 + +use tpch; + +SELECT COUNT(*) FROM customer c JOIN nation n +ON c.c_nationkey /*+ productivit y 100.0 */ /*+ hashjoin build (n) */ /*+ 100.0 */ = n.n_nationkey; diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/warnings/multiple-hints-warning/multiple-hints-warning.09.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/warnings/multiple-hints-warning/multiple-hints-warning.09.query.sqlpp new file mode 100644 index 0000000..760ca89 --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/warnings/multiple-hints-warning/multiple-hints-warning.09.query.sqlpp @@ -0,0 +1,30 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +/* +* Description : Test warnings for multiple hints +* Expected Res : Warning, ignore hint +* Date : 11/20/2022 +*/ +// requesttype=application/json +// param max-warnings:json=10 + +use tpch; + +SELECT COUNT(*) FROM customer c JOIN nation n +ON c.c_nationkey /*+ productivit y 100.0 */ /*+ 100.0 */ /*+ hashjoin build (n) */ = n.n_nationkey; diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/multiple-hints-warning/multiple-hints-warning.05.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/multiple-hints-warning/multiple-hints-warning.05.adm new file mode 100644 index 0000000..3ff59f6 --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/multiple-hints-warning/multiple-hints-warning.05.adm @@ -0,0 +1 @@ +{ "$1": 0 } \ No newline at end of file diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/multiple-hints-warning/multiple-hints-warning.06.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/multiple-hints-warning/multiple-hints-warning.06.adm new file mode 100644 index 0000000..3ff59f6 --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/multiple-hints-warning/multiple-hints-warning.06.adm @@ -0,0 +1 @@ +{ "$1": 0 } \ No newline at end of file diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/multiple-hints-warning/multiple-hints-warning.07.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/multiple-hints-warning/multiple-hints-warning.07.adm new file mode 100644 index 0000000..3ff59f6 --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/multiple-hints-warning/multiple-hints-warning.07.adm @@ -0,0 +1 @@ +{ "$1": 0 } \ No newline at end of file diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/multiple-hints-warning/multiple-hints-warning.08.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/multiple-hints-warning/multiple-hints-warning.08.adm new file mode 100644 index 0000000..3ff59f6 --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/multiple-hints-warning/multiple-hints-warning.08.adm @@ -0,0 +1 @@ +{ "$1": 0 } \ No newline at end of file diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/multiple-hints-warning/multiple-hints-warning.09.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/multiple-hints-warning/multiple-hints-warning.09.adm new file mode 100644 index 0000000..3ff59f6 --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/warnings/multiple-hints-warning/multiple-hints-warning.09.adm @@ -0,0 +1 @@ +{ "$1": 0 } \ No newline at end of file diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/sqlpp_queries.xml b/asterixdb/asterix-app/src/test/resources/runtimets/sqlpp_queries.xml index 72f11b6..783727a 100644 --- a/asterixdb/asterix-app/src/test/resources/runtimets/sqlpp_queries.xml +++ b/asterixdb/asterix-app/src/test/resources/runtimets/sqlpp_queries.xml @@ -15655,6 +15655,14 @@ <expected-warn><![CDATA[ASX1132: Invalid specification for hint productivity. Invalid format for productivity values (in line 30, at column 46)]]></expected-warn> <expected-warn><![CDATA[ASX1132: Invalid specification for hint productivity. Invalid format for productivity values (in line 30, at column 45)]]></expected-warn> <expected-warn><![CDATA[ASX1132: Invalid specification for hint hashjoin. ASX1001: Syntax error: In line 1 >>build ()<< Encountered ")" at column 8. (in line 30, at column 20)]]></expected-warn> + <expected-warn>ASX1107: Unexpected hint: productivit y 100.0. "hash-bcast", "indexnl", "hashjoin", "skip-index", "use-index", "selectivity", "productivity" expected at this location</expected-warn> + <expected-warn>ASX1107: Unexpected hint: productivit y 100.0. "hash-bcast", "indexnl", "hashjoin", "skip-index", "use-index", "selectivity", "productivity" expected at this location</expected-warn> + <expected-warn>ASX1107: Unexpected hint: productivit y 100.0. "hash-bcast", "indexnl", "hashjoin", "skip-index", "use-index", "selectivity", "productivity" expected at this location</expected-warn> + <expected-warn>ASX1107: Unexpected hint: 100.0. "hash-bcast", "indexnl", "hashjoin", "skip-index", "use-index", "selectivity", "productivity" expected at this location</expected-warn> + <expected-warn>ASX1107: Unexpected hint: 100.0. "hash-bcast", "indexnl", "hashjoin", "skip-index", "use-index", "selectivity", "productivity" expected at this location</expected-warn> + <expected-warn>ASX1107: Unexpected hint: productivit y 100.0. "hash-bcast", "indexnl", "hashjoin", "skip-index", "use-index", "selectivity", "productivity" expected at this location</expected-warn> + <expected-warn>ASX1107: Unexpected hint: 100.0. "hash-bcast", "indexnl", "hashjoin", "skip-index", "use-index", "selectivity", "productivity" expected at this location</expected-warn> + <expected-warn>ASX1107: Unexpected hint: productivit y 100.0. "hash-bcast", "indexnl", "hashjoin", "skip-index", "use-index", "selectivity", "productivity" expected at this location</expected-warn> </compilation-unit> </test-case> <test-case FilePath="warnings" check-warnings="true"> diff --git a/asterixdb/asterix-lang-sqlpp/src/main/javacc/SQLPP.jj b/asterixdb/asterix-lang-sqlpp/src/main/javacc/SQLPP.jj index a1b7daf..da5f0ab 100644 --- a/asterixdb/asterix-lang-sqlpp/src/main/javacc/SQLPP.jj +++ b/asterixdb/asterix-lang-sqlpp/src/main/javacc/SQLPP.jj @@ -693,6 +693,25 @@ } } + private List<Token> fetchHints(Token token, SqlppHint... expectedHints) { + Token hintToken = token.specialToken; + List<Token> hintTokenList = new ArrayList<Token>(); + while (hintToken != null) { + SourceLocation sourceLoc = getSourceLocation(hintToken); + hintCollector.remove(sourceLoc); + if (hintToken.hint == null) { + warnUnexpectedHint(hintToken.hintParams, sourceLoc, expectedHints); + } else if (!ArrayUtils.contains(expectedHints, hintToken.hint)) { + warnUnexpectedHint(hintToken.hint.getIdentifier(), sourceLoc, expectedHints); + } else { + hintTokenList.add(hintToken); + } + hintToken = hintToken.specialToken; + } + return hintTokenList; + } + + private void reportUnclaimedHints() { for (Map.Entry<SourceLocation, String> me : hintCollector.entrySet()) { warnUnexpectedHint(me.getValue(), me.getKey(), "None"); @@ -3996,22 +4015,17 @@ if (opToken == null) { opToken = token; } - Token hintToken = fetchHint(token, + List<Token> hintTokens = fetchHints(token, SqlppHint.HASH_BROADCAST_JOIN_HINT, SqlppHint.INDEXED_NESTED_LOOP_JOIN_HINT, SqlppHint.HASH_JOIN_HINT, SqlppHint.SKIP_SECONDARY_INDEX_SEARCH_HINT, SqlppHint.USE_SECONDARY_INDEX_SEARCH_HINT, SqlppHint.SINGLE_DATASET_PREDICATE_SELECTIVITY_HINT, SqlppHint.JOIN_PREDICATE_PRODUCTIVITY_HINT ); - while (hintToken != null) { + for (Token hintToken : hintTokens) { annotation = parseExpressionAnnotation(hintToken); if (annotation != null) { // annotation may be null if hints are malformed annotationList.add(annotation); } - hintToken = hintToken.specialToken; - if (hintToken != null) { - SourceLocation sourceLoc = getSourceLocation(hintToken); - hintCollector.remove(sourceLoc); - } } String operator = opToken.image.toLowerCase(); if (operator.equals("<>")){ @@ -4040,7 +4054,7 @@ )? { - if (annotation != null) { + if (annotationList.size() > 0) { op.addHints(annotationList); } return op==null? operand: op; @@ -4053,6 +4067,7 @@ OperatorExpr op = null; Expression operand = null; IExpressionAnnotation annotation = null; + List<IExpressionAnnotation> annotationList = new ArrayList<IExpressionAnnotation>(); } { operand = IsExpr() @@ -4060,13 +4075,18 @@ LOOKAHEAD(2) (<NOT> { not = true; })? <BETWEEN> { - Token hintToken = fetchHint(token, + List<Token> hintTokens = fetchHints(token, SqlppHint.INDEXED_NESTED_LOOP_JOIN_HINT, SqlppHint.SKIP_SECONDARY_INDEX_SEARCH_HINT, SqlppHint.HASH_JOIN_HINT, SqlppHint.USE_SECONDARY_INDEX_SEARCH_HINT, SqlppHint.SINGLE_DATASET_PREDICATE_SELECTIVITY_HINT ); - if (hintToken != null) { - annotation = parseExpressionAnnotation(hintToken); + for (Token hintToken : hintTokens) { + annotation = parseExpressionAnnotation(hintToken); + if (annotation != null) { + // annotation may be null if hints are malformed + annotationList.add(annotation); + } } + String operator = token.image.toLowerCase(); if(not){ operator = "not_" + operator; @@ -4098,8 +4118,8 @@ )? { - if (annotation != null) { - op.addHint(annotation); + if (annotationList.size() > 0) { + op.addHints(annotationList); } return op==null ? operand: op; } -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18238 To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: I42399559414f94204b22897d0fae43ea8c0f54d4 Gerrit-Change-Number: 18238 Gerrit-PatchSet: 4 Gerrit-Owner: Vijay Sarathy <vijay.sara...@couchbase.com> Gerrit-Reviewer: Ali Alsuliman <ali.al.solai...@gmail.com> Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins <jenk...@fulliautomatix.ics.uci.edu> Gerrit-Reviewer: Vijay Sarathy <vijay.sara...@couchbase.com> Gerrit-MessageType: merged
