>From Ritik Raj <[email protected]>: Ritik Raj has submitted this change. ( https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/21392?usp=email )
Change subject: [ASTERIXDB-3787][STO] Fix infinite loop reading array-nested union-typed columns ...................................................................... [ASTERIXDB-3787][STO] Fix infinite loop reading array-nested union-typed columns - user model changes: no - storage format changes: no - interface changes: no PathExtractorVisitor.getReversedDelimiters() reversed the shared delimiters field in place via Collections.reverse instead of returning a reversed copy. The reader requires delimiter levels in descending order (largest nesting level first). For a union-typed leaf, getOrCreateReaders loops over the union's flat children, calling createReader -> getReversedDelimiters once per child. Each call re-reversed the same field, so odd-numbered children got the correct descending order and even-numbered children got it flipped back to ascending. With a single delimiter the flip is a no-op, so the bug only surfaced when the leaf sat under two or more nested arrays. An ascending delimiter array corrupts RepeatedPrimitiveColumnValuesReader: levelToDelimiterMap is built wrong and setDelimiterIndex uses the wrong threshold, so isDelimiter()/getDelimiterIndex() never reach the value EndOfRepeatedGroupAssembler waits for. The group-end condition never fires, the assembler keeps jumping back to the group's first value, and the query spins forever on the same tuple. Ex-ref: MB-72388 Change-Id: Ib4c961272a6eea730efeb7d75c95cd9ea72e9aaa Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/21392 Tested-by: Jenkins <[email protected]> Reviewed-by: Ritik Raj <[email protected]> Integration-Tests: Jenkins <[email protected]> Reviewed-by: Rithwik Koul <[email protected]> --- A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/column/filter/nested-array-mixed-type-empty/nested-array-mixed-type-empty.001.ddl.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/column/filter/nested-array-mixed-type-empty/nested-array-mixed-type-empty.002.update.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/column/filter/nested-array-mixed-type-empty/nested-array-mixed-type-empty.003.query.sqlpp A asterixdb/asterix-app/src/test/resources/runtimets/results/column/filter/nested-array-mixed-type-empty/nested-array-mixed-type-empty.003.adm M asterixdb/asterix-app/src/test/resources/runtimets/sqlpp_queries.xml M asterixdb/asterix-column/src/main/java/org/apache/asterix/column/metadata/schema/visitor/PathExtractorVisitor.java 6 files changed, 113 insertions(+), 2 deletions(-) Approvals: Ritik Raj: Looks good to me, approved Rithwik Koul: Looks good to me, but someone else must approve Jenkins: Verified; Verified Objections: Anon. E. Moose #1000171: Violations found diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/column/filter/nested-array-mixed-type-empty/nested-array-mixed-type-empty.001.ddl.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/column/filter/nested-array-mixed-type-empty/nested-array-mixed-type-empty.001.ddl.sqlpp new file mode 100644 index 0000000..522a804 --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/column/filter/nested-array-mixed-type-empty/nested-array-mixed-type-empty.001.ddl.sqlpp @@ -0,0 +1,38 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +/* + * Description : Minimal reproduction of the column-filter hang: a filter on a field reached through two + * nested array unnests, where the compared leaf holds mixed numeric types (int and double) + * and is therefore a UNION column, and the predicate matches no row. This used to hang + * forever while building the repeated column readers. + * Expected Res : Success (empty result) + */ + +DROP DATAVERSE test IF EXISTS; +CREATE DATAVERSE test; + +USE test; + +CREATE DATASET test PRIMARY KEY (id: string) +WITH { + "storage-format": { + "format": "column" + } +}; diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/column/filter/nested-array-mixed-type-empty/nested-array-mixed-type-empty.002.update.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/column/filter/nested-array-mixed-type-empty/nested-array-mixed-type-empty.002.update.sqlpp new file mode 100644 index 0000000..8c3d0d2 --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/column/filter/nested-array-mixed-type-empty/nested-array-mixed-type-empty.002.update.sqlpp @@ -0,0 +1,34 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +USE test; + +UPSERT INTO test ({ + "id": "1", + "payload": { + "array_field": [ + { + "nested_array": [ + { "field_1": "val1", "field_2": { "field3": 25 } }, + { "field_1": "val2", "field_2": { "field3": 25.6 } } + ] + } + ] + } +}); diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/column/filter/nested-array-mixed-type-empty/nested-array-mixed-type-empty.003.query.sqlpp b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/column/filter/nested-array-mixed-type-empty/nested-array-mixed-type-empty.003.query.sqlpp new file mode 100644 index 0000000..0d1d5ba --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/queries_sqlpp/column/filter/nested-array-mixed-type-empty/nested-array-mixed-type-empty.003.query.sqlpp @@ -0,0 +1,27 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +USE test; + +SELECT * +FROM test t +UNNEST t.payload.array_field n1 +UNNEST n1.nested_array n2 +WHERE n2.field_1 = "val1" + AND n2.field_2.field3 > 100; diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/results/column/filter/nested-array-mixed-type-empty/nested-array-mixed-type-empty.003.adm b/asterixdb/asterix-app/src/test/resources/runtimets/results/column/filter/nested-array-mixed-type-empty/nested-array-mixed-type-empty.003.adm new file mode 100644 index 0000000..e69de29 --- /dev/null +++ b/asterixdb/asterix-app/src/test/resources/runtimets/results/column/filter/nested-array-mixed-type-empty/nested-array-mixed-type-empty.003.adm diff --git a/asterixdb/asterix-app/src/test/resources/runtimets/sqlpp_queries.xml b/asterixdb/asterix-app/src/test/resources/runtimets/sqlpp_queries.xml index d4437c0..a1acc30 100644 --- a/asterixdb/asterix-app/src/test/resources/runtimets/sqlpp_queries.xml +++ b/asterixdb/asterix-app/src/test/resources/runtimets/sqlpp_queries.xml @@ -16761,6 +16761,11 @@ </compilation-unit> </test-case> <test-case FilePath="column"> + <compilation-unit name="filter/nested-array-mixed-type-empty"> + <output-dir compare="Text">filter/nested-array-mixed-type-empty</output-dir> + </compilation-unit> + </test-case> + <test-case FilePath="column"> <compilation-unit name="filter/double"> <output-dir compare="Text">filter/double</output-dir> </compilation-unit> diff --git a/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/metadata/schema/visitor/PathExtractorVisitor.java b/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/metadata/schema/visitor/PathExtractorVisitor.java index 1b63361..d2ef165 100644 --- a/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/metadata/schema/visitor/PathExtractorVisitor.java +++ b/asterixdb/asterix-column/src/main/java/org/apache/asterix/column/metadata/schema/visitor/PathExtractorVisitor.java @@ -138,7 +138,14 @@ } private int[] getReversedDelimiters() { - Collections.reverse(delimiters); - return delimiters.toIntArray(); + // toIntArray() returns a fresh copy; reverse the copy so the shared delimiters list is left intact for the + // next reader built from the same path (e.g., each branch of a union-typed leaf). + int[] reversed = delimiters.toIntArray(); + for (int i = 0, j = reversed.length - 1; i < j; i++, j--) { + int tmp = reversed[i]; + reversed[i] = reversed[j]; + reversed[j] = tmp; + } + return reversed; } } -- To view, visit https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/21392?usp=email To unsubscribe, or for help writing mail filters, visit https://asterix-gerrit.ics.uci.edu/settings?usp=email Gerrit-MessageType: merged Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Change-Id: Ib4c961272a6eea730efeb7d75c95cd9ea72e9aaa Gerrit-Change-Number: 21392 Gerrit-PatchSet: 12 Gerrit-Owner: Rithwik Koul <[email protected]> Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins <[email protected]> Gerrit-Reviewer: Rithwik Koul <[email protected]> Gerrit-Reviewer: Ritik Raj <[email protected]>
