comphead commented on code in PR #6743: URL: https://github.com/apache/datafusion-comet/pull/6743#discussion_r4231745886
########## spark/src/test/resources/sql-tests/expressions/array/arrays_overlap.sql: ########## @@ -285,3 +285,18 @@ SELECT arrays_overlap(array(NULL, 99), b) FROM test_arrays_overlap -- conditional (CASE WHEN) arrays query SELECT arrays_overlap(CASE WHEN a IS NOT NULL THEN a ELSE array(0) END, b) FROM test_arrays_overlap + +-- A constant array reaches native as a scalar (a folded literal, or make_array over literals +-- here, where ConstantFolding is excluded). Beside a column, every row has to be compared with +-- it: read as a one-row list beside the full column, it ran past its end when the constant came +-- second, and when it came first only the first row was compared and that answer returned for +-- the whole batch. +statement +CREATE TABLE test_overlap_batch(id bigint) USING parquet + +-- One file, so the rows share a batch. +statement +INSERT INTO test_overlap_batch SELECT id FROM range(0, 4, 1, 1) + +query +SELECT id, arrays_overlap(array(id), array(1L)), arrays_overlap(array(1L), array(id)) FROM test_overlap_batch Review Comment: From reading the code, the constant-second queries at lines 29 and 280 already have the #6742 shape. They pass on `main` because the `VALUES` insert at line 22 writes one file per row under `local[5]` (see #6628), so every batch has one row. Could we write `test_arrays_overlap` as one file instead of adding `test_overlap_batch`? `INSERT INTO test_arrays_overlap SELECT /*+ COALESCE(1) */ * FROM VALUES ... AS t(a, b)` is the pattern in `min_max_floating_point.sql`. A constant-first query whose answer changes by row, such as `arrays_overlap(array(3, 4, 5), a)`, would complete it. With a one-file table I expect lines 29 and 280 to panic without the fix, and the table already has a null array, an empty array and null elements. The flat `scalar_beside_column_*` Rust test would then repeat this, and the struct one could be a query over a one-file `test_overlap_struct`. I have not run any of it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
