grorge123 opened a new issue, #6742: URL: https://github.com/apache/datafusion-comet/issues/6742
### Describe the bug `arrays_overlap` gives wrong answers, or panics, when one argument is a constant array and the other comes from a column. When either argument is a scalar, `SparkArraysOverlap` (`native/spark-expr/src/array_funcs/arrays_overlap.rs`) turns the scalar into a one-row list with `to_array(1)`, while the column keeps its full length, and returns row 0's result as a scalar. With the constant first, only the first row is compared and that answer is used for the whole batch. With the constant second, the kernel reads the one-row list past its end and panics. ### Steps to reproduce One Parquet file, so the four rows share a batch: ```sql CREATE TABLE t(id bigint) USING parquet; INSERT INTO t SELECT id FROM range(0, 4, 1, 1); SELECT id, arrays_overlap(array(1L), array(id)) FROM t; SELECT id, arrays_overlap(array(id), array(1L)) FROM t; ``` On `main` at 8d8d1a26b (Spark 4.1 profile), the first query returns `false` for every row, where Spark returns `true` for `id = 1`. The second fails with `Comet native panic: panicked at spark-expr/src/array_funcs/arrays_overlap.rs:253:56: index out of bounds: the len is 2 but the index is 2`. ### Expected behavior Each row is compared against the constant, as in Spark: `true` for `id = 1` and `false` otherwise, for both argument orders. ### Additional context Found while working on #5526. The existing constant-array cases in `arrays_overlap.sql` pass on `main`; the reproduction above needs several rows in one batch. Assisted-by: Claude Code (claude-opus-5-5) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
