peterxcli opened a new issue, #5672:
URL: https://github.com/apache/datafusion-comet/issues/5672
### Describe the bug
`spark_read_side_padding_internal`
(`native/spark-expr/src/static_invoke/char_varchar_utils/read_side_padding.rs:225`)
iterates the length array with `let length = length.unwrap();`, so any NULL in
the length column panics with `called Option::unwrap() on a None value` and
fails the task with a `CometNativeException`. Spark's `StringRPad` /
`StringLPad` are `NullIntolerant` and return NULL for that row.
The path is default-on: `CometStringRPad` / `CometStringLPad` only require
the pad to be a literal, so `rpad(s, len_col)` runs natively. Both the 2-arg
`[Array, Array]` arm (lines 144-162) and the 3-arg `[Array, Array, Scalar]` arm
(163-183) reach the unwrap.
### Steps to reproduce
```sql
CREATE TABLE t USING parquet AS SELECT * FROM VALUES ('abc', 5), ('abc',
CAST(NULL AS INT)) AS t(s, l);
SELECT rpad(s, l) FROM t; -- same for lpad(s, l) and rpad(s, l, 'x')
```
Spark: `abc `, `NULL`. Comet: task fails — `called Option::unwrap() on a
None value` at `read_side_padding.rs:225`.
Unit level: `spark_rpad(["abc","abc"], lengths=[Some(4), None], " ")` panics.
### Expected behavior
NULL for every row whose length is NULL, like Spark.
### Proposed solution
Replace the unwrap with a match that appends null (and continues) when
either the string or the length is null, in both arms. Add a unit test with a
NULL length and a NULL-length row to `string_rpad.sql` / `testStringPadding`.
### Additional context
The fuzz test cannot catch this because `FuzzDataGenerator` never generates
NULL integers (#5389). The unwrap was introduced in #2630; #2096 was a
different, earlier `rpad` panic (non-literal length hitting the
unsupported-arguments arm).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
