cfmcgrady opened a new issue, #2476:
URL: https://github.com/apache/datafusion-comet/issues/2476
### Describe the bug
While working on #2425, I encountered an exception when calling `reverse`
with an `ArrayType` input.
Got an error on `SPARK-48280: Expression Walker for expression evaluation`
```
2025-09-28T16:48:38.6458922Z [info] CollationExpressionWalkerSuite:
2025-09-28T16:48:38.8444445Z [info] - SPARK-48280: Expression Walker for
expression evaluation (124 milliseconds)
2025-09-28T16:48:39.4213875Z 16:48:39.420 WARN
org.apache.spark.sql.catalyst.analysis.FunctionResolution: Two-parameter
TRIM/LTRIM/RTRIM function signatures are deprecated. Use SQL syntax `TRIM((BOTH
| LEADING | TRAILING)? trimStr FROM str)` instead.
2025-09-28T16:48:39.4215799Z
2025-09-28T16:48:49.0826110Z 16:48:49.081 ERROR
org.apache.spark.executor.Executor: Exception in task 0.0 in stage 212.0 (TID
196)
2025-09-28T16:48:49.0827848Z org.apache.comet.CometNativeException: Error
from DataFusion: The REVERSE function can only accept strings, but got
List(Field { name: "$data$", data_type: Utf8, nullable: true, dict_id: 0,
dict_is_ordered: false, metadata: {} })..
2025-09-28T16:48:49.0829539Z at
org.apache.comet.Native.executePlan(Native Method)
2025-09-28T16:48:49.0830385Z at
org.apache.comet.CometExecIterator.$anonfun$getNextBatch$2(CometExecIterator.scala:173)
2025-09-28T16:48:49.0831526Z at
org.apache.comet.CometExecIterator.$anonfun$getNextBatch$2$adapted(CometExecIterator.scala:172)
2025-09-28T16:48:49.0832618Z at
org.apache.comet.vector.NativeUtil.getNextBatch(NativeUtil.scala:212)
2025-09-28T16:48:49.0833526Z at
org.apache.comet.CometExecIterator.$anonfun$getNextBatch$1(CometExecIterator.scala:172)
2025-09-28T16:48:49.0834820Z at
org.apache.comet.Tracing$.withTrace(Tracing.scala:31)
2025-09-28T16:48:49.0835584Z at
org.apache.comet.CometExecIterator.getNextBatch(CometExecIterator.scala:170)
2025-09-28T16:48:49.0845539Z 16:48:49.083 WARN
org.apache.spark.scheduler.TaskSetManager: Lost task 0.0 in stage 212.0 (TID
196) (4b4d7512dc01 executor driver): org.apache.comet.CometNativeException:
Error from DataFusion: The REVERSE function can only accept strings, but got
List(Field { name: "$data$", data_type: Utf8, nullable: true, dict_id: 0,
dict_is_ordered: false, metadata: {} })..
2025-09-28T16:48:49.0847604Z at
org.apache.comet.Native.executePlan(Native Method)
2025-09-28T16:48:49.0848408Z at
org.apache.comet.CometExecIterator.$anonfun$getNextBatch$2(CometExecIterator.scala:173)
2025-09-28T16:48:49.0849276Z at
org.apache.comet.CometExecIterator.$anonfun$getNextBatch$2...
```
### Steps to reproduce
```scala
sql("set
spark.sql.optimizer.excludedRules=org.apache.spark.sql.catalyst.optimizer.ConstantFolding")
sql("SELECT reverse(array(2, 1, 4, 3))").show
```
```
Job aborted due to stage failure: Task 0 in stage 1.0 failed 1 times, most
recent failure: Lost task 0.0 in stage 1.0 (TID 1) (10.27.37.205 executor
driver): org.apache.comet.CometNativeException: Error from DataFusion: The
REVERSE function can only accept strings, but got List(Field { name: "item",
data_type: Int32, nullable: true, dict_id: 0, dict_is_ordered: false, metadata:
{} })..
at org.apache.comet.Native.executePlan(Native Method)
at
org.apache.comet.CometExecIterator.$anonfun$getNextBatch$2(CometExecIterator.scala:173)
at
org.apache.comet.CometExecIterator.$anonfun$getNextBatch$2$adapted(CometExecIterator.scala:172)
at org.apache.comet.vector.NativeUtil.getNextBatch(NativeUtil.scala:212)
at
org.apache.comet.CometExecIterator.$anonfun$getNextBatch$1(CometExecIterator.scala:172)
at org.apache.comet.Tracing$.withTrace(Tracing.scala:31)
at
org.apache.comet.CometExecIterator.getNextBatch(CometExecIterator.scala:170)
at
org.apache.comet.CometExecIterator.hasNext(CometExecIterator.scala:221)
at
org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage2.cometcolumnartorow_nextBatch_0$(Unknown
Source)
at
org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage2.processNext(Unknown
Source)
at
org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43)
at
org.apache.spark.sql.execution.WholeStageCodegenEvaluatorFactory$WholeStageCodegenPartitionEvaluator$$anon$1.hasNext(WholeStageCodegenEvaluatorFactory.scala:43)
at
org.apache.spark.sql.execution.SparkPlan.$anonfun$getByteArrayRdd$1(SparkPlan.scala:388)
at
org.apache.spark.rdd.RDD.$anonfun$mapPartitionsInternal$2(RDD.scala:893)
at
org.apache.spark.rdd.RDD.$anonfun$mapPartitionsInternal$2$adapted(RDD.scala:893)
at
org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:367)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:331)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:93)
at
org.apache.spark.TaskContext.runTaskWithListeners(TaskContext.scala:166)
at org.apache.spark.scheduler.Task.run(Task.scala:141)
at
org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$4(Executor.scala:620)
at
org.apache.spark.util.SparkErrorUtils.tryWithSafeFinally(SparkErrorUtils.scala:64)
at
org.apache.spark.util.SparkErrorUtils.tryWithSafeFinally$(SparkErrorUtils.scala:61)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:94)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:623)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:840)
```
### Expected behavior
_No response_
### Additional context
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
