hsiang-c opened a new issue, #25782:
URL: https://github.com/apache/datafusion/issues/25782
### Describe the bug
When DataFusion Comet process SparkSQL queries such as
```sql
SELECT 1 FROM (SELECT count(*) FROM VALUES (1) AS v(a))
```
The job fails with the following exceptions:
```shell
org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in
stage 2.0 failed 1 times, most recent failure: Lost task 0.0 in stage 2.0 (TID
2) (192.168.1.72 executor driver): org.apache.comet.CometNativeException:
Invalid argument error: must either specify a row count or at least one column
at org.apache.comet.Native.executePlan(Native Method)
at
org.apache.comet.CometExecIterator.$anonfun$getNextBatch$2(CometExecIterator.scala:240)
at
org.apache.comet.CometExecIterator.$anonfun$getNextBatch$2$adapted(CometExecIterator.scala:239)
at org.apache.comet.vector.NativeUtil.getNextBatch(NativeUtil.scala:196)
at
org.apache.comet.CometExecIterator.$anonfun$getNextBatch$1(CometExecIterator.scala:239)
at org.apache.comet.Tracing$.withTrace(Tracing.scala:43)
at
org.apache.comet.CometExecIterator.getNextBatch(CometExecIterator.scala:237)
at
org.apache.comet.CometExecIterator.hasNext(CometExecIterator.scala:281)
```
with physical plan likes this
```shell
== Physical Plan ==
*(1) CometColumnarToRow
+- CometHashAggregate
+- CometExchange SinglePartition, ENSURE_REQUIREMENTS,
CometNativeShuffle, [plan_id=123]
+- CometHashAggregate
+- CometLocalTableScan
```
### To Reproduce
1. Apply https://github.com/apache/datafusion-comet/pull/6235 in DataFusion
Comet
2. In Spark, turn on Comet with one additional config
`spark.comet.exec.localTableScan.enabled` and run the above query `SELECT 1
FROM (SELECT count(*) FROM VALUES (1) AS v(a))`
### Expected behavior
Produces identical result as Spark and DuckDB do
### Spark
```sql
scala> spark.sql("SELECT 1 FROM (SELECT count(*) FROM VALUES (1) AS
v(a))").show()
+---+
| 1|
+---+
| 1|
+---+
```
### DuckDB
```sql
memory D SELECT 1 FROM (SELECT count(*) FROM VALUES (1) AS v(a));
┌───────┐
│ 1 │
│ int32 │
├───────┤
│ 1 │
└───────┘
```
### Additional context
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
