comphead commented on code in PR #2729:
URL: https://github.com/apache/datafusion-comet/pull/2729#discussion_r2611508508
##########
spark/src/main/scala/org/apache/comet/serde/operator/CometIcebergNativeScan.scala:
##########
@@ -734,12 +807,47 @@ object CometIcebergNativeScan extends
CometOperatorSerde[CometBatchScanExec] wit
IcebergReflection.getDeleteFilesFromTask(task,
fileScanTaskClass)
val hasDeletes = !deletes.isEmpty
- // Use pre-extracted scanSchema for schema evolution
support
+ // Schema to pass to iceberg-rust's FileScanTask.
+ // This is used by RecordBatchTransformer for field type
lookups (e.g., in
+ // constants_map) and default value generation. The
actual projection is
+ // controlled by project_field_ids.
+ //
+ // Schema selection logic:
+ // 1. If hasDeletes=true: Use taskSchema (file-specific
schema) because
+ // delete files reference specific schema versions and
we need exact schema
+ // matching for MOR.
Review Comment:
```suggestion
// matching for merge-on-read.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
