stevenzwu commented on code in PR #13310:
URL: https://github.com/apache/iceberg/pull/13310#discussion_r2198345197
##########
spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/source/SparkPositionDeltaWrite.java:
##########
@@ -426,17 +428,35 @@ public DeltaWriter<InternalRow> createWriter(int
partitionId, long taskId) {
.writeProperties(writeProperties)
.build();
+ Function<InternalRow, InternalRow> rowLineageProjector =
+ context.dataSchema() != null
+ &&
context.dataSchema().findField(MetadataColumns.ROW_ID.fieldId()) != null
+ ? new ProjectRowLineageFromMetadata()
Review Comment:
I was suggesting to pass `context.dataSchema()` to the
ProjectRowLineageFromMetadata and eagerly construct the `ProjectingInternalRow`
in the constructor if row_id field exists.
We can also move this if-else condition into the
`ProjectRowLineageFromMetadata` class, which can return null if write schema
doesn't contain row_id field.
Then we also don't need to cast `InternalRow` to `ProjectingInternalRow`
inside the `ProjectRowLineageFromMetadata` class
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
