stevenzwu commented on code in PR #13310:
URL: https://github.com/apache/iceberg/pull/13310#discussion_r2198413757
##########
spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/source/SparkPositionDeltaWrite.java:
##########
@@ -426,17 +428,35 @@ public DeltaWriter<InternalRow> createWriter(int
partitionId, long taskId) {
.writeProperties(writeProperties)
.build();
+ Function<InternalRow, InternalRow> rowLineageProjector =
+ context.dataSchema() != null
+ &&
context.dataSchema().findField(MetadataColumns.ROW_ID.fieldId()) != null
+ ? new ProjectRowLineageFromMetadata()
Review Comment:
if `context.dataSchema() ` is null for a V3 table with row lineage, the row
lineage would be broken with this code. That is my concern.
If we stay with lazy initialization model and move this if-else condition
inside the `ProjectRowLineageFromMetadata`, it might solve the problem.
We can document the decision of using lazy initialization in comment so that
readers can understand the reason behind it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
