anew commented on code in PR #57192:
URL: https://github.com/apache/spark/pull/57192#discussion_r3599127660
##########
sql/pipelines/src/main/scala/org/apache/spark/sql/pipelines/autocdc/ChangeArgs.scala:
##########
@@ -153,21 +153,28 @@ object ScdType {
/**
* Configuration for an AutoCDC flow.
*
- * @param keys The column(s) that uniquely identify a row in the
source data.
- * @param sequencing Expression ordering CDC events to correctly resolve
out-of-order
- * arrivals. Must be a sortable type.
- * @param deleteCondition Expression that marks a source row as a DELETE. When
None, all
- * rows are treated as upserts.
- * @param storedAsScdType The SCD strategy these args should be applied to.
- * @param columnSelection Which source columns to select in the target table.
None means
- * all columns.
+ * @param keys The column(s) that uniquely identify a row in
the source data.
+ * @param sequencing Expression ordering CDC events to correctly
resolve out-of-order
+ * arrivals. Must be a sortable type.
+ * @param deleteCondition Expression that marks a source row as a
DELETE. When None, all
+ * rows are treated as upserts.
+ * @param storedAsScdType The SCD strategy these args should be applied
to.
+ * @param columnSelection Which source columns to select in the target
table. None means
+ * all columns.
+ * @param trackHistorySelection SCD2 only. Selects the selected user-data
columns whose values
+ * define a run: two consecutive upsert events
for the same key are
Review Comment:
Added a cross-reference from the trackHistorySelection doc to the "run of
upsert events" concept in Scd2BatchProcessor.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
