Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

pvary commented on PR #8553: URL: https://github.com/apache/iceberg/pull/8553#issuecomment-1824077702 Merged to main. Thanks @stevenzwu, @dchristle, @sundargates, @gyfora and @jerqi for the reviews! -- This is an automated message from the Apache Git Service. To respond to the message

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

pvary merged PR #8553: URL: https://github.com/apache/iceberg/pull/8553 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apach

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

pvary commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1403082807 ## flink/v1.17/flink/src/main/java/org/apache/iceberg/flink/source/reader/ColumnStatsWatermarkExtractor.java: ## @@ -0,0 +1,98 @@ +/* + * Licensed to the Apache Software

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

pvary commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1403079553 ## flink/v1.17/flink/src/main/java/org/apache/iceberg/flink/source/IcebergSource.java: ## @@ -453,6 +492,18 @@ public IcebergSource build() { contextBuilder.proj

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

pvary commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1403031058 ## flink/v1.17/flink/src/test/java/org/apache/iceberg/flink/source/TestIcebergSourceWithWatermarkExtractor.java: ## @@ -0,0 +1,451 @@ +/* + * Licensed to the Apache Softw

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

stevenzwu commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1402429495 ## flink/v1.17/flink/src/main/java/org/apache/iceberg/flink/source/IcebergSource.java: ## @@ -429,6 +444,30 @@ public Builder setAll(Map properties) { return t

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

pvary commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1401866044 ## flink/v1.17/flink/src/test/java/org/apache/iceberg/flink/source/TestIcebergSourceWithWatermarkExtractor.java: ## @@ -0,0 +1,451 @@ +/* + * Licensed to the Apache Softw

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

pvary commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1401857507 ## flink/v1.17/flink/src/test/java/org/apache/iceberg/flink/source/TestIcebergSourceWithWatermarkExtractor.java: ## @@ -0,0 +1,434 @@ +/* + * Licensed to the Apache Softw

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

pvary commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1401803993 ## flink/v1.17/flink/src/test/java/org/apache/iceberg/flink/source/TestIcebergSourceWithWatermarkExtractor.java: ## @@ -0,0 +1,434 @@ +/* + * Licensed to the Apache Softw

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

pvary commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1401800615 ## flink/v1.17/flink/src/test/java/org/apache/iceberg/flink/source/TestIcebergSourceWithWatermarkExtractor.java: ## @@ -0,0 +1,434 @@ +/* + * Licensed to the Apache Softw

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

pvary commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1401797073 ## flink/v1.17/flink/src/test/java/org/apache/iceberg/flink/source/TestIcebergSourceWithWatermarkExtractor.java: ## @@ -0,0 +1,434 @@ +/* + * Licensed to the Apache Softw

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

pvary commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1401790452 ## flink/v1.17/flink/src/test/java/org/apache/iceberg/flink/source/TestIcebergSourceWithWatermarkExtractor.java: ## @@ -0,0 +1,434 @@ +/* + * Licensed to the Apache Softw

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

pvary commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1401785416 ## flink/v1.17/flink/src/test/java/org/apache/iceberg/flink/source/TestIcebergSourceFailoverWithWatermarkExtractor.java: ## @@ -0,0 +1,112 @@ +/* + * Licensed to the Apac

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

pvary commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1401784310 ## flink/v1.17/flink/src/main/java/org/apache/iceberg/flink/source/reader/ColumnStatsWatermarkExtractor.java: ## @@ -0,0 +1,98 @@ +/* + * Licensed to the Apache Software

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

gyfora commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1401758704 ## flink/v1.17/flink/src/main/java/org/apache/iceberg/flink/source/IcebergSource.java: ## @@ -453,6 +492,18 @@ public IcebergSource build() { contextBuilder.pro

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

pvary commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1401243005 ## flink/v1.17/flink/src/main/java/org/apache/iceberg/flink/source/IcebergSource.java: ## @@ -453,6 +492,18 @@ public IcebergSource build() { contextBuilder.proj

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

dchristle commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1401083561 ## flink/v1.17/flink/src/main/java/org/apache/iceberg/flink/source/IcebergSource.java: ## @@ -453,6 +492,18 @@ public IcebergSource build() { contextBuilder.

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

dchristle commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1401083561 ## flink/v1.17/flink/src/main/java/org/apache/iceberg/flink/source/IcebergSource.java: ## @@ -453,6 +492,18 @@ public IcebergSource build() { contextBuilder.

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

dchristle commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1401083561 ## flink/v1.17/flink/src/main/java/org/apache/iceberg/flink/source/IcebergSource.java: ## @@ -453,6 +492,18 @@ public IcebergSource build() { contextBuilder.

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

stevenzwu commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r140147 ## flink/v1.17/flink/src/main/java/org/apache/iceberg/flink/source/reader/ColumnStatsWatermarkExtractor.java: ## @@ -0,0 +1,98 @@ +/* + * Licensed to the Apache Softw

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

stevenzwu commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1400990227 ## flink/v1.17/flink/src/main/java/org/apache/iceberg/flink/source/IcebergSource.java: ## @@ -453,6 +492,18 @@ public IcebergSource build() { contextBuilder.

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

pvary commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1400301510 ## flink/v1.17/flink/src/main/java/org/apache/iceberg/flink/source/reader/ColumnStatsWatermarkExtractor.java: ## @@ -0,0 +1,93 @@ +/* + * Licensed to the Apache Software

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

pvary commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1400299663 ## flink/v1.17/flink/src/test/java/org/apache/iceberg/flink/source/TestIcebergSourceWithWatermarkExtractor.java: ## @@ -0,0 +1,377 @@ +/* + * Licensed to the Apache Softw

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

pvary commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1400292149 ## flink/v1.17/flink/src/test/java/org/apache/iceberg/flink/source/TestIcebergSourceWithWatermarkExtractor.java: ## @@ -0,0 +1,351 @@ +/* + * Licensed to the Apache Softw

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

pvary commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1400292444 ## flink/v1.17/flink/src/test/java/org/apache/iceberg/flink/source/TestIcebergSourceWithWatermarkExtractor.java: ## @@ -0,0 +1,351 @@ +/* + * Licensed to the Apache Softw

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

pvary commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1400291513 ## flink/v1.17/flink/src/test/java/org/apache/iceberg/flink/source/TestIcebergSourceWithWatermarkExtractor.java: ## @@ -0,0 +1,375 @@ +/* + * Licensed to the Apache Softw

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

pvary commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1400220361 ## flink/v1.17/flink/src/test/java/org/apache/iceberg/flink/source/TestIcebergSourceWithWatermarkExtractor.java: ## @@ -0,0 +1,359 @@ +/* + * Licensed to the Apache Softw

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

pvary commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1400207577 ## flink/v1.17/flink/src/test/java/org/apache/iceberg/flink/source/TestIcebergSourceFailoverWithWatermarkExtractor.java: ## @@ -0,0 +1,112 @@ +/* + * Licensed to the Apac

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

pvary commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1400201726 ## data/src/test/java/org/apache/iceberg/data/GenericAppenderHelper.java: ## @@ -120,6 +121,10 @@ private static DataFile appendToLocalFile( appenderFactory.setAll

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

pvary commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1399328620 ## flink/v1.17/flink/src/test/java/org/apache/iceberg/flink/source/TestIcebergSourceWithWatermarkExtractor.java: ## @@ -0,0 +1,375 @@ +/* + * Licensed to the Apache Softw

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

pvary commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1399025309 ## flink/v1.17/flink/src/main/java/org/apache/iceberg/flink/source/reader/ColumnStatsWatermarkExtractor.java: ## @@ -0,0 +1,93 @@ +/* + * Licensed to the Apache Software

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

pvary commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1399024202 ## flink/v1.17/flink/src/main/java/org/apache/iceberg/flink/source/reader/ColumnStatsWatermarkExtractor.java: ## @@ -0,0 +1,93 @@ +/* + * Licensed to the Apache Software

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

pvary commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1398110471 ## flink/v1.17/flink/src/test/java/org/apache/iceberg/flink/source/TestIcebergSourceWithWatermarkExtractor.java: ## @@ -0,0 +1,359 @@ +/* + * Licensed to the Apache Softw

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

pvary commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1398110471 ## flink/v1.17/flink/src/test/java/org/apache/iceberg/flink/source/TestIcebergSourceWithWatermarkExtractor.java: ## @@ -0,0 +1,359 @@ +/* + * Licensed to the Apache Softw

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

pvary commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1398107792 ## flink/v1.17/flink/src/main/java/org/apache/iceberg/flink/source/IcebergSource.java: ## @@ -453,6 +492,18 @@ public IcebergSource build() { contextBuilder.proj

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

pvary commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1398105061 ## flink/v1.17/flink/src/test/java/org/apache/iceberg/flink/source/reader/TestColumnStatsWatermarkExtractor.java: ## @@ -0,0 +1,177 @@ +/* + * Licensed to the Apache Soft

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

pvary commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1398104467 ## flink/v1.17/flink/src/main/java/org/apache/iceberg/flink/source/reader/ColumnStatsWatermarkExtractor.java: ## @@ -0,0 +1,72 @@ +/* + * Licensed to the Apache Software

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

stevenzwu commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1397782440 ## flink/v1.17/flink/src/main/java/org/apache/iceberg/flink/source/IcebergSource.java: ## @@ -453,6 +492,18 @@ public IcebergSource build() { contextBuilder.

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

stevenzwu commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1397782440 ## flink/v1.17/flink/src/main/java/org/apache/iceberg/flink/source/IcebergSource.java: ## @@ -453,6 +492,18 @@ public IcebergSource build() { contextBuilder.

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

stevenzwu commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1397721229 ## flink/v1.17/flink/src/main/java/org/apache/iceberg/flink/source/reader/ColumnStatsWatermarkExtractor.java: ## @@ -0,0 +1,72 @@ +/* + * Licensed to the Apache Softw

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

stevenzwu commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1397716578 ## flink/v1.17/flink/src/main/java/org/apache/iceberg/flink/source/IcebergSource.java: ## @@ -429,6 +444,30 @@ public Builder setAll(Map properties) { return t

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

pvary commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1397558768 ## flink/v1.17/flink/src/test/java/org/apache/iceberg/flink/source/reader/TestColumnStatsWatermarkExtractor.java: ## @@ -0,0 +1,215 @@ +/* + * Licensed to the Apache Soft

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

pvary commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1397557645 ## flink/v1.17/flink/src/test/java/org/apache/iceberg/flink/source/reader/TestColumnStatsWatermarkExtractor.java: ## @@ -0,0 +1,215 @@ +/* + * Licensed to the Apache Soft

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

pvary commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1397557005 ## flink/v1.17/flink/src/main/java/org/apache/iceberg/flink/source/reader/WatermarkExtractorRecordEmitter.java: ## @@ -0,0 +1,66 @@ +/* + * Licensed to the Apache Softwar

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

pvary commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1397517633 ## flink/v1.17/flink/src/test/java/org/apache/iceberg/flink/source/TestIcebergSourceWithWatermarkExtractor.java: ## @@ -0,0 +1,375 @@ +/* + * Licensed to the Apache Softw

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

pvary commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1397517358 ## flink/v1.17/flink/src/main/java/org/apache/iceberg/flink/source/reader/TimestampBasedWatermarkExtractor.java: ## @@ -0,0 +1,63 @@ +/* + * Licensed to the Apache Softwa

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

pvary commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1397486522 ## flink/v1.17/flink/src/main/java/org/apache/iceberg/flink/source/reader/IcebergWatermarkExtractor.java: ## @@ -18,19 +18,11 @@ */ package org.apache.iceberg.flink.so

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

pvary commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1397485790 ## flink/v1.17/flink/src/test/java/org/apache/iceberg/flink/source/TestIcebergSourceWithWatermarkExtractor.java: ## @@ -0,0 +1,377 @@ +/* + * Licensed to the Apache Softw

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

pvary commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1397087181 ## flink/v1.17/flink/src/test/java/org/apache/iceberg/flink/source/TestIcebergSourceWithWatermarkExtractor.java: ## @@ -0,0 +1,351 @@ +/* + * Licensed to the Apache Softw

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

pvary commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1397085585 ## flink/v1.17/flink/src/test/java/org/apache/iceberg/flink/source/TestIcebergSourceWithWatermarkExtractor.java: ## @@ -0,0 +1,351 @@ +/* + * Licensed to the Apache Softw

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

pvary commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1397050299 ## flink/v1.17/flink/src/test/java/org/apache/iceberg/flink/source/TestIcebergSourceWithWatermarkExtractor.java: ## @@ -0,0 +1,375 @@ +/* + * Licensed to the Apache Softw

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

pvary commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1396782299 ## flink/v1.17/flink/src/test/java/org/apache/iceberg/flink/source/TestIcebergSourceWithWatermarkExtractor.java: ## @@ -0,0 +1,351 @@ +/* + * Licensed to the Apache Softw

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

pvary commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1396766806 ## flink/v1.17/flink/src/test/java/org/apache/iceberg/flink/source/TestIcebergSourceWithWatermarkExtractor.java: ## @@ -0,0 +1,351 @@ +/* + * Licensed to the Apache Softw

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

pvary commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1396766098 ## flink/v1.17/flink/src/test/java/org/apache/iceberg/flink/source/reader/TestColumnStatsWatermarkExtractor.java: ## @@ -0,0 +1,215 @@ +/* + * Licensed to the Apache Soft

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

pvary commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1396764048 ## flink/v1.17/flink/src/test/java/org/apache/iceberg/flink/source/TestIcebergSourceFailoverWithWatermarkExtractor.java: ## @@ -0,0 +1,112 @@ +/* + * Licensed to the Apac

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

pvary commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1396759830 ## flink/v1.17/flink/src/main/java/org/apache/iceberg/flink/source/reader/WatermarkExtractorRecordEmitter.java: ## @@ -0,0 +1,66 @@ +/* + * Licensed to the Apache Softwar

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

pvary commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1396756527 ## flink/v1.17/flink/src/main/java/org/apache/iceberg/flink/source/reader/IcebergWatermarkExtractor.java: ## @@ -18,19 +18,11 @@ */ package org.apache.iceberg.flink.so

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

pvary commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1396755774 ## flink/v1.17/flink/src/test/java/org/apache/iceberg/flink/source/TestIcebergSourceWithWatermarkExtractor.java: ## @@ -0,0 +1,377 @@ +/* + * Licensed to the Apache Softw

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

pvary commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1396751097 ## flink/v1.17/flink/src/main/java/org/apache/iceberg/flink/source/reader/ColumnStatsWatermarkExtractor.java: ## @@ -0,0 +1,72 @@ +/* + * Licensed to the Apache Software

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

pvary commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1396746159 ## flink/v1.17/flink/src/main/java/org/apache/iceberg/flink/source/IcebergSource.java: ## @@ -429,6 +444,30 @@ public Builder setAll(Map properties) { return this;

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

stevenzwu commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1396577355 ## flink/v1.17/flink/src/main/java/org/apache/iceberg/flink/source/reader/WatermarkExtractorRecordEmitter.java: ## @@ -0,0 +1,66 @@ +/* + * Licensed to the Apache Sof

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

stevenzwu commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1394630754 ## flink/v1.17/flink/src/main/java/org/apache/iceberg/flink/source/reader/WatermarkExtractorRecordEmitter.java: ## @@ -0,0 +1,65 @@ +/* + * Licensed to the Apache Sof

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

pvary commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1395896201 ## flink/v1.17/flink/src/test/java/org/apache/iceberg/flink/source/TestIcebergSourceWithWatermarkExtractor.java: ## @@ -0,0 +1,375 @@ +/* + * Licensed to the Apache Softw

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

pvary commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1395895654 ## flink/v1.17/flink/src/main/java/org/apache/iceberg/flink/source/reader/WatermarkExtractorRecordEmitter.java: ## @@ -0,0 +1,65 @@ +/* + * Licensed to the Apache Softwar

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

pvary commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1395894584 ## flink/v1.17/flink/src/main/java/org/apache/iceberg/flink/source/reader/WatermarkExtractorRecordEmitter.java: ## @@ -0,0 +1,65 @@ +/* + * Licensed to the Apache Softwar

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

pvary commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1395894184 ## flink/v1.17/flink/src/main/java/org/apache/iceberg/flink/source/reader/TimestampBasedWatermarkExtractor.java: ## @@ -0,0 +1,63 @@ +/* + * Licensed to the Apache Softwa

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

pvary commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1395891709 ## flink/v1.17/flink/src/main/java/org/apache/iceberg/flink/source/reader/TimestampBasedWatermarkExtractor.java: ## @@ -0,0 +1,63 @@ +/* + * Licensed to the Apache Softwa

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

pvary commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1395890875 ## flink/v1.17/flink/src/main/java/org/apache/iceberg/flink/source/split/SplitComparators.java: ## @@ -56,4 +57,20 @@ public static SerializableComparator fileSequenceNu

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

pvary commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1395890308 ## flink/v1.17/flink/src/main/java/org/apache/iceberg/flink/source/reader/IcebergWatermarkExtractor.java: ## @@ -18,19 +18,11 @@ */ package org.apache.iceberg.flink.so

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

pvary commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1395543097 ## flink/v1.17/flink/src/test/java/org/apache/iceberg/flink/source/TestIcebergSourceWithWatermarkExtractor.java: ## @@ -0,0 +1,377 @@ +/* + * Licensed to the Apache Softw

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

pvary commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1395498235 ## flink/v1.17/flink/src/test/java/org/apache/iceberg/flink/source/TestIcebergSourceWithWatermarkExtractor.java: ## @@ -0,0 +1,375 @@ +/* + * Licensed to the Apache Softw

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

pvary commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1395457601 ## flink/v1.17/flink/src/test/java/org/apache/iceberg/flink/source/TestIcebergSourceWithWatermarkExtractor.java: ## @@ -0,0 +1,375 @@ +/* + * Licensed to the Apache Softw

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

pvary commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1395453030 ## flink/v1.17/flink/src/test/java/org/apache/iceberg/flink/source/TestIcebergSourceWithWatermarkExtractor.java: ## @@ -0,0 +1,375 @@ +/* + * Licensed to the Apache Softw

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

pvary commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1395450708 ## flink/v1.17/flink/src/test/java/org/apache/iceberg/flink/source/TestIcebergSourceFailover.java: ## @@ -97,6 +98,11 @@ protected List generateRecords(int numRecords, lo

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

pvary commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1395444790 ## flink/v1.17/flink/src/main/java/org/apache/iceberg/flink/source/reader/TimestampBasedWatermarkExtractor.java: ## @@ -0,0 +1,63 @@ +/* + * Licensed to the Apache Softwa

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

pvary commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1395344590 ## flink/v1.17/flink/src/main/java/org/apache/iceberg/flink/source/reader/TimestampBasedWatermarkExtractor.java: ## @@ -0,0 +1,63 @@ +/* + * Licensed to the Apache Softwa

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

pvary commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1395338328 ## flink/v1.17/flink/src/main/java/org/apache/iceberg/flink/source/reader/IcebergWatermarkExtractor.java: ## @@ -18,19 +18,11 @@ */ package org.apache.iceberg.flink.so

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

pvary commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1395324782 ## flink/v1.17/flink/src/test/java/org/apache/iceberg/flink/source/TestIcebergSourceWithWatermarkExtractor.java: ## @@ -0,0 +1,377 @@ +/* + * Licensed to the Apache Softw

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

stevenzwu commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1395154710 ## flink/v1.17/flink/src/test/java/org/apache/iceberg/flink/source/TestIcebergSourceWithWatermarkExtractor.java: ## @@ -0,0 +1,375 @@ +/* + * Licensed to the Apache S

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

stevenzwu commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1394590199 ## flink/v1.17/flink/src/main/java/org/apache/iceberg/flink/source/reader/TimestampBasedWatermarkExtractor.java: ## @@ -0,0 +1,63 @@ +/* + * Licensed to the Apache So

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

pvary commented on PR #8553: URL: https://github.com/apache/iceberg/pull/8553#issuecomment-1812436783 @stevenzwu, @sundargates: Finally, I was able to get #8803 in. So I updated the PR, and the new versions allows to set a `Timestamp` column name as a watermark source column: ``` Ice

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

pvary commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1394119859 ## flink/v1.17/flink/src/main/java/org/apache/iceberg/flink/source/reader/IcebergTimestampWatermarkExtractor.java: ## @@ -0,0 +1,65 @@ +/* + * Licensed to the Apache Soft

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

pvary commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1394118479 ## flink/v1.17/flink/src/main/java/org/apache/iceberg/flink/source/reader/IcebergWatermarkExtractor.java: ## @@ -0,0 +1,28 @@ +/* + * Licensed to the Apache Software Foun

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

pvary commented on code in PR #8553: URL: https://github.com/apache/iceberg/pull/8553#discussion_r1394117352 ## flink/v1.17/flink/src/main/java/org/apache/iceberg/flink/source/IcebergSource.java: ## @@ -429,6 +440,20 @@ public Builder setAll(Map properties) { return this;

Re: [PR] Flink: Emit watermarks from the IcebergSource [iceberg]

pvary commented on PR #8553: URL: https://github.com/apache/iceberg/pull/8553#issuecomment-1757872875 Created #8803 to have the possibility to avoid keeping all of the stats when creating the ScanTasks -- This is an automated message from the Apache Git Service. To respond to the message,