The GitHub Actions job "Weekly Flink Connector Kafka CI" on 
flink-connector-kafka.git/main has failed.
Run started by GitHub user MartijnVisser (triggered by MartijnVisser).

Head commit for run:
a7e805a339964b795d1b805527cf3f68117e3b65 / lnbest0707 
<[email protected]>
[FLINK-39238] Support watermark alignment in dynamic Kafka reader (#240)

Flink source has watermark alignment config like 
scan.watermark.alignment.max-drift. The operator would then control:

- checkWatermarkAlignment for whole reader
- checkSplitWatermarkAlignment for the split reader

In dynamic Kafka source, the split reader pause/resume is missed 
(pauseOrResumeSplits not implemented) and the exceptions might raise once such 
config enabled.

This feature is important for workflows like ingesting data from Kafka to Data 
Lake. Missing the alignment could create far more number of small files once 
the ingestion progress diverges and runs our of controls.

Report URL: 
https://github.com/apache/flink-connector-kafka/actions/runs/23392232949

With regards,
GitHub Actions via GitBox

Reply via email to