changliiu opened a new pull request, #34967: URL: https://github.com/apache/beam/pull/34967
V1 change stream can use null end timestamp for the query, however V2 the end timestamp of the query should be NOT NULL, and should be at most 30 mins from the max(now, start_timestamp). To allow users to still omit the connector endTimestamp field to run the connector forever, but to give a valid endTimestamp when try to query change stream, we set the change stream endTimestamp in this case as now + 2 mins. This solution works as the Apache beam checkpoints the ReadChangeStreamPartition execution every 5s or 5MB of output data produced. Moreover the change stream query has a hard 1 min deadline. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
