Re: Credentials Rotation Failure on IO-Datastores cluster

2023-10-31 Thread Svetak Sundhar via dev
I took a quick look -- the error is the following: *22:17:26* ERROR: (gcloud.container.clusters.update) ResponseError: code=400, message=Operation operation-1698804621818-e9c8fe33-d4a2-44cd-86aa-9c4e09dea259 is currently upgrading cluster io-datastores. Please wait and try again once it is done.

Credentials Rotation Failure on IO-Datastores cluster

2023-10-31 Thread Apache Jenkins Server
Something went wrong during the automatic credentials rotation for IO-Datastores Cluster, performed at Wed Nov 01 00:52:45 UTC 2023. It may be necessary to check the state of the cluster certificates. For further details refer to the following links: * Failing job:

Credentials Rotation Failure on Metrics cluster (2023-11-01)

2023-10-31 Thread gacti...@beam.apache.org
Something went wrong during the automatic credentials rotation for Metrics Cluster, performed at 2023-11-01. It may be necessary to check the state of the cluster certificates. For further details refer to the following links:\n * Failing job:

Re: Processing time watermarks in KinesisIO

2023-10-31 Thread Robert Bradshaw via dev
On Tue, Oct 31, 2023 at 10:28 AM Jan Lukavský wrote: > > On 10/31/23 17:44, Robert Bradshaw via dev wrote: > > There are really two cases that make sense: > > > > (1) We read the event timestamps from the kafka records themselves and > > have some external knowledge that guarantees (or at least

Re: Processing time watermarks in KinesisIO

2023-10-31 Thread Jan Lukavský
On 10/31/23 17:44, Robert Bradshaw via dev wrote: There are really two cases that make sense: (1) We read the event timestamps from the kafka records themselves and have some external knowledge that guarantees (or at least provides a very good heuristic) about what the timestamps of unread

Re: Processing time watermarks in KinesisIO

2023-10-31 Thread Robert Bradshaw via dev
There are really two cases that make sense: (1) We read the event timestamps from the kafka records themselves and have some external knowledge that guarantees (or at least provides a very good heuristic) about what the timestamps of unread messages could be in the future to set the watermark.

Beam High Priority Issue Report (47)

2023-10-31 Thread beamactions
This is your daily summary of Beam's current high priority issues that may need attention. See https://beam.apache.org/contribute/issue-priorities for the meaning and expectations around issue priorities. Unassigned P1 Issues: https://github.com/apache/beam/issues/29099 [Bug]: FnAPI Java

Re: Processing time watermarks in KinesisIO

2023-10-31 Thread Jan Lukavský
I think that instead of deprecating and creating new version, we could leverage the proposed update compatibility flag for this [1]. I still have some doubts if the processing-time watermarking (and event-time assignment) makes sense. Do we have a valid use-case for that? This is actually the