Re: [DISCUSS] Processing time timers in "batch" (faster-than-wall-time [re]processing)

2024-02-26 Thread Jan Lukavský
I think that before we introduce a possibly somewhat duplicate new feature we should be certain that it is really semantically different. I'll rephrase the two cases:  a) need to wait and block data (delay) - the use case is the motivating example of Throttle transform  b) act without data,

Re: [DISCUSS] Processing time timers in "batch" (faster-than-wall-time [re]processing)

2024-02-26 Thread Kenneth Knowles
Yea I like DelayTimer, or SleepTimer, or WaitTimer or some such. OutputTime is always an event time timestamp so it isn't even allowed to be set outside the window (or you'd end up with an element assigned to a window that it isn't within, since OutputTime essentially represents reserving the

Re: [DISCUSS] Processing time timers in "batch" (faster-than-wall-time [re]processing)

2024-02-26 Thread Robert Burke
Agreed that a retroactive behavior change would be bad, even if tied to a beam version change. I agree that it meshes well with the general theme of State & Timers exposing underlying primitives for implementing Windowing and similar. I'd say the distinction between the two might be additional

Re: [DISCUSS] Processing time timers in "batch" (faster-than-wall-time [re]processing)

2024-02-26 Thread Kenneth Knowles
Pulling out focus points: On Fri, Feb 23, 2024 at 7:21 PM Robert Bradshaw via dev wrote: > I can't act on something yet [...] but I expect to be able to [...] at some time in the processing-time future. I like this as a clear and internally-consistent feature description. It describes

Re: GsoC Apache Beam ManagerIO project communication

2024-02-26 Thread Danny McCormick via dev
Hey Frank, thanks for reaching out, I'm glad to hear you're interested in this project! I'll try to answer your questions below. > As I understand it, ManageIO serves as an abstraction layer between the Runner (pipeline) and the Connector, facilitating dynamic configuration, data access, and

Beam High Priority Issue Report (37)

2024-02-26 Thread beamactions
This is your daily summary of Beam's current high priority issues that may need attention. See https://beam.apache.org/contribute/issue-priorities for the meaning and expectations around issue priorities. Unassigned P0 Issues: https://github.com/apache/beam/issues/30377 [Failing Test]: