Dataflow (among other runners) has the ability to "upgrade" running
pipelines with new code (e.g. capturing bug fixes, dependency updates,
and limited topology changes). Unfortunately some improvements (e.g.
new and improved ways of writing to BigQuery, optimized use of side
inputs, a change in
> One easy and standard way to make it more resilient would be to make it
idempotent instead of counting on uptime or receiving any particular event.
Yep, agreed that this wouldn't be super hard if someone wants to take it
on. Basically it would just be updating the tool to run on a schedule and
Agree. As long as we are getting enough of them, then our records as well
as any automation depending on it are fine. One easy and standard way to
make it more resilient would be to make it idempotent instead of counting
on uptime or receiving any particular event.
Kenn
On Tue, Oct 24, 2023 at
This is your daily summary of Beam's current high priority issues that may need
attention.
See https://beam.apache.org/contribute/issue-priorities for the meaning and
expectations around issue priorities.
Unassigned P1 Issues:
https://github.com/apache/beam/issues/29099 [Bug]: FnAPI Java
