Hi, I have updated the proposal <https://s.apache.org/beam-fn-api-progress-reporting> based on the comments received. The major change is that the SDK no longer reports cumulative backlog, but includes more details for each transform itself. This provides a Runner more information about each transform in the fused sub-graph, while still being able to compute bundle progress and backlog. Thanks to Robert, Eugene and Luke for some key insights.
Prototyping is in progress, will share with you all when I have something concrete. Regards, Vikas On 17 July 2017 at 13:24, Kenneth Knowles <[email protected]> wrote: > This seems really well thought-out. A useful read for anyone interested in > Splittable DoFn, too. > > On Thu, Jul 13, 2017 at 11:34 AM, Vikas RK <[email protected]> wrote: > > > Hi, > > > > I wanted to share a drafted proposal for Progress Reporting in Fn API. > > > > https://s.apache.org/beam-fn-api-progress-reporting > > <https://www.google.com/url?q=https%3A%2F%2Fs.apache.org% > > 2Fbeam-fn-api-progress-reporting&sa=D&sntz=1&usg=AFQjCNHXxzlsB- > > 9oogQIr6Hril9c67SdEw> > > . > > > > Would like to get comments and feedback from the community. > > > > Regards, > > Vikas > > >
