Could you give an example of such a usecase? (I suppose I'm not quite following what it means for a timer to be unstable...)
On Mon, Jul 2, 2018 at 6:20 PM Reuven Lax <[email protected]> wrote: > One issue: we definitely have some strong use cases where we want this on > ProcessTimer but not on ProcessElement. Since both are on the same DoFn, > I'm not sure how you would represent this as a separate transform. > > On Mon, Jul 2, 2018 at 5:05 PM Robert Bradshaw <[email protected]> > wrote: > >> Thanks for the writeup. >> >> I'm wondering with, rather than phrasing this as an annotation on DoFn >> methods that gets plumbed down through the portability representation, if >> it would make more sense to introduce a new, primitive "EnsureStableInput" >> transform. For those runners whose reshuffle provide stable inputs, they >> could use that as an implementation, and other runners could provide other >> suitable implementations. >> >> >> >> On Mon, Jul 2, 2018 at 3:26 PM Robin Qiu <[email protected]> wrote: >> >>> Hi everyone, >>> >>> Thanks for your feedback on the doc. I have revamped it according to all >>> of the comments. The major changes I have made are: >>> * The problem description should be more general and accurate now. >>> * I added more background information, such as details about Reshuffle, >>> so I should be easier to understand now. >>> * I made it clear what is the scope of my current project and what could >>> be left to future work. >>> * It now reflects the current progress of my work, and discusses how it >>> should work with the portable pipeline representation (WIP) >>> >>> Also, I forgot to mention last time that this doc may be interesting to >>> those of you interested in Reshuffle, because Reshuffle is used as a >>> current workaround for the problem described in the doc. >>> >>> More comments are always welcome. >>> >>> Best, >>> Robin >>> >>> On Fri, Jun 15, 2018 at 7:34 AM Kenneth Knowles <[email protected]> wrote: >>> >>>> Thanks for the write up. It is great to see someone pushing this >>>> through. >>>> >>>> I wanted to bring Luke's high-level question back to the list for >>>> visibility: what about portability and other SDKs? >>>> >>>> Portability is probably trivial, but the "other SDKs" question is >>>> probably best answered by folks working on them who can have opinions about >>>> how it works in their SDKs idioms. >>>> >>>> Kenn >>>> >>>> >>>
