Another update: the reference DirectRunner implementation of the Python user state and timers API is out for review: https://github.com/apache/beam/pull/6304
On Mon, Jul 9, 2018 at 2:18 PM Charles Chen <c...@google.com> wrote: > An update: https://github.com/apache/beam/pull/5691 has been merged. I > hope to send out a reference implementation in the DirectRunner soon. On > the roadmap after that is work on the relevant portability interfaces here > so we can get this working on runners like Beam Python on Flink. > > On Wed, Jun 20, 2018 at 10:00 AM Charles Chen <c...@google.com> wrote: > >> An update on the implementation: I recently sent out the user-facing >> pipeline construction part of the API implementation out for review: >> https://github.com/apache/beam/pull/5691. >> >> On Tue, Jun 5, 2018 at 5:26 PM Charles Chen <c...@google.com> wrote: >> >>> Thanks everyone for contributing here. We've reached rough consensus on >>> the approach we should take with this API, and I've summarized this in the >>> new "Community consensus" sections I added to the doc ( >>> https://s.apache.org/beam-python-user-state-and-timers). I will begin >>> initial implementation of this API soon. >>> >>> On Wed, May 23, 2018 at 8:08 PM Thomas Weise <t...@apache.org> wrote: >>> >>>> Nice proposal; it's exciting to see this about to be added to the SDK >>>> as it enables a set of more complex use cases. >>>> >>>> I also think that some of the content can later be repurposed as user >>>> documentation. >>>> >>>> Thanks, >>>> Thomas >>>> >>>> >>>> On Wed, May 23, 2018 at 11:49 AM, Charles Chen <c...@google.com> wrote: >>>> >>>>> Thanks everyone for the detailed comments and discussions. It looks >>>>> like by now, we mostly agree with the requirements and overall direction >>>>> needed for the API, though there is continuing discussion on specific >>>>> details. I want to highlight two new sections of the doc, which address >>>>> some discussions that have come up: >>>>> >>>>> - *Existing state and transactionality*: this section addresses >>>>> how we will address an existing transactionality inconsistency in the >>>>> existing Java API. ( >>>>> >>>>> https://docs.google.com/document/d/1GadEkAmtbJQjmqiqfSzGw3b66TKerm8tyn6TK4blAys/edit#heading=h.ofyl9jspiz3b >>>>> ) >>>>> - *State for merging windows*: this section addresses how we will >>>>> deal with non-combinable state in conjunction with merging windows. ( >>>>> >>>>> https://docs.google.com/document/d/1GadEkAmtbJQjmqiqfSzGw3b66TKerm8tyn6TK4blAys/edit#heading=h.ctxkcgabtzpy >>>>> ) >>>>> >>>>> Let me know any further comments and suggestions. >>>>> >>>>> On Tue, May 22, 2018 at 9:29 AM Kenneth Knowles <k...@google.com> >>>>> wrote: >>>>> >>>>>> Nice. I know that Java users have found it helpful to have this >>>>>> lower-level way of writing pipelines when the high-level primitives don't >>>>>> quite have the tight control they are looking for. I hope it will be a >>>>>> big >>>>>> draw for Python, too. >>>>>> >>>>>> (commenting on the doc) >>>>>> >>>>>> Kenn >>>>>> >>>>>> On Mon, May 21, 2018 at 5:15 PM Charles Chen <c...@google.com> wrote: >>>>>> >>>>>>> I want to share a proposal for adding user state and timer support >>>>>>> to the Beam Python SDK and get the community's thoughts on how such an >>>>>>> API >>>>>>> should look: https://s.apache.org/beam-python-user-state-and-timers >>>>>>> >>>>>>> Let me know what you think and please add any comments and >>>>>>> suggestions you may have. >>>>>>> >>>>>>> Best, >>>>>>> Charles >>>>>>> >>>>>> >>>>
