Very cool! Thanks Charles! On Tue, Sep 18, 2018, 9:56 AM Charles Chen <c...@google.com> wrote:
> An update: the reference DirectRunner implementation of (and common > execution code for) the Python user state and timers API has been merged: > https://github.com/apache/beam/pull/6304 > > On Thu, Aug 30, 2018 at 1:48 AM Charles Chen <c...@google.com> wrote: > >> Another update: the reference DirectRunner implementation of the Python >> user state and timers API is out for review: >> https://github.com/apache/beam/pull/6304 >> >> On Mon, Jul 9, 2018 at 2:18 PM Charles Chen <c...@google.com> wrote: >> >>> An update: https://github.com/apache/beam/pull/5691 has been merged. I >>> hope to send out a reference implementation in the DirectRunner soon. On >>> the roadmap after that is work on the relevant portability interfaces here >>> so we can get this working on runners like Beam Python on Flink. >>> >>> On Wed, Jun 20, 2018 at 10:00 AM Charles Chen <c...@google.com> wrote: >>> >>>> An update on the implementation: I recently sent out the user-facing >>>> pipeline construction part of the API implementation out for review: >>>> https://github.com/apache/beam/pull/5691. >>>> >>>> On Tue, Jun 5, 2018 at 5:26 PM Charles Chen <c...@google.com> wrote: >>>> >>>>> Thanks everyone for contributing here. We've reached rough consensus >>>>> on the approach we should take with this API, and I've summarized this in >>>>> the new "Community consensus" sections I added to the doc ( >>>>> https://s.apache.org/beam-python-user-state-and-timers). I will >>>>> begin initial implementation of this API soon. >>>>> >>>>> On Wed, May 23, 2018 at 8:08 PM Thomas Weise <t...@apache.org> wrote: >>>>> >>>>>> Nice proposal; it's exciting to see this about to be added to the SDK >>>>>> as it enables a set of more complex use cases. >>>>>> >>>>>> I also think that some of the content can later be repurposed as user >>>>>> documentation. >>>>>> >>>>>> Thanks, >>>>>> Thomas >>>>>> >>>>>> >>>>>> On Wed, May 23, 2018 at 11:49 AM, Charles Chen <c...@google.com> >>>>>> wrote: >>>>>> >>>>>>> Thanks everyone for the detailed comments and discussions. It looks >>>>>>> like by now, we mostly agree with the requirements and overall direction >>>>>>> needed for the API, though there is continuing discussion on specific >>>>>>> details. I want to highlight two new sections of the doc, which address >>>>>>> some discussions that have come up: >>>>>>> >>>>>>> - *Existing state and transactionality*: this section addresses >>>>>>> how we will address an existing transactionality inconsistency in the >>>>>>> existing Java API. ( >>>>>>> >>>>>>> https://docs.google.com/document/d/1GadEkAmtbJQjmqiqfSzGw3b66TKerm8tyn6TK4blAys/edit#heading=h.ofyl9jspiz3b >>>>>>> ) >>>>>>> - *State for merging windows*: this section addresses how we >>>>>>> will deal with non-combinable state in conjunction with merging >>>>>>> windows. ( >>>>>>> >>>>>>> https://docs.google.com/document/d/1GadEkAmtbJQjmqiqfSzGw3b66TKerm8tyn6TK4blAys/edit#heading=h.ctxkcgabtzpy >>>>>>> ) >>>>>>> >>>>>>> Let me know any further comments and suggestions. >>>>>>> >>>>>>> On Tue, May 22, 2018 at 9:29 AM Kenneth Knowles <k...@google.com> >>>>>>> wrote: >>>>>>> >>>>>>>> Nice. I know that Java users have found it helpful to have this >>>>>>>> lower-level way of writing pipelines when the high-level primitives >>>>>>>> don't >>>>>>>> quite have the tight control they are looking for. I hope it will be a >>>>>>>> big >>>>>>>> draw for Python, too. >>>>>>>> >>>>>>>> (commenting on the doc) >>>>>>>> >>>>>>>> Kenn >>>>>>>> >>>>>>>> On Mon, May 21, 2018 at 5:15 PM Charles Chen <c...@google.com> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> I want to share a proposal for adding user state and timer support >>>>>>>>> to the Beam Python SDK and get the community's thoughts on how such >>>>>>>>> an API >>>>>>>>> should look: >>>>>>>>> https://s.apache.org/beam-python-user-state-and-timers >>>>>>>>> >>>>>>>>> Let me know what you think and please add any comments and >>>>>>>>> suggestions you may have. >>>>>>>>> >>>>>>>>> Best, >>>>>>>>> Charles >>>>>>>>> >>>>>>>> >>>>>>
