Augmenting Luke's suggestion with @RequiresStableInput is our design for ensuring values written to the key+value store are stable. This will have some cost. It is also not supported by all runners.
Kenn On Mon, Nov 4, 2019 at 3:44 PM Luke Cwik <lc...@google.com> wrote: > This has come up for Dataflow customers as well where people would like to > directly serve the content that is stored in state and I'm not aware of any > current plans to expose this from Beam at the moment. > > You could implement this yourself by writing the value to state and also > writing the value out to memcache (or other distributed key+value store) > using window + user key as the memcache key but you may make the "state" > viewable before it becomes "committed" in a Runner since processing of the > bundle could be slow or could fail. If you need to have a point in time > view of what the runner knows then you would need support from runners to > be able to do this. > > > On Mon, Nov 4, 2019 at 11:46 AM Dengpan Y <bytesstr...@gmail.com> wrote: > >> Hi, I recently see several Beam users using Samza Runner asking for a >> tool/feature to query the internal state of PTransform, especially the >> DoFn, for instance, the user can use the @StateId to define named state, >> and the user would like to query the content of the state for a specific >> key of that state. >> >> Does Beam have any plan to support this feature from Beam SDK, such as >> develop a PTransform that can be wired into the pipeline so that the user >> can query the state on demand? >> Or If this has to be done on the Runner side, is there any best >> practice/idea that we can follow? >> >> Thanks >> Dengpan >> >>
