Hi Ben, This is only for testing purposes. The runner has its own policy to handle watermarks. For example, it may decide to drop some watermarks if updates are too frequent. I was thinking about converting the watermark policy into an application-visible behavior and use the PAssert to check the result. So, what comes to mind is to use a global window and a custom trigger that fires on every watermark advances. So that each watermark becomes an output tuple. If it is too much trouble, I can work around it by testing in lower levels.
Thanks, Shen On Wed, Jun 7, 2017 at 12:35 PM, Ben Chambers <[email protected]> wrote: > There hasn't been a need for user defined triggers and we've found it is > really hard to create triggers with proper behavior. > > Can you elaborate on why you are trying to use triggers to understand > watermarks in this way? It's not clear how this trigger would be useful to > that understanding. > > On Wed, Jun 7, 2017, 10:22 AM Shen Li <[email protected]> wrote: > > > Hi Lukasz, > > > > Thanks again for the suggestion. Is there any reason for not allowing > users > > create custom triggers? > > > > Shen > > > > On Wed, Jun 7, 2017 at 12:13 PM, Lukasz Cwik <[email protected]> > > wrote: > > > > > You should really take a look at TestStream and have runners integrate > > with > > > it instead. > > > > > > There are already several tests which validate TestStream compatible > > > runners to make sure their trigger evaluations are correct. > > > > > > On Wed, Jun 7, 2017 at 10:10 AM, Shen Li <[email protected]> wrote: > > > > > > > Hi Lukasz, > > > > > > > > Thanks for the suggestion. I am trying to test how the runner > generates > > > > watermarks. So I would like to have the trigger to fire on every > > > watermark > > > > advancing event. > > > > > > > > Shen > > > > > > > > On Wed, Jun 7, 2017 at 10:49 AM, Lukasz Cwik > <[email protected] > > > > > > > wrote: > > > > > > > > > Look into the AfterPane#elementCountAtLeast trigger as it seems to > be > > > the > > > > > closest to your description. It fires as soon as any data is > > available. > > > > > > > > > > Are you sure you don't want some kind of watermark based trigger > with > > > > just > > > > > with a small interval size? > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Jun 7, 2017 at 8:22 AM, Shen Li <[email protected]> > wrote: > > > > > > > > > > > Hi Lukasz, > > > > > > > > > > > > Thanks for your response. Is it possible to implement the > following > > > > logic > > > > > > using existing triggers: always fire the trigger on a > GlobalWindow > > > > > whenever > > > > > > watermark advances? > > > > > > > > > > > > Shen > > > > > > > > > > > > On Wed, Jun 7, 2017 at 10:05 AM, Lukasz Cwik > > > <[email protected] > > > > > > > > > > > wrote: > > > > > > > > > > > > > Users are unable to create custom trigger implementations. If > you > > > > tell > > > > > us > > > > > > > what you want your trigger to do, we may be able to suggest an > > > > > > alternative. > > > > > > > > > > > > > > On Wed, Jun 7, 2017 at 7:41 AM, Shen Li <[email protected]> > > > wrote: > > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > > > I created a custom trigger class (XYZ) by extending the > > > > OnceTrigger. > > > > > > > During > > > > > > > > execution, I got this error "Cannot translate trigger class > XYZ > > > to > > > > a > > > > > > > > runner-API proto." It seems that the Triggers.ProtoConverter > > > class > > > > > > needs > > > > > > > to > > > > > > > > declare a convertSpecific method for my trigger XYZ. How can > I > > > use > > > > my > > > > > > > > custom trigger without modifying Beam's Triggers class? > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > Shen > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
