Great question, one thing that we did not cover in the blog and I think we should have is the use case where you would want to bootstrap the pipeline.
One option would be on startup to have an extra bounded source that is read and flattened into the main pipeline, the source will need to contain values in Timestamped<V> format which would correspond to the first window that you would like to kickstart the process from. Will see if I can try and find some time to code up an example and add that and the looping timer code into the Beam patterns. https://beam.apache.org/documentation/patterns/overview/ Cheers Reza On Fri, 21 Jun 2019 at 07:59, Manu Zhang <[email protected]> wrote: > Indeed interesting pattern. > > One minor question. It seems the timer is triggered by the first element > so what if there is no data in the "first interval" ? > > Thanks for the write-up. > Manu > > On Wed, Jun 19, 2019 at 12:15 PM Reza Rokni <[email protected]> wrote: > >> Hi folks, >> >> Just wanted to drop a note here on a new pattern that folks may find >> interesting, called Looping Timers. It allows for default values to be >> created in interval windows in the absence of any external data coming into >> the pipeline. The details are in this blog below: >> >> https://beam.apache.org/blog/2019/06/11/looping-timers.html >> >> Its main utility is when dealing with time series data. There are still >> rough edges, like dealing with TTL and it would be great to hear >> feedback on ways it can be improved. >> >> The next pattern to publish in this domain will assist will hold and >> propagation of values from one interval window to the next, which coupled >> to looping timers starts to solve some interesting problems. >> >> Cheers >> >> Reza >> >> >> >> -- >> >> This email may be confidential and privileged. If you received this >> communication by mistake, please don't forward it to anyone else, please >> erase all copies and attachments, and please let me know that it has gone >> to the wrong person. >> >> The above terms reflect a potential business arrangement, are provided >> solely as a basis for further discussion, and are not intended to be and do >> not constitute a legally binding obligation. No legally binding obligations >> will be created, implied, or inferred until an agreement in final form is >> executed in writing by all parties involved. >> > -- This email may be confidential and privileged. If you received this communication by mistake, please don't forward it to anyone else, please erase all copies and attachments, and please let me know that it has gone to the wrong person. The above terms reflect a potential business arrangement, are provided solely as a basis for further discussion, and are not intended to be and do not constitute a legally binding obligation. No legally binding obligations will be created, implied, or inferred until an agreement in final form is executed in writing by all parties involved.
