Do you have a event that would initiate the creation of the Side Input? If not you may want to look at this pattern as a way to trigger the API call in process time :
https://beam.apache.org/documentation/patterns/overview/ On Wed, 23 Oct 2019, 00:27 Maximilian Michels, <m...@apache.org> wrote: > Hi Jitendra, > > Side inputs are materialized based on their windowing. If you assign a > 10 second window to the side inputs, they can be renewed every 10 > seconds. Whenever you access the side input, the newest instance of the > side input will be retrieved. > > Cheers, > Max > > On 22.10.19 10:42, jitendra sharma wrote: > > Hi All, > > > > We have written a streaming application that will consume the Kafka > > event and will call the rest API for some validation information and > > passed as side input for validation for each event. > > > > I am just wondering how many times side input will be called for > > streaming applications as side input value is dynamic in nature(every 10 > > seconds)? I don't want to make calls to Rest API for each event. > > > > I am looking for at least every 10-second side input that would be > > refreshed. > > > > Any ideas/suggestions are appreciated. > > > > Regards, > > Jitendra Sharma >
