Hi Rion,
Sorry for the late reply, another simpler method might indeed be in
initializeState,
the operator directly read the data from the kafka to initialize the state.
Best,
Yun
--Original Mail --
Sender:Rion Williams
Send Date:Mon May 17 19:53:35 2021
Hi Yun,
That’s very helpful and good to know that the problem/use-case has been thought
about. Since my need is probably shorter-term than later, I’ll likely need to
explore a workaround.
Do you know of an approach that might not require the use of check pointing and
restarting? I was looking
Hi Rion,
I think FLIP-150[1] should be able to solve this scenario.
Since FLIP-150 is still under discussion, for now a temporary method come
to me might be
1. Write a first job to read the kafka and update the broadcast state of some
operator. The job
would keep the source alive after all
Hey folks,
After digging into this a bit it does seem like Broadcast State would fit the
bill for this scenario and keeping the downstream operators up-to-date as
messages arrived in my Kafka topic.
My question is - is there a pattern for pre-populating the state initially? In
my case, I need
