Re: Shared state between two process functions
Hi Maybe you can take a look at broadcast state[1] [1] https://ci.apache.org/projects/flink/flink-docs-release-1.10/dev/stream/state/broadcast_state.html Best, Congxian Robert Metzger 于2020年6月16日周二 上午2:18写道： > Thanks for sharing some details on the use case: Are you able to move the > common computation into one operator that runs before the ProcessFunctions, > and you are sending the results there? > You can build quite advanced dataflow graphs with Flink to model your > problem. > > On Mon, Jun 15, 2020 at 9:01 AM Jaswin Shah > wrote: > >> Basically, I have multiple processafunctions and they are doing >> some.computations based on some historical results and the historical >> events and results are common across the process functions due to which I >> have a lot of redundant processing in many process functions so, I have >> been thinking of some shared state accessible between multiple >> keyedprocessfunctions. >> >> Get Outlook for Android <https://aka.ms/ghei36> >> >> -- >> *From:* Yun Gao >> *Sent:* Monday, June 15, 2020 8:27:38 AM >> *To:* Jaswin Shah ; user@flink.apache.org < >> user@flink.apache.org> >> *Subject:* Re: Shared state between two process functions >> >> Hi Jaswin, >> >>Currently the state belongs to single operators, thus it should be not >> possible to share states between different operators. Could you also share >> the original problem want to solve by sharing states ? >> >> Best, >> Yun >> >> >> --Original Mail -- >> *Sender:*Jaswin Shah >> *Send Date:*Sun Jun 14 18:57:54 2020 >> *Recipients:*user@flink.apache.org >> *Subject:*Shared state between two process functions >> >> Hi, >> >> Is it possible to create the shared state(MapState) between two different >> keyedProcessFunction? If it's possible, how can we do that in flink? >> >> Thanks, >> Jaswin >> >>
Re: Shared state between two process functions
Thanks for sharing some details on the use case: Are you able to move the common computation into one operator that runs before the ProcessFunctions, and you are sending the results there? You can build quite advanced dataflow graphs with Flink to model your problem. On Mon, Jun 15, 2020 at 9:01 AM Jaswin Shah wrote: > Basically, I have multiple processafunctions and they are doing > some.computations based on some historical results and the historical > events and results are common across the process functions due to which I > have a lot of redundant processing in many process functions so, I have > been thinking of some shared state accessible between multiple > keyedprocessfunctions. > > Get Outlook for Android <https://aka.ms/ghei36> > > -- > *From:* Yun Gao > *Sent:* Monday, June 15, 2020 8:27:38 AM > *To:* Jaswin Shah ; user@flink.apache.org < > user@flink.apache.org> > *Subject:* Re: Shared state between two process functions > > Hi Jaswin, > >Currently the state belongs to single operators, thus it should be not > possible to share states between different operators. Could you also share > the original problem want to solve by sharing states ? > > Best, > Yun > > > --Original Mail -- > *Sender:*Jaswin Shah > *Send Date:*Sun Jun 14 18:57:54 2020 > *Recipients:*user@flink.apache.org > *Subject:*Shared state between two process functions > > Hi, > > Is it possible to create the shared state(MapState) between two different > keyedProcessFunction? If it's possible, how can we do that in flink? > > Thanks, > Jaswin > >
Re: Shared state between two process functions
Basically, I have multiple processafunctions and they are doing some.computations based on some historical results and the historical events and results are common across the process functions due to which I have a lot of redundant processing in many process functions so, I have been thinking of some shared state accessible between multiple keyedprocessfunctions. Get Outlook for Android<https://aka.ms/ghei36> From: Yun Gao Sent: Monday, June 15, 2020 8:27:38 AM To: Jaswin Shah ; user@flink.apache.org Subject: Re: Shared state between two process functions Hi Jaswin, Currently the state belongs to single operators, thus it should be not possible to share states between different operators. Could you also share the original problem want to solve by sharing states ? Best, Yun --Original Mail -- Sender:Jaswin Shah Send Date:Sun Jun 14 18:57:54 2020 Recipients:user@flink.apache.org Subject:Shared state between two process functions Hi, Is it possible to create the shared state(MapState) between two different keyedProcessFunction? If it's possible, how can we do that in flink? Thanks, Jaswin
Re: Shared state between two process functions
Hi Jaswin, Currently the state belongs to single operators, thus it should be not possible to share states between different operators. Could you also share the original problem want to solve by sharing states ? Best, Yun --Original Mail -- Sender:Jaswin Shah Send Date:Sun Jun 14 18:57:54 2020 Recipients:user@flink.apache.org Subject:Shared state between two process functions Hi, Is it possible to create the shared state(MapState) between two different keyedProcessFunction? If it's possible, how can we do that in flink? Thanks, Jaswin
Shared state between two process functions
Hi, Is it possible to create the shared state(MapState) between two different keyedProcessFunction? If it's possible, how can we do that in flink? Thanks, Jaswin
