Hi Yu, Excuse my late reply... We simply want Flink to be our centralized stream analysis platform, where we 1) consume data, 2) generate analysis, 3) present analysis. I honestly don't want "stream analysis" to spill out to other components in our ecosystem (e.g., sinking insights into a DB-like place).
So the case for QS for us is centralization of input, output, presentation. State Processor API for instance also counts as a presentation tool for us (on top of migration tool). This kind of all-in-one (in, out, ui) packaging helps with small teams. --- Oytun Tez *M O T A W O R D* The World's Fastest Human Translation Platform. oy...@motaword.com — www.motaword.com On Wed, Aug 14, 2019 at 3:13 AM Yu Li <car...@gmail.com> wrote: > Good to know your thoughts and the coming talk in Flink Forward Berlin > Oytun, interesting topic and great job! And it's great to hear the voice > from application perspective. > > Could you share, if possible, the reason why you need to open the state to > outside instead of writing the result to sink and directly query there? In > another thread there's a case that sink writes to different kafka topics so > state is the only place to get a global view, is this the same case you're > facing? Or some different requirements? I believe more attention will be > drawn to QS if more and more user requirements emerge (smile). > > Thanks. > > Best Regards, > Yu > > > On Wed, 14 Aug 2019 at 00:50, Oytun Tez <oy...@motaword.com> wrote: > >> Thank you for the honest response, Yu! >> >> There is so much that comes to mind when we look at Flink as a >> "application framework" (my talk >> <https://europe-2019.flink-forward.org/conference-program#not-so-big-%E2%80%93-flink-as-a-true-application-framework> >> in Flink Forward in Berlin will be about this). QS is one of them >> (need-wise, not QS itself necessarily). I opened the design doc Vino Yang >> created, I'll try to contribute to that. >> >> Meanwhile, for the sake of opening our state to outside, we will put a >> stupid simple operator in between to keep a *duplicate* of the state... >> >> Thanks again! >> >> >> >> >> >> --- >> Oytun Tez >> >> *M O T A W O R D* >> The World's Fastest Human Translation Platform. >> oy...@motaword.com — www.motaword.com >> >> >> On Tue, Aug 13, 2019 at 6:29 PM Yu Li <car...@gmail.com> wrote: >> >>> Hi Oytun, >>> >>> Sorry but TBH such support will probably not be added in the foreseeable >>> future due to lack of committer bandwidth (not only support queryable >>> broadcast state but all about QueryableState module) as pointed out in >>> other threads [1] [2]. >>> >>> However, I think you could open a JIRA for this so when things changed >>> it could be taken care of. Thanks. >>> >>> [1] https://s.apache.org/MaOl >>> [2] https://s.apache.org/r8k8a >>> >>> Best Regards, >>> Yu >>> >>> >>> On Tue, 13 Aug 2019 at 20:34, Oytun Tez <oy...@motaword.com> wrote: >>> >>>> Hi there, >>>> >>>> Can we set a broadcast state as queryable? I've looked around, not much >>>> to find about it. I am receiving UnknownKvStateLocation when I try to query >>>> with the descriptor/state name I give to the broadcast state. >>>> >>>> If it doesn't work, what could be the alternative? My mind goes around >>>> ctx.getBroadcastState and making it queryable in the operator level (I'd >>>> rather not). >>>> >>>> --- >>>> Oytun Tez >>>> >>>> *M O T A W O R D* >>>> The World's Fastest Human Translation Platform. >>>> oy...@motaword.com — www.motaword.com >>>> >>>
