Thank you Frank and Dawid for providing the context here. From: Frank Dekervel <fr...@kapernikov.com> Date: Friday, February 4, 2022 at 9:56 AM To: user@flink.apache.org <user@flink.apache.org> Subject: Re: Queryable State Deprecation EXTERNAL SENDER ________________________________
Hello, To give an extra datapoint: after a not so successful experiment with faust-streaming we moved our application to flink. Since flinks queryable state was apparently stagnant, we implemented what was needed to sink the state to an external data store for querying. However, if queryable state was in good shape we would definately have used it. Making sure that the state is always reflected correctly in our external system turned out to be non-trivial for a number of reasons: our state is not trivially convertable to rows in a table, and sometimes we had (due to our own bugs, but still) inconsistencies between the internal flink state and the externally materialized state, especially after replaying from a checkpoint/savepoint after a crash (we cannot use exactly_once sinks in all occasions). Also, obviously, we could not use flinks partitioning/parallellism to help making state querying more scalable. Greetings, Frank On 04.02.22 14:06, Dawid Wysakowicz wrote: Hi Karthik, The reason we deprecated it is because we lacked committers who could spend time on getting the Queryable state to a production ready state. I might be speaking for myself here, but I think the main use case for the queryable state is to have an insight into the current state of the application for debugging purposes. If it is used for data serving purposes, we believe it's better to sink the data into an external store, which can provide better discoverability and more user friendly APIs for querying the results. As for debugging/tracking insights you may try to achieve similar results with metrics. Best, Dawid On 01/02/2022 16:36, Jatti, Karthik wrote: Hi, I see on the Flink Roadmap that Queryable state API is scheduled to be deprecated but I couldn’t find much information on confluence or this mailing group’s archives to understand the background as to why it’s being deprecated and what would be a an alternative. Any pointers to help me get some more information here would be great. Thanks, Karthik ________________________________ The information in the email message containing a link to this page, including any attachments thereto (collectively, “the e-mail”), is only for use by the intended recipient(s). The e-mail may contain information that is confidential, proprietary and/or privileged. If you have reason to believe that you are not the intended recipient, please notify the sender that you may have received this e-mail in error and delete all copies of it, including attachments, from your computer. Any viewing, copying, disclosure or distribution of this information by an unintended recipient is prohibited and by an intended recipient may be governed by arrangements in place between the sender’s and recipient’s respective firms. Eze Software does not represent that the e-mail is virus-free, complete or accurate. Eze Software accepts no liability for any damage sustained in connection with the content or transmission of the e-mail. ________________________________ Copyright © 2013 Eze Castle Software LLC. All Rights Reserved.
