Hi vino, +1 for improvement on queryable state feature. This reminds me of the state-processing-api module, which is very helpful when we analyze state in offline. However currently we don’t have many ways to know what is happening about the state inside a running application, which makes me feel that this has a good potential. Since these two modules are seperate but doing the similar work(anaylyzing state), maybe we have to think more about their orientation, or maybe integrate them in a graceful way in the future. Anyway, this is a great work and it’d be better if we can hear more thoughts and use cases.
Best Regards, Jiayi Liao Original Message Sender: vino yang<yanghua1...@gmail.com> Recipient: dev@flink.apache.org<dev@flink.apache.org> Date: Tuesday, Oct 22, 2019 15:42 Subject: [DISCUSS] Introduce a location-oriented two-stage query mechanism toimprove the queryable state. Hi guys, Currently, queryable state's client is hard to use. Because it requires users to know the address of TaskManager and the port of the proxy. Actually, most users who do not have good knowledge about the Flink's inner and runtime in production. The queryable state clients directly interact with query state client proxies which host on each TaskExecutor. This design requires users to know too much detail. We introduce a location service component to improve the architecture of the queryable state and hide the details of the task executors. We first give a brief introduction to our design in Section 2 and then detail the implementation in Section 3. At last, we describe some future work that can be done. I have given an initialized implementation in my Flink repository[2]. One thing that needs to be stated is that we have not changed the existing solution, so it still works according to the previous modes. The design documentation is here[3]. Any suggestion and feedback are welcome and appriciated. [1]: https://statefun.io/ [2]: https://github.com/yanghua/flink/tree/improve-queryable-state-master [3]: https://docs.google.com/document/d/181qYVIiHQGrc3hCj3QBn1iEHF4bUztdw4XO8VSaf_uI/edit?usp=sharing Best, Vino
