pan3793 commented on issue #32: URL: https://github.com/apache/incubator-kyuubi/issues/32#issuecomment-897378601
@iodone, thanks for sharing your thoughts. And I will give my views on the disadvantages you mentioned of solution 1. > 1. Hidden Session to the user, Session resource release timing is uncertain, may lead to session resource leakage. By per query per session mechanism and the mechanism of timing checks to avoid? I'm not proposing to drop the session_id concept in REST API, make session_id optional means the user can still provide a session_id. The session is not hidden for users, we can still keep the REST API of sessions, and return the implicitly created session_id to the client to make sure the user can reuse/release the session in the following request. > 2. You can see that the HTTP API is a short connection, while Kyuubi and Kyuubi engine directly establish a long link, which means that a Session state is maintained in memory to maintain the connection with Kyuubi Engine. We know that at least two HTTP API requests are required to execute a complete HTTP API SQL query. When Kyuubi is extended to multiple instances, since the HTTP API can be distributed to any instance, it is possible that the first two HTTP API requests will be distributed to different instances of Kyuubi, so that the second HTTP API request will not find the correct Session because the Session is stored on the other instance. Possible need to introduce a Session sharing mechanism (introducing a three-party persistent storage component)? The key point here I think is how to share states between Kyuubi Servers. We already have a discussion on it, the basic idea is 1. store the state in shared storage(e.g. Zookeeper, MySQL, Redis) rather than in memory. 2. move some states from Kyuubi Server to the engine side, suppose Kyuubi Server only keep and share the `session_id` and `engine instance` mapping, any Kyuubi Server can establish the thrift connection to the target engine when HTTP request arrived. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
