yaooqinn opened a new issue #1960: URL: https://github.com/apache/incubator-kyuubi/issues/1960
### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) ### Search before asking - [X] I have searched in the [issues](https://github.com/apache/incubator-kyuubi/issues?q=is%3Aissue) and found no similar issues. ### What would you like to be improved? When a resource manager such as YARN is in high traffic, many engines will be pending and fail to get initialized in time. But sometimes, the pending engine is still waiting for YARN to accept it and run it. In the meanwhile, it already becomes orphaned and may get a re-submit at the server-side. This may become an endless loop. We should avoid these kinds of engines be running. ### How should we improve? One way to fix this is to add a timestamp that indicates the submit time of an engine, then when we actually before we actually create the SparkSession instance, we check whether the `current time - submit time` already exceeds the maximum engine initialization time. If true, we skip; otherwise, we go the current way ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
