JoonPark1 opened a new pull request, #7227:
URL: https://github.com/apache/kyuubi/pull/7227
…cordingly once engine submit timeout is reached - prevent subsequent kyuubi
OOM
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines:
https://kyuubi.readthedocs.io/en/master/contributing/code/index.html
2. If the PR is related to an issue in
https://github.com/apache/kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title,
e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g.,
'[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### Why are the changes needed?
This PR change is to address bug #7226. It updates the behavior of updating
metadata store accordingly for batch jobs that have timed out due to waiting
for available spark driver engine. This will fix the subsequent restarted
kyuubi server from repeatedly polling for the spark application status of each
and every batch job, which can cause consecutive OOM errors under k8 cluster
deployment mode for kyuubi.
### How was this patch tested?
This patch was tested through integration test that was added to test suite
class called "SparkOnKubernetesTestsSuite.scala".
### Was this patch authored or co-authored using generative AI tooling?
No!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
