pan3793 opened a new pull request, #6998:
URL: https://github.com/apache/kyuubi/pull/6998
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines:
https://kyuubi.readthedocs.io/en/master/contributing/code/index.html
2. If the PR is related to an issue in
https://github.com/apache/kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title,
e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g.,
'[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### Why are the changes needed?
<!--
Please clarify why the changes are needed. For instance,
1. If you propose a new API, clarify the use case for a new API.
2. If you fix a bug, you can clarify why it is a bug, and what versions
are affected.
-->
Fix the missing `assert` in `SparkProcessBuilderSuite - spark process
builder`.
Fix the flaky test `SparkProcessBuilderSuite - capture error from spark
process builder` by increasing `kyuubi.session.engine.startup.maxLogLines` from
10 to 4096, this is easy to fail, especially in Spark 4.0 due to increased
error stack trace. for example,
https://github.com/apache/kyuubi/actions/runs/13974413470/job/39290129824
```
SparkProcessBuilderSuite:
- spark process builder
- capture error from spark process builder *** FAILED ***
The code passed to eventually never returned normally. Attempted 167 times
over 1.5007926256666668 minutes. Last failure message:
"org.apache.kyuubi.KyuubiSQLException: Suppressed:
org.apache.spark.util.Utils$OriginalTryStackTraceException: Full stacktrace of
original doTryWithCallerStacktrace caller
See more:
/home/runner/work/kyuubi/kyuubi/kyuubi-server/target/work/kentyao/kyuubi-spark-sql-engine.log.2
at
org.apache.kyuubi.KyuubiSQLException$.apply(KyuubiSQLException.scala:69)
at
org.apache.kyuubi.engine.ProcBuilder.$anonfun$start$1(ProcBuilder.scala:239)
at java.base/java.lang.Thread.run(Thread.java:1583)
.
FYI: The last 10 line(s) of log are:
25/03/24 12:53:39 INFO MemoryStore: MemoryStore started with capacity
434.4 MiB
25/03/24 12:53:39 INFO MemoryStore: MemoryStore cleared
25/03/24 12:53:39 INFO BlockManager: BlockManager stopped
25/03/24 12:53:39 INFO BlockManagerMaster: BlockManagerMaster stopped
25/03/24 12:53:39 INFO
OutputCommitCoordinator$OutputCommitCoordinatorEndpoint:
OutputCommitCoordinator stopped!
25/03/24 12:53:39 INFO SparkContext: Successfully stopped SparkContext
25/03/24 12:53:39 INFO ShutdownHookManager: Shutdown hook called
25/03/24 12:53:39 INFO ShutdownHookManager: Deleting directory
/tmp/spark-18455622-344e-48ac-92eb-4b368c35e697
25/03/24 12:53:39 INFO ShutdownHookManager: Deleting directory
/home/runner/work/kyuubi/kyuubi/kyuubi-server/target/work/kentyao/artifacts/spark-7479249b-44a2-4fe5-aa0f-544074f9c356
25/03/24 12:53:39 INFO ShutdownHookManager: Deleting directory
/tmp/spark-5ba8250f-1ff2-4e0d-a365-27d7518308e1" did not contain
"org.apache.hadoop.hive.ql.metadata.HiveException:".
(SparkProcessBuilderSuite.scala:77)
```
### How was this patch tested?
<!--
If tests were added, say they were added here. Please make sure to add some
test cases that check the changes thoroughly including negative and positive
cases if possible.
If it was tested in a way different from regular unit tests, please clarify
how you tested step by step, ideally copy and paste-able, so that other
reviewers can test and check, and descendants can verify in the future.
If tests were not added, please describe why they were not added and/or why
it was difficult to add.
-->
Pass GHA, and verified locally with Spark 4.0.0 RC3 by running tests 10
times with constant success.
### Was this patch authored or co-authored using generative AI tooling?
<!--
If a generative AI tooling has been used in the process of authoring this
patch, please include
phrase 'Generated-by: ' followed by the name of the tool and its version.
If no, write 'No'.
Please refer to the [ASF Generative Tooling
Guidance](https://www.apache.org/legal/generative-tooling.html) for details.
-->
No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
