[
https://issues.apache.org/jira/browse/SPARK-23275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiao Li resolved SPARK-23275.
-----------------------------
Resolution: Fixed
Assignee: Dilip Biswal
Fix Version/s: 2.3.0
> hive/tests have been failing when run locally on the laptop (Mac) with OOM
> ---------------------------------------------------------------------------
>
> Key: SPARK-23275
> URL: https://issues.apache.org/jira/browse/SPARK-23275
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 2.3.1
> Reporter: Dilip Biswal
> Assignee: Dilip Biswal
> Priority: Major
> Fix For: 2.3.0
>
>
> hive tests have been failing when they are run locally (Mac Os) after a
> recent change in the trunk. After running the tests for some time, the test
> fails with OOM with Error: unable to create new native thread.
> I noticed the thread count goes all the way up to 2000+ after which we start
> getting these OOM errors. Most of the threads seem to be related to the
> connection pool in hive metastore (BoneCP-xxxxx-xxxx ). This behaviour change
> is happening after we made the following change to HiveClientImpl.reset()
> {code}
> def reset(): Unit = withHiveState {
> try {
> // code
> } finally {
> runSqlHive("USE default") ===> this is causing the issue
> }
> {code}
> I am proposing to temporarily back-out part of a fix made to address
> SPARK-23000 to resolve this issue while we work-out the exact reason for this
> sudden increase in thread counts.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
