[
https://issues.apache.org/jira/browse/SPARK-13804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Nguyen updated SPARK-13804:
-----------------------------------
Description:
1. HiveThriftServer2 was started with startWithContext
2. Multiple temp tables were loaded and registered via registerTempTable .
3. HiveThriftServer2 was accessed via JDBC to access to those tables.
4. Some temp tables were dropped via
hiveContext.dropTempTable(registerTableName); and reloaded to refresh their
data. There are 1 to 7 million rows in these tables.
5. The same queries ran in step 3 were re-ran over the existing JDBC
connection. This time HiveThriftServer2 receives those queries but at times
HiveThriftServer2 hangs and does not return the results. CPU utilization on
both Spark driver and child nodes was around 1%. 10GB of RAM was used out of
30GB on the driver, and 3GB of RAM out of 30GB was used on the child nodes. So
there was no resource starvation.
6. Wait about 5 minutes and rerun the same queries in step 5, and this time,
HiveThriftServer2 returns the results of those queries fine.
This issue occurs intermittently when the steps 1-5 are repeated, so it may
take several attempts to reproduce this issue.
was:
1. HiveThriftServer2 was started with startWithContext
2. Multiple temp tables were loaded and registered via registerTempTable .
3. HiveThriftServer2 was accessed via JDBC to access to those tables.
4. Some temp tables were dropped via
hiveContext.dropTempTable(registerTableName); and reloaded to refresh their
data. There are 1 to 7 million rows in these tables.
5. The same queries ran in step 3 were re-ran over the existing JDBC
connection. This time HiveThriftServer2 receives those queries but at times
HiveThriftServer2 hangs and does not return the results. CPU utilization on
both Spark driver and child nodes was around 1%. 1GB of RAM is used out of
30GB. So there was no resource starvation.
6. Wait about 5 minutes and rerun the same queries in step 5, and this time,
HiveThriftServer2 returns the results of those queries fine.
This issue occurs intermittently when the steps 1-5 are repeated, so it may
take several attempts to reproduce this issue.
> HiveThriftServer2 hangs intermittently
> --------------------------------------
>
> Key: SPARK-13804
> URL: https://issues.apache.org/jira/browse/SPARK-13804
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 1.6.0
> Reporter: Michael Nguyen
>
> 1. HiveThriftServer2 was started with startWithContext
> 2. Multiple temp tables were loaded and registered via registerTempTable .
> 3. HiveThriftServer2 was accessed via JDBC to access to those tables.
> 4. Some temp tables were dropped via
> hiveContext.dropTempTable(registerTableName); and reloaded to refresh their
> data. There are 1 to 7 million rows in these tables.
> 5. The same queries ran in step 3 were re-ran over the existing JDBC
> connection. This time HiveThriftServer2 receives those queries but at times
> HiveThriftServer2 hangs and does not return the results. CPU utilization on
> both Spark driver and child nodes was around 1%. 10GB of RAM was used out of
> 30GB on the driver, and 3GB of RAM out of 30GB was used on the child nodes.
> So there was no resource starvation.
> 6. Wait about 5 minutes and rerun the same queries in step 5, and this time,
> HiveThriftServer2 returns the results of those queries fine.
> This issue occurs intermittently when the steps 1-5 are repeated, so it may
> take several attempts to reproduce this issue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
