dingdong created SPARK-22831:
--------------------------------
Summary: Long running spark app gets slower and slower over
time,Maybe cause by unsafe/Platform
Key: SPARK-22831
URL: https://issues.apache.org/jira/browse/SPARK-22831
Project: Spark
Issue Type: Bug
Components: Java API, Spark Core, SQL
Affects Versions: 2.1.0
Environment: * Linux
* yarn-client
* jdk7
*
Reporter: dingdong
I have a long running spark app running on yarn.I load the data which is about
150G from hbase and cached it in the memory.
This app provide ad-hoc query with sparkSql.At the beginning,the spark is very
fast that just take a few seconds per job.
But after a few hours later,the spark running slower and slower until take over
one minute per job.After profiling with jvisualvm,I found that the methods of
unsafe/platform (Platform.getLong etc) take most of the time.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
