[
https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16247973#comment-16247973
]
Apache Spark commented on SPARK-19644:
--
User 'zsxwing' has created a pull request for this issue:
[
https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16243171#comment-16243171
]
Apache Spark commented on SPARK-19644:
--
User 'zsxwing' has created a pull request for this issue:
[
https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16234655#comment-16234655
]
Shixiong Zhu commented on SPARK-19644:
--
I added more components since it also affects them. The
[
https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16234643#comment-16234643
]
Shixiong Zhu commented on SPARK-19644:
--
By the way, you can confirm this issue by checking if the
[
https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16234638#comment-16234638
]
Shixiong Zhu commented on SPARK-19644:
--
I happened to investigate a similar issue and found out the
[
https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16233756#comment-16233756
]
deng commented on SPARK-19644:
--
did the issue has been fixed? I am using spark 2.1.0 and i also encount the
[
https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16053509#comment-16053509
]
Dongjoon Hyun commented on SPARK-19644:
---
I see. Thank you anyway, [~deenbandhu].
> Memory leak in
[
https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16053502#comment-16053502
]
Deenbandhu Agarwal commented on SPARK-19644:
And the spark cassandra connector is also not
[
https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16053498#comment-16053498
]
Deenbandhu Agarwal commented on SPARK-19644:
yes i can try but is there any report of such
[
https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16053494#comment-16053494
]
Dongjoon Hyun commented on SPARK-19644:
---
Hi, [~deenbandhu].
Could you try this with the latest
[
https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15931778#comment-15931778
]
Deenbandhu Agarwal commented on SPARK-19644:
full gc is trigger so many times and its
[
https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15931774#comment-15931774
]
Sean Owen commented on SPARK-19644:
---
Use jcmd to trigger a full GC on the process to see what happens.
[
https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15931771#comment-15931771
]
Deenbandhu Agarwal commented on SPARK-19644:
Yes i have tried restricting the number of jobs
[
https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15931768#comment-15931768
]
Sean Owen commented on SPARK-19644:
---
I don't see any GC time here.
Is this not simple stuff like you
[
https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15931765#comment-15931765
]
Deenbandhu Agarwal commented on SPARK-19644:
It's not even clear it has GCed ?
The increase
[
https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15931150#comment-15931150
]
Sean Owen commented on SPARK-19644:
---
I don't think there is evidence of a memory leak here. It's not
[
https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15929979#comment-15929979
]
Deenbandhu Agarwal commented on SPARK-19644:
Any updates ??
> Memory leak in Spark Streaming
[
https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15879823#comment-15879823
]
Deenbandhu Agarwal commented on SPARK-19644:
I am using scala 2.11
> Memory leak in Spark
[
https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15879397#comment-15879397
]
Shixiong Zhu commented on SPARK-19644:
--
[~deenbandhu] Do you use Scala 2.10 or Scala 2.11?
> Memory
[
https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15875702#comment-15875702
]
Deenbandhu Agarwal commented on SPARK-19644:
I have analysed the issue more. I performed some
[
https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15874614#comment-15874614
]
Deenbandhu Agarwal commented on SPARK-19644:
Sorry for the delayed response.
No, I didn't
[
https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15872672#comment-15872672
]
Shixiong Zhu commented on SPARK-19644:
--
[~deenbandhu] Could you check the GC root, please? These
[
https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15871528#comment-15871528
]
Deenbandhu Agarwal commented on SPARK-19644:
We are not using any state or window operation
[
https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15871519#comment-15871519
]
Sean Owen commented on SPARK-19644:
---
This is still not necessarily true in general. Your app could be
[
https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15871515#comment-15871515
]
Deenbandhu Agarwal commented on SPARK-19644:
Yes that's right running out of memory doesn't
[
https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15871503#comment-15871503
]
Sean Owen commented on SPARK-19644:
---
You only show 400MB of lists in your screen dump.
Running out of
[
https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15871494#comment-15871494
]
Deenbandhu Agarwal commented on SPARK-19644:
And after 40-50 hours full gc is too frequent
[
https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15871477#comment-15871477
]
Deenbandhu Agarwal commented on SPARK-19644:
No i had just given 2 GB heap and if there is
[
https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15871452#comment-15871452
]
Sean Owen commented on SPARK-19644:
---
There are just linked list objects. Why are they too high? if you
[
https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15871444#comment-15871444
]
Deenbandhu Agarwal commented on SPARK-19644:
you can ignore that change in memory but if you
[
https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15871436#comment-15871436
]
Sean Owen commented on SPARK-19644:
---
What you have described so far is not a memory leak in Spark. It's
31 matches
Mail list logo
