[jira] [Commented] (SPARK-19644) Memory leak in Spark Streaming (Encoder/Scala Reflection)

2017-11-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16247973#comment-16247973 ] Apache Spark commented on SPARK-19644: -- User 'zsxwing' has created a pull request fo

[jira] [Commented] (SPARK-19644) Memory leak in Spark Streaming (Encoder/Scala Reflection)

2017-11-07 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16243171#comment-16243171 ] Apache Spark commented on SPARK-19644: -- User 'zsxwing' has created a pull request fo

[jira] [Commented] (SPARK-19644) Memory leak in Spark Streaming (Encoder/Scala Reflection)

2017-11-01 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16234655#comment-16234655 ] Shixiong Zhu commented on SPARK-19644: -- I added more components since it also affect

[jira] [Commented] (SPARK-19644) Memory leak in Spark Streaming

2017-11-01 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16234643#comment-16234643 ] Shixiong Zhu commented on SPARK-19644: -- By the way, you can confirm this issue by ch

[jira] [Commented] (SPARK-19644) Memory leak in Spark Streaming

2017-11-01 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16234638#comment-16234638 ] Shixiong Zhu commented on SPARK-19644: -- I happened to investigate a similar issue an

[jira] [Commented] (SPARK-19644) Memory leak in Spark Streaming

2017-11-01 Thread deng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16233756#comment-16233756 ] deng commented on SPARK-19644: -- did the issue has been fixed? I am using spark 2.1.0 and i a

[jira] [Commented] (SPARK-19644) Memory leak in Spark Streaming

2017-06-18 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16053509#comment-16053509 ] Dongjoon Hyun commented on SPARK-19644: --- I see. Thank you anyway, [~deenbandhu]. >

[jira] [Commented] (SPARK-19644) Memory leak in Spark Streaming

2017-06-18 Thread Deenbandhu Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16053502#comment-16053502 ] Deenbandhu Agarwal commented on SPARK-19644: And the spark cassandra connecto

[jira] [Commented] (SPARK-19644) Memory leak in Spark Streaming

2017-06-18 Thread Deenbandhu Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16053498#comment-16053498 ] Deenbandhu Agarwal commented on SPARK-19644: yes i can try but is there any r

[jira] [Commented] (SPARK-19644) Memory leak in Spark Streaming

2017-06-18 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16053494#comment-16053494 ] Dongjoon Hyun commented on SPARK-19644: --- Hi, [~deenbandhu]. Could you try this with

[jira] [Commented] (SPARK-19644) Memory leak in Spark Streaming

2017-03-19 Thread Deenbandhu Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15931778#comment-15931778 ] Deenbandhu Agarwal commented on SPARK-19644: full gc is trigger so many times

[jira] [Commented] (SPARK-19644) Memory leak in Spark Streaming

2017-03-19 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15931774#comment-15931774 ] Sean Owen commented on SPARK-19644: --- Use jcmd to trigger a full GC on the process to se

[jira] [Commented] (SPARK-19644) Memory leak in Spark Streaming

2017-03-19 Thread Deenbandhu Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15931771#comment-15931771 ] Deenbandhu Agarwal commented on SPARK-19644: Yes i have tried restricting the

[jira] [Commented] (SPARK-19644) Memory leak in Spark Streaming

2017-03-19 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15931768#comment-15931768 ] Sean Owen commented on SPARK-19644: --- I don't see any GC time here. Is this not simple

[jira] [Commented] (SPARK-19644) Memory leak in Spark Streaming

2017-03-19 Thread Deenbandhu Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15931765#comment-15931765 ] Deenbandhu Agarwal commented on SPARK-19644: It's not even clear it has GCed

[jira] [Commented] (SPARK-19644) Memory leak in Spark Streaming

2017-03-18 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15931150#comment-15931150 ] Sean Owen commented on SPARK-19644: --- I don't think there is evidence of a memory leak h

[jira] [Commented] (SPARK-19644) Memory leak in Spark Streaming

2017-03-17 Thread Deenbandhu Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15929979#comment-15929979 ] Deenbandhu Agarwal commented on SPARK-19644: Any updates ?? > Memory leak in

[jira] [Commented] (SPARK-19644) Memory leak in Spark Streaming

2017-02-22 Thread Deenbandhu Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15879823#comment-15879823 ] Deenbandhu Agarwal commented on SPARK-19644: I am using scala 2.11 > Memory

[jira] [Commented] (SPARK-19644) Memory leak in Spark Streaming

2017-02-22 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15879397#comment-15879397 ] Shixiong Zhu commented on SPARK-19644: -- [~deenbandhu] Do you use Scala 2.10 or Scala

[jira] [Commented] (SPARK-19644) Memory leak in Spark Streaming

2017-02-21 Thread Deenbandhu Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15875702#comment-15875702 ] Deenbandhu Agarwal commented on SPARK-19644: I have analysed the issue more.

[jira] [Commented] (SPARK-19644) Memory leak in Spark Streaming

2017-02-20 Thread Deenbandhu Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15874614#comment-15874614 ] Deenbandhu Agarwal commented on SPARK-19644: Sorry for the delayed response.

[jira] [Commented] (SPARK-19644) Memory leak in Spark Streaming

2017-02-17 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15872672#comment-15872672 ] Shixiong Zhu commented on SPARK-19644: -- [~deenbandhu] Could you check the GC root, p

[jira] [Commented] (SPARK-19644) Memory leak in Spark Streaming

2017-02-17 Thread Deenbandhu Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15871528#comment-15871528 ] Deenbandhu Agarwal commented on SPARK-19644: We are not using any state or wi

[jira] [Commented] (SPARK-19644) Memory leak in Spark Streaming

2017-02-17 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15871519#comment-15871519 ] Sean Owen commented on SPARK-19644: --- This is still not necessarily true in general. You

[jira] [Commented] (SPARK-19644) Memory leak in Spark Streaming

2017-02-17 Thread Deenbandhu Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15871515#comment-15871515 ] Deenbandhu Agarwal commented on SPARK-19644: Yes that's right running out of

[jira] [Commented] (SPARK-19644) Memory leak in Spark Streaming

2017-02-17 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15871503#comment-15871503 ] Sean Owen commented on SPARK-19644: --- You only show 400MB of lists in your screen dump.

[jira] [Commented] (SPARK-19644) Memory leak in Spark Streaming

2017-02-17 Thread Deenbandhu Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15871494#comment-15871494 ] Deenbandhu Agarwal commented on SPARK-19644: And after 40-50 hours full gc is

[jira] [Commented] (SPARK-19644) Memory leak in Spark Streaming

2017-02-17 Thread Deenbandhu Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15871477#comment-15871477 ] Deenbandhu Agarwal commented on SPARK-19644: No i had just given 2 GB heap an

[jira] [Commented] (SPARK-19644) Memory leak in Spark Streaming

2017-02-17 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15871452#comment-15871452 ] Sean Owen commented on SPARK-19644: --- There are just linked list objects. Why are they t

[jira] [Commented] (SPARK-19644) Memory leak in Spark Streaming

2017-02-17 Thread Deenbandhu Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15871444#comment-15871444 ] Deenbandhu Agarwal commented on SPARK-19644: you can ignore that change in me

[jira] [Commented] (SPARK-19644) Memory leak in Spark Streaming

2017-02-17 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15871436#comment-15871436 ] Sean Owen commented on SPARK-19644: --- What you have described so far is not a memory lea