[jira] [Assigned] (SPARK-13845) Driver OOM after few days when running streaming
[ https://issues.apache.org/jira/browse/SPARK-13845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13845: Assignee: (was: Apache Spark) > Driver OOM after few days when running streaming > > > Key: SPARK-13845 > URL: https://issues.apache.org/jira/browse/SPARK-13845 > Project: Spark > Issue Type: Bug > Components: Spark Core >Affects Versions: 1.5.2, 1.6.1 >Reporter: jeanlyn > > We have a streaming job using *FlumePollInputStream* always driver OOM after > few days, here is some driver heap dump before OOM > {noformat} > num #instances #bytes class name > -- >1: 13845916 553836640 org.apache.spark.storage.BlockStatus >2: 14020324 336487776 org.apache.spark.storage.StreamBlockId >3: 13883881 333213144 scala.collection.mutable.DefaultEntry >4: 8907 89043952 [Lscala.collection.mutable.HashEntry; >5: 62360 65107352 [B >6:163368 24453904 [Ljava.lang.Object; >7:293651 20342664 [C > ... > {noformat} > *BlockStatus* and *StreamBlockId* keep on growing, and the driver OOM in the > end. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-13845) Driver OOM after few days when running streaming
[ https://issues.apache.org/jira/browse/SPARK-13845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13845: Assignee: Apache Spark > Driver OOM after few days when running streaming > > > Key: SPARK-13845 > URL: https://issues.apache.org/jira/browse/SPARK-13845 > Project: Spark > Issue Type: Bug > Components: Spark Core >Affects Versions: 1.5.2, 1.6.1 >Reporter: jeanlyn >Assignee: Apache Spark > > We have a streaming job using *FlumePollInputStream* always driver OOM after > few days, here is some driver heap dump before OOM > {noformat} > num #instances #bytes class name > -- >1: 13845916 553836640 org.apache.spark.storage.BlockStatus >2: 14020324 336487776 org.apache.spark.storage.StreamBlockId >3: 13883881 333213144 scala.collection.mutable.DefaultEntry >4: 8907 89043952 [Lscala.collection.mutable.HashEntry; >5: 62360 65107352 [B >6:163368 24453904 [Ljava.lang.Object; >7:293651 20342664 [C > ... > {noformat} > *BlockStatus* and *StreamBlockId* keep on growing, and the driver OOM in the > end. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org