[
https://issues.apache.org/jira/browse/SPARK-4999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14260812#comment-14260812
]
Saisai Shao commented on SPARK-4999:
------------------------------------
{code}
// Since storeInBlockManager = false, the storage level does not
matter.
new WriteAheadLogBackedBlockRDD[T](ssc.sparkContext,
blockIds, logSegments, storeInBlockManager = true,
StorageLevel.MEMORY_ONLY_SER)
} else {
{code}
The code is in ReceiverInputDStream.scala line 91. I think it would be more
reasonable to set storeInBlockManager = false by default.
> No need to put WAL-backed block into block manager by default
> -------------------------------------------------------------
>
> Key: SPARK-4999
> URL: https://issues.apache.org/jira/browse/SPARK-4999
> Project: Spark
> Issue Type: Bug
> Components: Streaming
> Affects Versions: 1.2.0
> Reporter: Saisai Shao
>
> Currently WAL-backed block is read out from HDFS and put into BlockManger
> with storage level MEMORY_ONLY_SER by default, since WAL-backed block is
> already fault-tolerant, no need to put into BlockManger again by default.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
