Saisai Shao created SPARK-5372:
----------------------------------
Summary: Change the default storage level of window operators
Key: SPARK-5372
URL: https://issues.apache.org/jira/browse/SPARK-5372
Project: Spark
Issue Type: Bug
Components: Streaming
Affects Versions: 1.2.0
Reporter: Saisai Shao
Current storage level of window operators is MEMORY_ONLY_SER, if the memory is
not enough to hold all the window data, cached RDD will be discarded, which
will lead to unexpected behavior.
Besides the default storage level of input data is MEMORY_AND_DISK_SER_2, it is
better to align to this storage level to change the storage level of window
operators to MEMORY_AND_DISK_SER.
This changing has no effect when memory is enough. So I'd propose to change the
default storage level to MEMORY_AND_DISK_SER.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
