Mukul Murthy created SPARK-24525:
------------------------------------
Summary: Provide an option to limit MemorySink memory usage
Key: SPARK-24525
URL: https://issues.apache.org/jira/browse/SPARK-24525
Project: Spark
Issue Type: Improvement
Components: Structured Streaming
Affects Versions: 2.3.1
Reporter: Mukul Murthy
MemorySink stores stream results in memory and is mostly used for testing and
displaying streams, but for large streams, this can OOM the driver. We should
add an option to limit the number of rows and the total size of a memory sink
and not add any new data once either limit is hit.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
