[
https://issues.apache.org/jira/browse/SPARK-47013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lingeshwaran Radhakrishnan updated SPARK-47013:
-----------------------------------------------
Summary: Document the config spark.sql.streaming.minBatchesToRetain (was:
Add the config spark.sql.streaming.minBatchesToRetain to the docs)
> Document the config spark.sql.streaming.minBatchesToRetain
> ----------------------------------------------------------
>
> Key: SPARK-47013
> URL: https://issues.apache.org/jira/browse/SPARK-47013
> Project: Spark
> Issue Type: Documentation
> Components: Structured Streaming
> Affects Versions: 3.5.0
> Reporter: Lingeshwaran Radhakrishnan
> Priority: Major
>
> Add the config spark.sql.streaming.minBatchesToRetain to the [streaming
> docs|https://spark.apache.org/docs/latest/configuration.html#spark-streaming]
> page which basically controls the minimum number of batches that must be
> retained and made recoverable.
> This would also help control the lifecycle of the state files held in the
> checkpoint folder i.e, State files are cleaned up based on the config
> spark.sql.streaming.minBatchesToRetain
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
