[
https://issues.apache.org/jira/browse/FLINK-17865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yu Li updated FLINK-17865:
--------------------------
Release Note: After FLINK-17865 the default value of
`state.backend.fs.memory-threshold` has been increased from 1K to 20K to
prevent too many small files created on remote FS for small states. NOTICE:
jobs with large parallelism on source or stateful operators may have "JM OOM"
or "RPC message exceeding maximum frame size" problem with this change, and
please manually set the configuration back to 1K if you encounter such issues.
(was: After FLINK-17865 the the default value of
`state.backend.fs.memory-threshold` has been increased from 1K to 20K to
prevent too many small files created on remote FS for small states. NOTICE:
jobs with large parallelism on source or stateful operators may have "JM OOM"
or "RPC message exceeding maximum frame size" problem with this change, and
please manually set the configuration back to 1K if you encounter such issues. )
> Increase default size of `state.backend.fs.memory-threshold`
> ------------------------------------------------------------
>
> Key: FLINK-17865
> URL: https://issues.apache.org/jira/browse/FLINK-17865
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / State Backends
> Affects Versions: 1.11.0
> Reporter: Yu Li
> Assignee: Yun Tang
> Priority: Critical
> Labels: pull-request-available
> Fix For: 1.11.0
>
>
> As discussed in [ML thread|https://s.apache.org/us0en], we decided to
> increase the default value of `state.backend.fs.memory-threshold` from 1K to
> *20K* to prevent too many small files created on remote FS.
> We need to add an explicit release note to notify our users about this
> change, especially for those having jobs with large parallelism on source or
> stateful operators.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
