[
https://issues.apache.org/jira/browse/FLINK-33819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17805154#comment-17805154
]
Hangxiang Yu commented on FLINK-33819:
--------------------------------------
{quote}It is obvious that closing Compression is not only about benefits, but
also brings some space amplification. What I want to express is that we may
need to provide such a configuration for users to balance how to exchange space
for time
{quote}
Yeah, that's also what we found in the production environment which could
improve the performance when we use NoCompression for L0 and L1 at the expense
of space.
So I'm +1 to introduce such a configuration. Of course, we should remain the
default behavious at the first version.
{quote}After switching the Compression Type, the newly generated file will be
compressed using the new Compress Type, and the existing file can still be read
and written with old Compress Type.
{quote}
Yes, you're right. I have verified this. RocksDB will handle this situation.
[~mayuehappy] Would you like to take this ?
> Support setting CompressType in RocksDBStateBackend
> ---------------------------------------------------
>
> Key: FLINK-33819
> URL: https://issues.apache.org/jira/browse/FLINK-33819
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / State Backends
> Affects Versions: 1.18.0
> Reporter: Yue Ma
> Priority: Major
> Fix For: 1.19.0
>
> Attachments: image-2023-12-14-11-32-32-968.png,
> image-2023-12-14-11-35-22-306.png
>
>
> Currently, RocksDBStateBackend does not support setting the compression
> level, and Snappy is used for compression by default. But we have some
> scenarios where compression will use a lot of CPU resources. Turning off
> compression can significantly reduce CPU overhead. So we may need to support
> a parameter for users to set the CompressType of Rocksdb.
> !image-2023-12-14-11-35-22-306.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
