[jira] [Commented] (FLINK-24046) Refactor the relationship bwtween PredefinedOptions and RocksDBConfigurableOptions
[
https://issues.apache.org/jira/browse/FLINK-24046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17438531#comment-17438531
]
Zakelly Lan commented on FLINK-24046:
-
I'm going to fix it this way:
# Provide a default value for each option in {{RocksDBConfigurableOptions}};
# Refactor {{PredefinedOptions}}, make each element of this enum carry a config
map that will override the default value of specified keys.
# Change the {{RocksDBResourceContainer#getDbOptions}} and
{{RocksDBResourceContainer#getColumnOptions}}, load config values from the
provided ReadableConfig and specified PredefinedOptions. Override relationship:
user specified > predefined options > default value. Then we could remove the
{{DefaultConfigurableOptionsFactory}}.
# Keep the {{ConfigurableRocksDBOptionsFactory}} interface and load the
user-defined options factory, and call corresponding methods to apply
customized RocksDB options after loading from the config.
> Refactor the relationship bwtween PredefinedOptions and
> RocksDBConfigurableOptions
> --
>
> Key: FLINK-24046
> URL: https://issues.apache.org/jira/browse/FLINK-24046
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / State Backends
>Reporter: Yun Tang
>Assignee: Zakelly Lan
>Priority: Major
> Fix For: 1.14.1
>
>
> RocksDBConfigurableOptions mainly focus on the settings of DBOptions and
> ColumnFamilyOptions. The original design of this class is used to let user
> could configure RocksDB via configurations instead of programmatically
> implemented RocksDBOptionsFactory.
> To make the minimal change, original options in RocksDBConfigurableOptions
> have no default value so that we would not make anything happen in
> DefaultConfigurableOptionsFactory just as before.
> However, this make user not so clear of the option meaning with no default
> value, and we could consider change the relationship between them.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-24046) Refactor the relationship bwtween PredefinedOptions and RocksDBConfigurableOptions
[
https://issues.apache.org/jira/browse/FLINK-24046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17439002#comment-17439002
]
Yun Tang commented on FLINK-24046:
--
[~Zakelly] Since {{DefaultConfigurableOptionsFactory}} had existed in
[documentation|https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/ops/state/state_backends/#passing-options-factory-to-rocksdb]
for long time. I don't think we should remove it directly to backward
compatibility.
> Refactor the relationship bwtween PredefinedOptions and
> RocksDBConfigurableOptions
> --
>
> Key: FLINK-24046
> URL: https://issues.apache.org/jira/browse/FLINK-24046
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / State Backends
>Reporter: Yun Tang
>Assignee: Zakelly Lan
>Priority: Major
> Fix For: 1.14.1
>
>
> RocksDBConfigurableOptions mainly focus on the settings of DBOptions and
> ColumnFamilyOptions. The original design of this class is used to let user
> could configure RocksDB via configurations instead of programmatically
> implemented RocksDBOptionsFactory.
> To make the minimal change, original options in RocksDBConfigurableOptions
> have no default value so that we would not make anything happen in
> DefaultConfigurableOptionsFactory just as before.
> However, this make user not so clear of the option meaning with no default
> value, and we could consider change the relationship between them.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-24046) Refactor the relationship bwtween PredefinedOptions and RocksDBConfigurableOptions
[
https://issues.apache.org/jira/browse/FLINK-24046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17439015#comment-17439015
]
Zakelly Lan commented on FLINK-24046:
-
[~yunta] Good point. I think we could treat
'org.apache.flink.contrib.streaming.state.DefaultConfigurableOptionsFactory' as
a special value for the config 'state.backend.rocksdb.options-factory'. If user
set this, we does not load any option factory and just load config values from
the provided ReadableConfig and specified PredefinedOptions. It is the same
behavior as the {{DefaultConfigurableOptionsFactory}} does before.
> Refactor the relationship bwtween PredefinedOptions and
> RocksDBConfigurableOptions
> --
>
> Key: FLINK-24046
> URL: https://issues.apache.org/jira/browse/FLINK-24046
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / State Backends
>Reporter: Yun Tang
>Assignee: Zakelly Lan
>Priority: Major
> Fix For: 1.14.1
>
>
> RocksDBConfigurableOptions mainly focus on the settings of DBOptions and
> ColumnFamilyOptions. The original design of this class is used to let user
> could configure RocksDB via configurations instead of programmatically
> implemented RocksDBOptionsFactory.
> To make the minimal change, original options in RocksDBConfigurableOptions
> have no default value so that we would not make anything happen in
> DefaultConfigurableOptionsFactory just as before.
> However, this make user not so clear of the option meaning with no default
> value, and we could consider change the relationship between them.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-24046) Refactor the relationship bwtween PredefinedOptions and RocksDBConfigurableOptions
[ https://issues.apache.org/jira/browse/FLINK-24046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17439017#comment-17439017 ] Zakelly Lan commented on FLINK-24046: - Moreover, we could remove this special value since 15.0 > Refactor the relationship bwtween PredefinedOptions and > RocksDBConfigurableOptions > -- > > Key: FLINK-24046 > URL: https://issues.apache.org/jira/browse/FLINK-24046 > Project: Flink > Issue Type: Improvement > Components: Runtime / State Backends >Reporter: Yun Tang >Assignee: Zakelly Lan >Priority: Major > Fix For: 1.14.1 > > > RocksDBConfigurableOptions mainly focus on the settings of DBOptions and > ColumnFamilyOptions. The original design of this class is used to let user > could configure RocksDB via configurations instead of programmatically > implemented RocksDBOptionsFactory. > To make the minimal change, original options in RocksDBConfigurableOptions > have no default value so that we would not make anything happen in > DefaultConfigurableOptionsFactory just as before. > However, this make user not so clear of the option meaning with no default > value, and we could consider change the relationship between them. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-24046) Refactor the relationship bwtween PredefinedOptions and RocksDBConfigurableOptions
[
https://issues.apache.org/jira/browse/FLINK-24046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17441720#comment-17441720
]
Yun Tang commented on FLINK-24046:
--
As discussed offline, I think we should still keep the
{{DefaultConfigurableOptionsFactory}} but deprecate it, and make
{{state.backend.rocksdb.options-factory}} no default value so that Flink could
tell users that {{DefaultConfigurableOptionsFactory}} has been deprecated if
detected that {{DefaultConfigurableOptionsFactory}} configured by user
explicitly.
> Refactor the relationship bwtween PredefinedOptions and
> RocksDBConfigurableOptions
> --
>
> Key: FLINK-24046
> URL: https://issues.apache.org/jira/browse/FLINK-24046
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / State Backends
>Reporter: Yun Tang
>Assignee: Zakelly Lan
>Priority: Major
> Fix For: 1.14.1
>
>
> RocksDBConfigurableOptions mainly focus on the settings of DBOptions and
> ColumnFamilyOptions. The original design of this class is used to let user
> could configure RocksDB via configurations instead of programmatically
> implemented RocksDBOptionsFactory.
> To make the minimal change, original options in RocksDBConfigurableOptions
> have no default value so that we would not make anything happen in
> DefaultConfigurableOptionsFactory just as before.
> However, this make user not so clear of the option meaning with no default
> value, and we could consider change the relationship between them.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
