[jira] [Commented] (FLINK-24046) Refactor the relationship bwtween PredefinedOptions and RocksDBConfigurableOptions
[ https://issues.apache.org/jira/browse/FLINK-24046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17438531#comment-17438531 ] Zakelly Lan commented on FLINK-24046: - I'm going to fix it this way: # Provide a default value for each option in {{RocksDBConfigurableOptions}}; # Refactor {{PredefinedOptions}}, make each element of this enum carry a config map that will override the default value of specified keys. # Change the {{RocksDBResourceContainer#getDbOptions}} and {{RocksDBResourceContainer#getColumnOptions}}, load config values from the provided ReadableConfig and specified PredefinedOptions. Override relationship: user specified > predefined options > default value. Then we could remove the {{DefaultConfigurableOptionsFactory}}. # Keep the {{ConfigurableRocksDBOptionsFactory}} interface and load the user-defined options factory, and call corresponding methods to apply customized RocksDB options after loading from the config. > Refactor the relationship bwtween PredefinedOptions and > RocksDBConfigurableOptions > -- > > Key: FLINK-24046 > URL: https://issues.apache.org/jira/browse/FLINK-24046 > Project: Flink > Issue Type: Improvement > Components: Runtime / State Backends >Reporter: Yun Tang >Assignee: Zakelly Lan >Priority: Major > Fix For: 1.14.1 > > > RocksDBConfigurableOptions mainly focus on the settings of DBOptions and > ColumnFamilyOptions. The original design of this class is used to let user > could configure RocksDB via configurations instead of programmatically > implemented RocksDBOptionsFactory. > To make the minimal change, original options in RocksDBConfigurableOptions > have no default value so that we would not make anything happen in > DefaultConfigurableOptionsFactory just as before. > However, this make user not so clear of the option meaning with no default > value, and we could consider change the relationship between them. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-24046) Refactor the relationship bwtween PredefinedOptions and RocksDBConfigurableOptions
[ https://issues.apache.org/jira/browse/FLINK-24046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17439002#comment-17439002 ] Yun Tang commented on FLINK-24046: -- [~Zakelly] Since {{DefaultConfigurableOptionsFactory}} had existed in [documentation|https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/ops/state/state_backends/#passing-options-factory-to-rocksdb] for long time. I don't think we should remove it directly to backward compatibility. > Refactor the relationship bwtween PredefinedOptions and > RocksDBConfigurableOptions > -- > > Key: FLINK-24046 > URL: https://issues.apache.org/jira/browse/FLINK-24046 > Project: Flink > Issue Type: Improvement > Components: Runtime / State Backends >Reporter: Yun Tang >Assignee: Zakelly Lan >Priority: Major > Fix For: 1.14.1 > > > RocksDBConfigurableOptions mainly focus on the settings of DBOptions and > ColumnFamilyOptions. The original design of this class is used to let user > could configure RocksDB via configurations instead of programmatically > implemented RocksDBOptionsFactory. > To make the minimal change, original options in RocksDBConfigurableOptions > have no default value so that we would not make anything happen in > DefaultConfigurableOptionsFactory just as before. > However, this make user not so clear of the option meaning with no default > value, and we could consider change the relationship between them. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-24046) Refactor the relationship bwtween PredefinedOptions and RocksDBConfigurableOptions
[ https://issues.apache.org/jira/browse/FLINK-24046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17439015#comment-17439015 ] Zakelly Lan commented on FLINK-24046: - [~yunta] Good point. I think we could treat 'org.apache.flink.contrib.streaming.state.DefaultConfigurableOptionsFactory' as a special value for the config 'state.backend.rocksdb.options-factory'. If user set this, we does not load any option factory and just load config values from the provided ReadableConfig and specified PredefinedOptions. It is the same behavior as the {{DefaultConfigurableOptionsFactory}} does before. > Refactor the relationship bwtween PredefinedOptions and > RocksDBConfigurableOptions > -- > > Key: FLINK-24046 > URL: https://issues.apache.org/jira/browse/FLINK-24046 > Project: Flink > Issue Type: Improvement > Components: Runtime / State Backends >Reporter: Yun Tang >Assignee: Zakelly Lan >Priority: Major > Fix For: 1.14.1 > > > RocksDBConfigurableOptions mainly focus on the settings of DBOptions and > ColumnFamilyOptions. The original design of this class is used to let user > could configure RocksDB via configurations instead of programmatically > implemented RocksDBOptionsFactory. > To make the minimal change, original options in RocksDBConfigurableOptions > have no default value so that we would not make anything happen in > DefaultConfigurableOptionsFactory just as before. > However, this make user not so clear of the option meaning with no default > value, and we could consider change the relationship between them. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-24046) Refactor the relationship bwtween PredefinedOptions and RocksDBConfigurableOptions
[ https://issues.apache.org/jira/browse/FLINK-24046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17439017#comment-17439017 ] Zakelly Lan commented on FLINK-24046: - Moreover, we could remove this special value since 15.0 > Refactor the relationship bwtween PredefinedOptions and > RocksDBConfigurableOptions > -- > > Key: FLINK-24046 > URL: https://issues.apache.org/jira/browse/FLINK-24046 > Project: Flink > Issue Type: Improvement > Components: Runtime / State Backends >Reporter: Yun Tang >Assignee: Zakelly Lan >Priority: Major > Fix For: 1.14.1 > > > RocksDBConfigurableOptions mainly focus on the settings of DBOptions and > ColumnFamilyOptions. The original design of this class is used to let user > could configure RocksDB via configurations instead of programmatically > implemented RocksDBOptionsFactory. > To make the minimal change, original options in RocksDBConfigurableOptions > have no default value so that we would not make anything happen in > DefaultConfigurableOptionsFactory just as before. > However, this make user not so clear of the option meaning with no default > value, and we could consider change the relationship between them. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-24046) Refactor the relationship bwtween PredefinedOptions and RocksDBConfigurableOptions
[ https://issues.apache.org/jira/browse/FLINK-24046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17441720#comment-17441720 ] Yun Tang commented on FLINK-24046: -- As discussed offline, I think we should still keep the {{DefaultConfigurableOptionsFactory}} but deprecate it, and make {{state.backend.rocksdb.options-factory}} no default value so that Flink could tell users that {{DefaultConfigurableOptionsFactory}} has been deprecated if detected that {{DefaultConfigurableOptionsFactory}} configured by user explicitly. > Refactor the relationship bwtween PredefinedOptions and > RocksDBConfigurableOptions > -- > > Key: FLINK-24046 > URL: https://issues.apache.org/jira/browse/FLINK-24046 > Project: Flink > Issue Type: Improvement > Components: Runtime / State Backends >Reporter: Yun Tang >Assignee: Zakelly Lan >Priority: Major > Fix For: 1.14.1 > > > RocksDBConfigurableOptions mainly focus on the settings of DBOptions and > ColumnFamilyOptions. The original design of this class is used to let user > could configure RocksDB via configurations instead of programmatically > implemented RocksDBOptionsFactory. > To make the minimal change, original options in RocksDBConfigurableOptions > have no default value so that we would not make anything happen in > DefaultConfigurableOptionsFactory just as before. > However, this make user not so clear of the option meaning with no default > value, and we could consider change the relationship between them. -- This message was sent by Atlassian Jira (v8.20.1#820001)