[
https://issues.apache.org/jira/browse/FLINK-18242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17132910#comment-17132910
]
Yu Li commented on FLINK-18242:
-------------------------------
Thanks for the analysis [~yunta], but you may have neglected below code path in
`RocksDBStateBackend#setOptions`:
{code}
public void setOptions(OptionsFactory optionsFactory) {
this.rocksDbOptionsFactory = optionsFactory instanceof
RocksDBOptionsFactory
? (RocksDBOptionsFactory) optionsFactory
: new
RocksDBOptionsFactoryAdapter(optionsFactory);
}
{code}
While in `RocksDBOptionsFactoryAdaptor` we have below method:
{code}
public DBOptions createDBOptions(DBOptions currentOptions,
Collection<AutoCloseable> handlesToClose) {
return optionsFactory.createDBOptions(currentOptions);
}
{code}
Please notice that we also have below instructions in `RocksDBOptionsFactory`
javadoc:
{code}
/**
* Do not override these methods, they are only to maintain interface
compatibility with
* prior versions. They will be removed in one of the next versions.
*/
@Override
default DBOptions createDBOptions(DBOptions currentOptions) {
return createDBOptions(currentOptions, new ArrayList<>());
}
{code}
And since the usage mentioned in description here actually breaks the above
rule, IMHO it's a user error.
However, we should take some actions to prevent this from happening again,
possible ones including:
1. Make `DefaultConfigurableOptionsFactory` a final class. This may break the
compilation of existing application code, but it's way better than having the
problem mentioned here.
2. Write more explicit notice and suggestions in our documentation.
> Custom OptionsFactory settings seem to have no effect on RocksDB
> ----------------------------------------------------------------
>
> Key: FLINK-18242
> URL: https://issues.apache.org/jira/browse/FLINK-18242
> Project: Flink
> Issue Type: Bug
> Components: Runtime / State Backends
> Affects Versions: 1.10.0, 1.10.1, 1.11.0
> Reporter: Nico Kruber
> Priority: Major
> Attachments: DefaultConfigurableOptionsFactoryWithLog.java
>
>
> When I configure a custom {{OptionsFactory}} for RocksDB like this
> (similarly by specifying it via the {{state.backend.rocksdb.options-factory}}
> configuration):
> {code:java}
> Configuration globalConfig = GlobalConfiguration.loadConfiguration();
> String checkpointDataUri =
> globalConfig.getString(CheckpointingOptions.CHECKPOINTS_DIRECTORY);
> RocksDBStateBackend stateBackend = new RocksDBStateBackend(checkpointDataUri);
> stateBackend.setOptions(new DefaultConfigurableOptionsFactoryWithLog());
> env.setStateBackend((StateBackend) stateBackend);{code}
> it seems to be loaded
> {code:java}
> 2020-06-10 12:54:20,720 INFO
> org.apache.flink.contrib.streaming.state.RocksDBStateBackend - Using
> predefined options: DEFAULT.
> 2020-06-10 12:54:20,721 INFO
> org.apache.flink.contrib.streaming.state.RocksDBStateBackend - Using
> application-defined options factory:
> DefaultConfigurableOptionsFactoryWithLog{DefaultConfigurableOptionsFactory{configuredOptions={}}}.
> {code}
> but it seems like none of the options defined in there is actually used. Just
> as an example, my factory does set the info log level to {{INFO_LEVEL}} but
> this is what you will see in the created RocksDB instance:
> {code:java}
> > cat /tmp/flink-io-c95e8f48-0daa-4fb9-a9a7-0e4fb42e9135/*/db/OPTIONS*|grep
> > info_log_level
> info_log_level=HEADER_LEVEL
> info_log_level=HEADER_LEVEL{code}
> Together with the bug from FLINK-18241, it seems I cannot re-activate the
> RocksDB log that we disabled in FLINK-15068. FLINK-15747 was aiming at
> changing that particular configuration, but the problem seems broader since
> {{setDbLogDir()}} was actually also ignored and Flink itself does not change
> that setting.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
