Done: https://issues.apache.org/jira/browse/FLINK-10912
Thanks, Thomas On Thu, Nov 15, 2018 at 1:22 AM Till Rohrmann <trohrm...@apache.org> wrote: > I think it sound like a good idea to be able to specify an options factory > in the flink-conf.yaml. Please go ahead with creating the respective JIRA > issues. > > Cheers, > Till > > On Wed, Nov 14, 2018 at 7:33 AM Yun Tang <myas...@live.com> wrote: > > > Hi all > > > > We already found the programmatic way to configure RocksDB was not so > > friendly in our Alibaba's production environment, and refactored it by > > wrapping customer configurations for RocksDB. We pre-wrapped > configurations > > such as block cache size, whether to cache index&filter into block cache > > size, number of background flush threads, and could get the them from > > TaskManagerConfiguration if users want to change the default values. But > I > > think our community's way using options factory could also make sense if > > sophisticated user want to make fine-grained tuning which might use > > configuration not included in our pre-wrapped configuration. We might > > combine these two kind of operations together. > > > > I'd like to create a JIRA about this improvement, what's your opinions? > > > > Best > > Yun Tang > > > > ________________________________ > > From: Thomas Weise <t...@apache.org> > > Sent: Tuesday, November 13, 2018 1:53 > > To: dev@flink.apache.org > > Cc: u...@apache.org > > Subject: Re: How to use RocksDBStateBackend predefined options > > > > Sounds good. Perhaps it would also be good to allow the user to specify > an > > options factory in flink-conf.yaml for more flexibility? > > > > Thomas > > > > On Mon, Nov 12, 2018 at 9:48 AM Stefan Richter < > > s.rich...@data-artisans.com> > > wrote: > > > > > Hi, > > > > > > Ufuk is right, for historical reasons there is currently only the > > > programatic way but I think nothing speaks fundamentally against > offering > > > configuration via config in the future (maybe just a lot of config keys > > > must be introduced to cover all options). > > > > > > Best, > > > Stefan > > > > > > > On 9. Nov 2018, at 22:52, Ufuk Celebi <u...@apache.org> wrote: > > > > > > > > Hey Thomas, > > > > > > > > On Fri, Nov 9, 2018 at 6:07 PM Thomas Weise <t...@apache.org> wrote: > > > >> Is there a way to activate the predefined options via configuration > / > > > flink- > > > >> conf.yaml? Or only programmatically, like in [4]? The difficulty > with > > > the > > > >> programmatic route (assuming this works now), is that in my case the > > > client > > > >> is Beam and I'm not writing the code that submits the job. > > > > > > > > AFAIK no. You can only do it programmatically at the moment [1]. > > > > Having the option to configure all settings through the configuration > > > > file seems to be a valid feature request to me. > > > > > > > > @Stefan Richter (cc'd): Are there any reasons that speak against > > > > exposing these options via the configuration file in your opinion? > > > > > > > > Best, > > > > > > > > Ufuk > > > > > > > > [1] Looking at the code in [2] and [3], the only options that are > > > > exposed via the configuration file are local directories and the > timer > > > > service factory. > > > > [2] > > > > > > https://github.com/apache/flink/blob/master/flink-state-backends/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBOptions.java > > > > [3] > > > > > > https://github.com/apache/flink/blob/master/flink-state-backends/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBStateBackend.java > > > > > > > > >
