Max would it make sense to make the rocksdb runtime only at the runner
level just to hint its use. I assume that most Flink users might want
to have RocksDB as the default state backend?
runtimeOnly
"org.apache.flink:flink-statebackend-rocksdb_2.11:$flink_version"
On Tue, Apr 28, 2020 at 1:14 PM Sruthi Sree Kumar
<[email protected]> wrote:
>
> Hello,
>
> I have looked the Beam code. The statebackend that we pass should be an
> instance of FlinkStateBackendFactory . But there is no implementation for
> the Interface.
> None of the FlinkStateBackend Implements this interface.
>
> So even when I try to pass the default MemoryStateBackend as an argument, it
> throws an error.
>
> But I could manage to specify the statebackend (filesystem) using the config
> file in the config directory specified by the env variable ENV_FLINK_CONF_DIR.
>
> Regards,
> Sruthi
>
> On Tue, Apr 28, 2020 at 11:35 AM Maximilian Michels <[email protected]> wrote:
>>
>> Hi Sruthi,
>>
>> Not possible out-of-the-box at the moment. You'll have to add the
>> RocksDB Flink dependency in flink_runner.gradle, e.g.:
>>
>> compile "org.apache.flink:flink-statebackend-rocksdb_2.11:$flink_version"
>>
>> Also in the Flink config you have to set
>>
>> state.backend: rocksdb
>>
>> Then you can run Nexmark against the cluster, e.g.
>>
>> ./gradlew :sdks:java:testing:nexmark:run \
>> -Pnexmark.runner=":runners:flink:1.10" \
>> -Pnexmark.args="
>> --runner=FlinkRunner
>> --flinkMaster=<flink_master:8081>
>> --streaming=true
>> --shutdownSourcesOnFinalWatermark=true
>> --query=12
>> --suite=SMOKE
>> --manageResources=false
>> --monitorJobs=true
>> --enforceEncodability=true
>> --enforceImmutability=true"
>>
>>
>> Admittedly, this is a bit complicated. We could make that easier without
>> much work.
>>
>> Cheers,
>> Max
>>
>> On 28.04.20 10:26, Sruthi Sree Kumar wrote:
>> > Hello,
>> >
>> > Is it possible to run the nexmark queries by specifying a
>> > state-backed(Ex: RocksDB) ?
>> >
>> >
>> > Regards,
>> > Sruthi
