Re: Unit Test for KeyedProcessFunction with out-of-core state

[Alexey Trenikhun]

Hi Gordon,
We already use [1]. Unfortunately it doesn’t allow to detect out-of-core 
specific bugs like this:
POJO v = myMapState.get(myKey):
v.setStatus(1);
return;
// missing myMapState.put(myKey, v);

Thanks,
Alexey

________________________________
From: Tzu-Li (Gordon) Tai <tzuli...@apache.org>
Sent: Friday, September 4, 2020 12:35:48 AM
To: Alexey Trenikhun <yen...@msn.com>
Cc: Flink User Mail List <user@flink.apache.org>
Subject: Re: Unit Test for KeyedProcessFunction with out-of-core state

Hi Alexey,

Is there a specific reason why you want to test against RocksDB?

Otherwise, in Flink tests we use a `KeyedOneInputStreamOperatorTestHarness` [1] 
that allows you to wrap a user function and eliminate the need to worry about 
setting up heavy runtime context / dependencies such as the state backend.
As a unit test, this should be sufficient for you to implement basic test 
scenarios for your function, such as expected output given inputs, state etc.
Does this provide what you are looking for?

Cheers,
Gordon

[1] 
https://github.com/apache/flink/blob/1d5f44710270d1c615537f0d05ab49e699d3a6e5/flink-streaming-java/src/test/java/org/apache/flink/streaming/util/KeyedOneInputStreamOperatorTestHarness.java

On Fri, Sep 4, 2020 at 2:28 AM Alexey Trenikhun 
<yen...@msn.com<mailto:yen...@msn.com>> wrote:
Hello,
I want to unit test KeyedProcessFunction which uses with out-of-core state 
(like rocksdb).
Does Flink has mock for rocksdb, which can be used in unit tests ?

Thanks,
Alexey