Jorn, correct and I suppose that's where we are at this point. RocksDB based
backend is definitely looking promising for our use case. Since I haven't
gotten a definite no-no on using 30% for YARN cut-off ratio (about 1.8GB from
6GB memory) and off-heap flag turned on, we will continue on that p
Well you can only performance test it beforehand in different scenarios with
different configurations.
I am not sure what exactly your state holds (eg how many objects etc), but if
it is Java objects then 3 times might be a little bit low (depends also how you
initially tested state size) - ho
Hi Till,
I got the same feedback from Robert Metzger over in Stackflow. I have switched
my app to use RocksDB and as yes, it did stabilize the app :)
However, I am still struggling with how to map out my TMs and JMs memory,
number of slots per TMs etc. Currently I am using 60 slots with 10 TMs
Sorry for my rant...fairly new. Felt lost. The one (StreamTableEnvironment)
from java needs to use the constructor:
StreamTableEnvironment(StreamExecutionEnvironment execEnv, TableConfig
config) .
Now it works. Thanks. Still confused...why
BatchTableEnvironment tEnv = TableEnvironment.getTableE
I felt it is so messy... for the api design: Look at this...
StreamTableEnvironment can from three different packages, which I should
choose? I tried each of them and I just have one problem or another.
//import org.apache.flink.table.api.scala.StreamTableEnvironment;
//import org.apache.flink.ta
Where is this method(from which class/object) you mentioned? I can only find
(new TableConversions(ts)).toRetractStream(TypeInformation.of(Row.class))
I use flink 1.3.2 java api, and weird the compilation error says this method
is not available although I can see it in the api and my ide tip.
Thanks for your help, I solved the issue refactoring HTMStream adding new
api's
--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
We have also faced similar issues. The only thing that happens in sync when
using async snaphots is getting a persistent point in time picture which in
case of rocksdb backend is making symlinks. That would linearly increase
with number of files to symlink but this should be negligible. We could no
Hi all,
In our application we have a requirement to very low latency, preferably less
than 5ms.
We were able to achieve this so far, but when we start increasing the state
size, we see distinctive decrease in latency.
We have added MinPauseBetweenCheckpoints, and are using async snapshots.
*
