Currently in the MemCheckpointStreamFactory.checkSize() method, it can throw an
IOException with:
throw new IOException(
"Size of the state is larger than the
maximum permitted memory-backed state. Size="
+ size + " , maxSize="
+ maxSize
+ " . Consider using a
different state backend, like the File System State backend.");
But this will happen even if you’re using the File System State backend, right?
This came up here:
https://stackoverflow.com/questions/50149005/ioexception-size-of-the-state-is-larger-than-the-maximum-permitted-memory-backe
<https://stackoverflow.com/questions/50149005/ioexception-size-of-the-state-is-larger-than-the-maximum-permitted-memory-backe>
So shouldn’t it suggest using the RocksDB state backend instead?
— Ken
--------------------------
Ken Krugler
+1 530-210-6378
http://www.scaleunlimited.com
Custom big data solutions & training
Flink, Solr, Hadoop, Cascading & Cassandra
