Why flink-s3-fs-presto is recommended for checkpointing

2023-09-07 Thread Prabhu Joseph
Hi, The Flink Doc mentions flink-s3-fs-presto as the recommended one for checkpointing. Do we have any more details on how this was concluded? I am part of a platform team where we are deciding whether to support both filesystems or not. This will be a very useful input for us. Thanks in

Re: Elastic Block Store as checkpoint storage

2023-07-19 Thread Prabhu Joseph
h, for the local RocksDB working directory. We > don't remount them on failure though right now due to the additional > latency that is introduced by that. > > Cheers, > > Konstantin > > Am Mi., 12. Juli 2023 um 18:55 Uhr schrieb Prabhu Joseph < > prabhujose

Elastic Block Store as checkpoint storage

2023-07-12 Thread Prabhu Joseph
, Prabhu Joseph

Re: RocksDB State Backend GET returns null intermittently

2023-06-27 Thread Prabhu Joseph
ializer, as this is one of the >> main differences between RocksDBStateBackend and HashMapStateBackend >> (HashMapStateBackend does not perform serialization and deserialization). >> >> On Wed, Jun 21, 2023 at 3:44 PM Prabhu Joseph >> wrote: >> >>> Hi, >>> >>&

RocksDB State Backend GET returns null intermittently

2023-06-21 Thread Prabhu Joseph
://flink.apache.org/2021/01/18/using-rocksdb-state-backend-in-apache-flink-when-and-how/ Any ideas on what could be wrong or how to debug this? Thanks, Prabhu Joseph

Re: Flink Checkpoint times out with checkpointed data size doubles every checkpoint.

2023-06-20 Thread Prabhu Joseph
umulate the state in a big window, e.g. in a 2 hour >> Tumbling window the maximum job state will be reached in two hours only. So >> your job should be scaled or optimized. >> >> Best >> Alex >> >> On Tue, Jun 20, 2023 at 10:39 AM Prabhu Joseph < >&

Flink Checkpoint times out with checkpointed data size doubles every checkpoint.

2023-06-20 Thread Prabhu Joseph
Hi, Flink Checkpoint times out with checkpointed data size doubles every checkpoint. Any ideas on what could be wrong in the application or how to debug this? [image: checkpoint_issue.png]

Kryo Serialization fails with NPE

2023-05-25 Thread Prabhu Joseph
Hi, Need some pointers in fixing an issue with Kryo Serialization failing with NPE. Based on the stack trace, kryo fails to find the registered class for the given input type. Kryo fails to find the class type *FlinkA$FlinkB$$Lambda$2568/0x7ee2fbec64b0* This is how the actual code looks like.

Re: Query on RestartPipelinedRegionFailoverStrategy

2023-05-16 Thread Prabhu Joseph
-> e all are point-wise, In theory, > it should form two regions. > > Best regards, > > Weijie > > > Prabhu Joseph 于2023年5月15日周一 09:58写道: > >> Hi, I am testing the Flink Fine-Grained Recovery >> <https://cwiki.apache.org/confluence/display/FLINK/FLIP-1%3A+

Query on RestartPipelinedRegionFailoverStrategy

2023-05-14 Thread Prabhu Joseph
d task (a1 to e1) without restarting other parallel pipelines. Thanks, Prabhu Joseph