Re: Failed to resume from HA when the checkpoint has been deleted.

2024-06-11 Thread Jean-Marc Paulin
in that scenario. But maybe there isn't any. Best regards JM From: Zhanghao Chen Sent: Tuesday, June 11, 2024 03:56 To: Jean-Marc Paulin ; user@flink.apache.org Subject: [EXTERNAL] Re: Failed to resume from HA when the checkpoint has been deleted. Hi, In this case, you

Failed to resume from HA when the checkpoint has been deleted.

2024-06-10 Thread Jean-Marc Paulin
Hi, We have a 1.19 Flink streaming job, with HA enabled (ZooKeeper), checkpoint/savepoint in S3. We had an outage and now the jobmanager keeps restarting. We think it because it read the job id to be restarted from ZooKeeper, but because we lost our S3 Storage as part of the outage it cannot

Saw a java.lang.ClassNotFoundException: com.facebook.presto.hive.s3.PrestoS3FileSystem$UnrecoverableS3OperationException

2024-05-09 Thread Jean-Marc Paulin
Hi, We use S3 as our datastore for checkpoint/savepoints, and following an S3 error we saw that exception: ``` java.io.IOException: GET operation failed: Could not transfer error message at org.apache.flink.runtime.blob.BlobClient.getInternal(BlobClient.java:231) at

RE: Flink 1.18: Unable to resume from a savepoint with error InvalidPidMappingException

2024-04-23 Thread Jean-Marc Paulin
___ From: Yanfei Lei Sent: Monday, April 22, 2024 03:28 To: Jean-Marc Paulin Cc: user@flink.apache.org Subject: [EXTERNAL] Re: Flink 1.18: Unable to resume from a savepoint with error InvalidPidMappingException Hi JM, Yes, `InvalidPidMappingException` occurs because the tra

Flink 1.18: Unable to resume from a savepoint with error InvalidPidMappingException

2024-04-19 Thread Jean-Marc Paulin
Hi, we use Flink 1.18 with Kafka Sink, and we enabled `EXACTLY_ONCE` on one of our kafka sink. We set the transation timeout to 15 minutes. When we try to restore from a savepoint, way after that 15 minutes window, Flink enter in a RESTARTING loop. We see the error: ``` { "exception": {

Q: Not all the task slots are used. Are we missing a setting somewhere?

2024-02-23 Thread Jean-Marc Paulin
Hi, We used to run with 3 task managers with numberOfTaskSlots = 2. So all together we had 6 task slots and our application used them all. Trying to increase throughput, we increased the number of task managers to 6. So now we have 12 task slots all together. However our application still only

Is the kafka-connector doc missing a dependency on flink-connector-base

2023-12-04 Thread Jean-Marc Paulin
Hi, Trying to update the kafka connector to my project and I am missing a class. Is the doc missing a dependency on flink-connector-base ? org.apache.flink flink-connector-base compile I added it and it works. I think that's required but I would have expected this