您好,

我的 Flink job是以 reactive 模式运行,然后用了 Kubernetes HPA 来自动扩容/缩容
TaskManager。每当TaskManager
扩容/缩容的时候,Flink会在日志中报错:因为扩缩容之前的TaskManager没有在运行导致checkpoint失败,同时也有checkpoint失败的警报。
但实际上checkpoint 还能顺利进行, job也没有运行错误。 重启job后这个错误就会消失。想请教一下如何修复这个问题?

详细的日志如下
2022-12-13 05:08:22.339 [jobmanager-io-thread-1] INFO
 org.apache.flink.runtime.checkpoint.CheckpointCoordinator  - Completed
checkpoint 40393 for job 00000000000000000000000000000000 (488170 bytes,
checkpointDuration=2582 ms, finalizationTime=322 ms).
2022-12-13 05:08:28.083 [Checkpoint Timer] INFO
 org.apache.flink.runtime.checkpoint.CheckpointFailureManager  - Failed to
trigger checkpoint for job 00000000000000000000000000000000 since
Checkpoint triggering task Source: Custom Source -> Sink: Unnamed (1/79) of
job 00000000000000000000000000000000 is not being executed at the moment.
Aborting checkpoint. Failure reason: Not all required tasks are currently
running..
2022-12-13 05:09:19.437 [Checkpoint Timer] INFO
 org.apache.flink.runtime.checkpoint.CheckpointCoordinator  - Triggering
checkpoint 40394 (type=CheckpointType{name='Checkpoint',
sharingFilesStrategy=FORWARD_BACKWARD}) @ 1670908159435 for job
00000000000000000000000000000000.
2022-12-13 05:09:25.208 [jobmanager-io-thread-1] INFO
 org.apache.flink.fs.s3.common.writer.S3Committer  - Committing
flink-ingest-sps-nv-consumer/2022-11-15T01:10:30Z/00000000000000000000000000000000/chk-40394/_metadata
with MPU ID
_3vKXSVBMuBM7207EpGvCXOTRQskAiPPj88DSTTn55Uzuc_76dnubmTAPBovyWbKBKU8Wxqz6SuFBJ8cZnAOH_PkGEP36KJzMFYYPmT.xZvmLnM.YX1oJSHN3VP1TXpJECY8y80psYvRWvbt2e8CMeoa9JiOWiGYGRmqLGRdlQA-
2022-12-13 05:09:25.747 [jobmanager-io-thread-1] INFO
 org.apache.flink.runtime.checkpoint.CheckpointCoordinator  - Completed
checkpoint 40394 for job 00000000000000000000000000000000 (482850 bytes,
checkpointDuration=5982 ms, finalizationTime=330 ms).

Thanks,
Chen

Reply via email to