flink checkpoint 延迟的性能问题讨论

[15868861416]

各位大佬，
背景：
实际测试flink读Kafka 数据写入hudi, checkpoint的间隔时间是1min, 
state.backend分别为filesystem，测试结果如下：



写hudi的checkpoint 的延迟





写iceberg得延迟：



疑问： hudi的checkpoint的文件数据比iceberg要大很多，如何降低flink写hudi的checkpoint的延迟？


| |
博星
|
|
15868861...@163.com
|

??????Flink????????????????join??????????????n??????

[????]

1.left
 join
2.n??



  
| ?? | <1227581...@qq.com.INVALID> |
|  | 2024??06??16?? 21:08 |
| ?? | user-zh |
| ?? | |
|  | ??Flinkjoin??n?? |
??SQL??datastreamDWD??ClickHouse/Doris??



1227581...@qq.com



 




--  --
??: 
   "user-zh"

??????Flink????????????????join??????????????n??????

[????]

??flink sql apidatastream api??



  
| ?? | <1227581...@qq.com.INVALID> |
|  | 2024??06??16?? 20:35 |
| ?? | user-zh |
| ?? | |
|  | Flinkjoin??n?? |
??
1DWD??KafkaDWD
2Kafka
3??FlinkKafka1,2??FlinkKafka??DWD12DWS??


DWS







|
|

1227581...@qq.com
|
 

Flink????????????????join??????????????n??????

[????]

??
1DWD??KafkaDWD
2Kafka
3??FlinkKafka1,2??FlinkKafka??DWD12DWS??


DWS










1227581...@qq.com