[
https://issues.apache.org/jira/browse/FLINK-20269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiayi Liao updated FLINK-20269:
-------------------------------
Description:
The reason I notice this is, I find that the metadata's persistence can be very
slow (but the states' uploading process works fine) when the network is
unstable, and almost every time I dump the stack of the process, the bottleneck
happens on Hdfs client waiting for Datanodes' ack during metadata's persistance.
I wonder, is it really necessary to flush the stream after every
{{StreamStateHandle}}'s serialization?
was:
The reason I notice this is, I find that the metadata's persistence can be very
slow, especially when the network is unstable, and almost every time I dump the
stack of the process, the bottleneck happens on Hdfs client waiting for
Datanodes' ack during metadata's persistance.
I wonder, is it really necessary to flush the stream after every
{{StreamStateHandle}}'s serialization?
> The flush happens too frequent in SavepoinV2Serializer
> ------------------------------------------------------
>
> Key: FLINK-20269
> URL: https://issues.apache.org/jira/browse/FLINK-20269
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Checkpointing
> Affects Versions: 1.9.0
> Reporter: Jiayi Liao
> Priority: Major
>
> The reason I notice this is, I find that the metadata's persistence can be
> very slow (but the states' uploading process works fine) when the network is
> unstable, and almost every time I dump the stack of the process, the
> bottleneck happens on Hdfs client waiting for Datanodes' ack during
> metadata's persistance.
>
> I wonder, is it really necessary to flush the stream after every
> {{StreamStateHandle}}'s serialization?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
