Re: StreamingFileSink & checkpoint tuning

2022-07-02 Thread Weihua Hu
> > Best regards, > Yuxia > > -- > *发件人: *"Xin Ma" > *收件人: *"User" > *发送时间: *星期四, 2022年 6 月 30日 下午 11:05:51 > *主题: *StreamingFileSink & checkpoint tuning > > Hi, > > I recently encountered an issue while usin

Re: StreamingFileSink & checkpoint tuning

2022-06-30 Thread yuxia
gards, Yuxia 发件人: "Xin Ma" 收件人: "User" 发送时间: 星期四, 2022年 6 月 30日 下午 11:05:51 主题: StreamingFileSink & checkpoint tuning Hi, I recently encountered an issue while using StreamingFileSink. I have a flink job consuming records from various sources and write to s3 w

StreamingFileSink & checkpoint tuning

2022-06-30 Thread Xin Ma
Hi, I recently encountered an issue while using StreamingFileSink. I have a flink job consuming records from various sources and write to s3 with streaming file sink. But the job sometimes fails due to checkpoint timeout, and the root cause is checkpoint alignment failure as there is data skewnes