[jira] [Commented] (FLINK-26803) Merge small ChannelState file for Unaligned Checkpoint

2023-06-04 Thread Rui Fan (Jira)
[ https://issues.apache.org/jira/browse/FLINK-26803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17729170#comment-17729170 ] Rui Fan commented on FLINK-26803: - {quote}It is probably good to always have a FLIP approved for any

[jira] [Commented] (FLINK-26803) Merge small ChannelState file for Unaligned Checkpoint

2023-06-04 Thread Dong Lin (Jira)
[ https://issues.apache.org/jira/browse/FLINK-26803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17729166#comment-17729166 ] Dong Lin commented on FLINK-26803: -- [~fanrui] I see. Thanks for the information. The API change

[jira] [Commented] (FLINK-26803) Merge small ChannelState file for Unaligned Checkpoint

2023-06-04 Thread Rui Fan (Jira)
[ https://issues.apache.org/jira/browse/FLINK-26803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17729143#comment-17729143 ] Rui Fan commented on FLINK-26803: - {quote}The PR of this JIRA introduced a new public evolving API

[jira] [Commented] (FLINK-26803) Merge small ChannelState file for Unaligned Checkpoint

2023-06-04 Thread Dong Lin (Jira)
[ https://issues.apache.org/jira/browse/FLINK-26803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17729069#comment-17729069 ] Dong Lin commented on FLINK-26803: -- FLIP-306 seems to be relate to this JIRA. But the newly added

[jira] [Commented] (FLINK-26803) Merge small ChannelState file for Unaligned Checkpoint

2023-06-04 Thread Dong Lin (Jira)
[ https://issues.apache.org/jira/browse/FLINK-26803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17729067#comment-17729067 ] Dong Lin commented on FLINK-26803: -- The PR of this JIRA introduced a new public evolving API

[jira] [Commented] (FLINK-26803) Merge small ChannelState file for Unaligned Checkpoint

2023-01-25 Thread Rui Fan (Jira)
[ https://issues.apache.org/jira/browse/FLINK-26803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17680500#comment-17680500 ] Rui Fan commented on FLINK-26803: - [~pnowojski] Thanks a lot for your review~ > Merge small

[jira] [Commented] (FLINK-26803) Merge small ChannelState file for Unaligned Checkpoint

2022-06-27 Thread Piotr Nowojski (Jira)
[ https://issues.apache.org/jira/browse/FLINK-26803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17559261#comment-17559261 ] Piotr Nowojski commented on FLINK-26803: Hi [~fanrui] +1, from my side to the design doc. >

[jira] [Commented] (FLINK-26803) Merge small ChannelState file for Unaligned Checkpoint

2022-06-10 Thread fanrui (Jira)
[ https://issues.apache.org/jira/browse/FLINK-26803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17552631#comment-17552631 ] fanrui commented on FLINK-26803: Hi everyone, I've done a POC for merging the channel state files and

[jira] [Commented] (FLINK-26803) Merge small ChannelState file for Unaligned Checkpoint

2022-05-19 Thread Yuan Mei (Jira)
[ https://issues.apache.org/jira/browse/FLINK-26803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17539500#comment-17539500 ] Yuan Mei commented on FLINK-26803: -- small file problem is a problem for state backend as well. Yes,

[jira] [Commented] (FLINK-26803) Merge small ChannelState file for Unaligned Checkpoint

2022-05-18 Thread fanrui (Jira)
[ https://issues.apache.org/jira/browse/FLINK-26803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17538832#comment-17538832 ] fanrui commented on FLINK-26803: Hi [~pnowojski] thanks for your feedback. Yes, I mean that for mostly

[jira] [Commented] (FLINK-26803) Merge small ChannelState file for Unaligned Checkpoint

2022-05-18 Thread Piotr Nowojski (Jira)
[ https://issues.apache.org/jira/browse/FLINK-26803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17538819#comment-17538819 ] Piotr Nowojski commented on FLINK-26803: Sorry for late response [~fanrui]. Thanks for the

[jira] [Commented] (FLINK-26803) Merge small ChannelState file for Unaligned Checkpoint

2022-05-04 Thread fanrui (Jira)
[ https://issues.apache.org/jira/browse/FLINK-26803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17532058#comment-17532058 ] fanrui commented on FLINK-26803: Hi [~pnowojski]  [~akalashnikov] , I want to add some information about

[jira] [Commented] (FLINK-26803) Merge small ChannelState file for Unaligned Checkpoint

2022-05-04 Thread fanrui (Jira)
[ https://issues.apache.org/jira/browse/FLINK-26803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17531603#comment-17531603 ] fanrui commented on FLINK-26803: Hi [~pnowojski]  Thanks for your feedback. I'm sorry. I didn't express

[jira] [Commented] (FLINK-26803) Merge small ChannelState file for Unaligned Checkpoint

2022-05-04 Thread Piotr Nowojski (Jira)
[ https://issues.apache.org/jira/browse/FLINK-26803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17531595#comment-17531595 ] Piotr Nowojski commented on FLINK-26803: Would removing 15K out of 50K files solve the issue for

[jira] [Commented] (FLINK-26803) Merge small ChannelState file for Unaligned Checkpoint

2022-05-03 Thread fanrui (Jira)
[ https://issues.apache.org/jira/browse/FLINK-26803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17531182#comment-17531182 ] fanrui commented on FLINK-26803: Hi [~pnowojski] you're right. After my research, 15K of the 50K files

[jira] [Commented] (FLINK-26803) Merge small ChannelState file for Unaligned Checkpoint

2022-05-03 Thread Piotr Nowojski (Jira)
[ https://issues.apache.org/jira/browse/FLINK-26803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17531097#comment-17531097 ] Piotr Nowojski commented on FLINK-26803: If it's just a single file per {{OperatorChain}}, why

[jira] [Commented] (FLINK-26803) Merge small ChannelState file for Unaligned Checkpoint

2022-05-02 Thread fanrui (Jira)
[ https://issues.apache.org/jira/browse/FLINK-26803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17531034#comment-17531034 ] fanrui commented on FLINK-26803: Hi [~pnowojski] , thanks for your reply. In fact, there is one file

[jira] [Commented] (FLINK-26803) Merge small ChannelState file for Unaligned Checkpoint

2022-05-02 Thread Piotr Nowojski (Jira)
[ https://issues.apache.org/jira/browse/FLINK-26803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17530907#comment-17530907 ] Piotr Nowojski commented on FLINK-26803: {quote} one file for each operator in the chain {quote}

[jira] [Commented] (FLINK-26803) Merge small ChannelState file for Unaligned Checkpoint

2022-04-14 Thread fanrui (Jira)
[ https://issues.apache.org/jira/browse/FLINK-26803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17522363#comment-17522363 ] fanrui commented on FLINK-26803: Hi [~akalashnikov] , thanks for your quick feedback. For some flink

[jira] [Commented] (FLINK-26803) Merge small ChannelState file for Unaligned Checkpoint

2022-04-14 Thread Anton Kalashnikov (Jira)
[ https://issues.apache.org/jira/browse/FLINK-26803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17522333#comment-17522333 ] Anton Kalashnikov commented on FLINK-26803: --- Yep, I know that

[jira] [Commented] (FLINK-26803) Merge small ChannelState file for Unaligned Checkpoint

2022-04-14 Thread fanrui (Jira)
[ https://issues.apache.org/jira/browse/FLINK-26803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17522304#comment-17522304 ] fanrui commented on FLINK-26803: Hi [~akalashnikov] , thanks for your reply. In fact, there is only one

[jira] [Commented] (FLINK-26803) Merge small ChannelState file for Unaligned Checkpoint

2022-04-14 Thread Anton Kalashnikov (Jira)
[ https://issues.apache.org/jira/browse/FLINK-26803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17522261#comment-17522261 ] Anton Kalashnikov commented on FLINK-26803: --- [~fanrui], thanks for this idea. I did some

[jira] [Commented] (FLINK-26803) Merge small ChannelState file for Unaligned Checkpoint

2022-03-28 Thread fanrui (Jira)
[ https://issues.apache.org/jira/browse/FLINK-26803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17513251#comment-17513251 ] fanrui commented on FLINK-26803: Hi [~pnowojski]  [~arvid]  , could you help take a look in your free