Re: [ANNOUNCE] New Apache Flink PMC Member - Fan Rui

2024-06-05 Thread Hangxiang Yu
Congratulations, Rui ! On Thu, Jun 6, 2024 at 9:18 AM Lincoln Lee wrote: > Congratulations, Rui! > > Best, > Lincoln Lee > > > Lijie Wang 于2024年6月6日周四 09:11写道: > > > Congratulations, Rui! > > > > Best, > > Lijie > > > > Rodrigo Meneses 于2024年6月5日周三 21:35写道: > > > > > All the best > > > > > >

Re: [ANNOUNCE] New Apache Flink PMC Member - Weijie Guo

2024-06-04 Thread Hangxiang Yu
Congratulations, Weijie! On Tue, Jun 4, 2024 at 11:40 PM Zhanghao Chen wrote: > Congrats, Weijie! > > Best, > Zhanghao Chen > > From: Hang Ruan > Sent: Tuesday, June 4, 2024 16:37 > To: dev@flink.apache.org > Subject: Re: [ANNOUNCE] New Apache Flink PMC Member

[jira] [Created] (FLINK-35460) Check file size when position read for ForSt

2024-05-26 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-35460: Summary: Check file size when position read for ForSt Key: FLINK-35460 URL: https://issues.apache.org/jira/browse/FLINK-35460 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-35389) Implement List Async State API for ForStStateBackend

2024-05-17 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-35389: Summary: Implement List Async State API for ForStStateBackend Key: FLINK-35389 URL: https://issues.apache.org/jira/browse/FLINK-35389 Project: Flink Issue

Re: [DISCUSS] FLIP-444: Native file copy support

2024-05-12 Thread Hangxiang Yu
such interface > extension would be out of scope of this FLIP. In > that case, maybe if this could be extended in the future without breaking > compatibility we could leave it as a > future improvement? > > Best, > Piotrek > > > pt., 10 maj 2024 o 07:40 Hangxiang Yu na

Re: [DISCUSS] FLIP-444: Native file copy support

2024-05-09 Thread Hangxiang Yu
Hi Piotr. Thanks for your proposal. I have some comments, PTAL: 1. +1 about unifying the interface with DuplicatingFileSystem. IIUC, DuplicatingFileSystem also covers the logic from/to both local and remote paths. The implementations could define their own logic about how to fast copy/duplicate

Re: [VOTE] FLIP-447: Upgrade FRocksDB from 6.20.3 to 8.10.0

2024-05-06 Thread Hangxiang Yu
+1(binding) On Mon, May 6, 2024 at 12:25 PM Yuan Mei wrote: > +1(binding) > > Best > Yuan > > On Mon, May 6, 2024 at 11:28 AM Rui Fan <1996fan...@gmail.com> wrote: > > > +1 (binding) > > > > Best, > > Rui > > > > On Mon, May 6, 2024 at 11:01 AM Yanfei Lei wrote: > > > > > +1 (binding) > > > >

[jira] [Created] (FLINK-35268) Support TTL for Async State API

2024-04-29 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-35268: Summary: Support TTL for Async State API Key: FLINK-35268 URL: https://issues.apache.org/jira/browse/FLINK-35268 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-35262) Bridge between AsyncKeyedStateBackend and AsyncExecutionController

2024-04-29 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-35262: Summary: Bridge between AsyncKeyedStateBackend and AsyncExecutionController Key: FLINK-35262 URL: https://issues.apache.org/jira/browse/FLINK-35262 Project: Flink

Re: [DISCUSS] FLIP-447: Upgrade FRocksDB from 6.20.3 to 8.10.0

2024-04-24 Thread Hangxiang Yu
Hi, Yue. Very glad to see that IngestDB will be used to improve the rescaling performance of RocksDB. And +1 for the upgrade. Thanks for the great work! On Thu, Apr 25, 2024 at 5:16 AM Martijn Visser wrote: > +1 > > On Wed, Apr 24, 2024 at 5:31 PM Congxian Qiu > wrote: > > > Thanks for driving

Re: [ANNOUNCE] New Apache Flink PMC Member - Jing Ge

2024-04-15 Thread Hangxiang Yu
Congratulations, Jing! On Mon, Apr 15, 2024 at 11:11 AM spoon_lz wrote: > > > Congratulations! > > Best, > lz > > | | > spoon_lz > | > | > spoon...@126.com > | > > > Replied Message > | From | Biao Geng | > | Date | 04/15/2024 10:40 | > | To | | > | Subject | Re: [ANNOUNCE] New

Re: [ANNOUNCE] New Apache Flink PMC Member - Lincoln Lee

2024-04-15 Thread Hangxiang Yu
Congratulations, Lincoln! On Mon, Apr 15, 2024 at 10:17 AM Zakelly Lan wrote: > Congratulations, Lincoln! > > > Best, > Zakelly > > On Sat, Apr 13, 2024 at 12:48 AM Ferenc Csaky > wrote: > > > Congratulations, Lincoln! > > > > Best, > > Ferenc > > > > > > > > > > On Friday, April 12th, 2024 at

Re: [ANNOUNCE] New Apache Flink Committer - Zakelly Lan

2024-04-15 Thread Hangxiang Yu
Congratulations, Zakelly! On Mon, Apr 15, 2024 at 1:58 PM Yun Tang wrote: > Congratulations, Zakelly! > > Best > Yun Tang > > From: Yanquan Lv > Sent: Monday, April 15, 2024 13:23 > To: dev@flink.apache.org > Subject: Re: [ANNOUNCE] New Apache Flink Committer

Re: [VOTE] FLIP-441: Show the JobType and remove Execution Mode on Flink WebUI

2024-04-11 Thread Hangxiang Yu
+1 (binding) On Fri, Apr 12, 2024 at 10:22 AM Jinzhong Li wrote: > +1 (non binding) > > Bests, > Jinzhong > > On Thu, Apr 11, 2024 at 7:26 AM Muhammet Orazov > wrote: > > > Hey Rui, > > > > +1 (non-binding). > > > > Thanks for driving it! > > > > Best, > > Muhammet > > > > On 2024-04-10

[jira] [Created] (FLINK-35049) Implement Async State API for ForStStateBackend

2024-04-08 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-35049: Summary: Implement Async State API for ForStStateBackend Key: FLINK-35049 URL: https://issues.apache.org/jira/browse/FLINK-35049 Project: Flink Issue Type

[jira] [Created] (FLINK-35048) Implement all methods of AsyncKeyedStateBakend

2024-04-08 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-35048: Summary: Implement all methods of AsyncKeyedStateBakend Key: FLINK-35048 URL: https://issues.apache.org/jira/browse/FLINK-35048 Project: Flink Issue Type

[jira] [Created] (FLINK-35047) Introduce ForStStateBackend

2024-04-08 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-35047: Summary: Introduce ForStStateBackend Key: FLINK-35047 URL: https://issues.apache.org/jira/browse/FLINK-35047 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-35046) Introduce New KeyedStateBackend related Async interfaces

2024-04-08 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-35046: Summary: Introduce New KeyedStateBackend related Async interfaces Key: FLINK-35046 URL: https://issues.apache.org/jira/browse/FLINK-35046 Project: Flink

[jira] [Created] (FLINK-35045) Introduce ForStFileSystem to support reading and writing with ByteBuffer

2024-04-08 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-35045: Summary: Introduce ForStFileSystem to support reading and writing with ByteBuffer Key: FLINK-35045 URL: https://issues.apache.org/jira/browse/FLINK-35045 Project

[jira] [Created] (FLINK-35044) Introduce statebackend-forst module

2024-04-08 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-35044: Summary: Introduce statebackend-forst module Key: FLINK-35044 URL: https://issues.apache.org/jira/browse/FLINK-35044 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-35043) Release beta version of ForSt

2024-04-08 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-35043: Summary: Release beta version of ForSt Key: FLINK-35043 URL: https://issues.apache.org/jira/browse/FLINK-35043 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-34987) Introduce Internal State Interface for Async API

2024-04-02 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-34987: Summary: Introduce Internal State Interface for Async API Key: FLINK-34987 URL: https://issues.apache.org/jira/browse/FLINK-34987 Project: Flink Issue Type

[jira] [Created] (FLINK-34975) FLIP-427: ForSt - Disaggregated state Store

2024-03-31 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-34975: Summary: FLIP-427: ForSt - Disaggregated state Store Key: FLINK-34975 URL: https://issues.apache.org/jira/browse/FLINK-34975 Project: Flink Issue Type: New

Re: [VOTE] FLIP-426: Grouping Remote State Access

2024-03-31 Thread Hangxiang Yu
+1 (binding) On Sun, Mar 31, 2024 at 9:02 PM Yanfei Lei wrote: > +1 (binding) > > Best, > Yanfei > > yue ma 于2024年3月29日周五 16:10写道: > > > > +1 (non-binding) > > > > Jinzhong Li 于2024年3月27日周三 18:57写道: > > > > > Hi devs, > > > > > > I'd like to start a vote on the FLIP-426: Grouping Remote State

[RESULT][VOTE] FLIP-427: Disaggregated state Store

2024-03-31 Thread Hangxiang Yu
I'm happy to announce that FLIP-427: Disaggregated state Store[1] has been accepted with 7 approving votes (4 binding) [2]: - Yuan Mei (binding) - Feifan Wang (non-binding) - Piotr Nowojski (binding) - Rui Fan (binding) - Yun Tang (binding) - Yuepeng Pan (non-binding) - yue ma (non-binding)

Re: [VOTE] FLIP-427: Disaggregated state Store

2024-03-31 Thread Hangxiang Yu
Thanks all for the votes! I'm closing the vote and the result will be posted in a separate mail. On Fri, Mar 29, 2024 at 4:29 PM yue ma wrote: > +1(non-binding) > > Hangxiang Yu 于2024年3月27日周三 18:37写道: > > > Hi devs, > > > > Thanks all for your valuable feedback

Re: [VOTE] FLIP-428: Fault Tolerance/Rescale Integration for Disaggregated State

2024-03-31 Thread Hangxiang Yu
+1 (binding) On Mon, Apr 1, 2024 at 9:24 AM gongzhongqiang wrote: > +1(non-binding) > > Best, > > Zhongqiang Gong > > Jinzhong Li 于2024年3月27日周三 19:31写道: > > > Hi devs, > > > > > > I'd like to start a vote on the FLIP-428: Fault Tolerance/Rescale > > Integration for Disaggregated State [1]. The

Re: [DISCUSS] FLIP-427: Disaggregated State Store

2024-03-31 Thread Hangxiang Yu
UC, the working dir > will be deleted after TaskManager exit. And I think that's enough for > current stage, WDYT ? > > —— > > Best regards, > > Feifan Wang > > > > > At 2024-03-28 12:18:56, "Hangxiang Yu" wrote: > >Hi, Feifan

Re: Re: [ANNOUNCE] Apache Paimon is graduated to Top Level Project

2024-03-28 Thread Hangxiang Yu
Congratulations! On Fri, Mar 29, 2024 at 10:27 AM Benchao Li wrote: > Congratulations! > > Zakelly Lan 于2024年3月29日周五 10:25写道: > > > > Congratulations! > > > > > > Best, > > Zakelly > > > > On Thu, Mar 28, 2024 at 10:13 PM Jing Ge > wrote: > > > > > Congrats! > > > > > > Best regards, > > >

Re: [DISCUSS] FLIP-428: Fault Tolerance/Rescale Integration for Disaggregated State

2024-03-27 Thread Hangxiang Yu
Hi, Yun and Feifan. Thanks for your reply. About the cleanup of working dir, as mentioned in FLIP-427, "The life cycle of working dir is managed as before local strategy.". Since the current working dir and checkpoint dir are separate, The life cycle including creating and cleanup of working dir

Re: Re: [DISCUSS] FLIP-427: Disaggregated State Store

2024-03-27 Thread Hangxiang Yu
>this configuration, e.g. life cycle, supported file systems or storages. > I agree with considering moving this configuration up to the engine level > until there are other disaggreated backends. > > > [1] https://cwiki.apache.org/confluence/x/U4p3EQ > > —— >

Re: [DISCUSS] FLIP-427: Disaggregated State Store

2024-03-27 Thread Hangxiang Yu
t be if we introduce another disaggregated state > storage? Thus, I think `state.backend.disaggregated.working-dir` might be a > better configuration name. > > > Best > Yun Tang > > > From: Hangxiang Yu > Sent: Wednesday, March 20, 2024 11:3

[VOTE] FLIP-427: Disaggregated state Store

2024-03-27 Thread Hangxiang Yu
Hi devs, Thanks all for your valuable feedback about FLIP-427: Disaggregated state Store [1]. I'd like to start a vote on it. The discussion thread is here [2]. The vote will be open for at least 72 hours unless there is an objection or insufficient votes. [1]

Re: [DISCUSS] Flink Website Menu Adjustment

2024-03-25 Thread Hangxiang Yu
Thanks Zhongqiang for driving this. +1 for the proposal. On Tue, Mar 26, 2024 at 1:36 PM Shawn Huang wrote: > +1 for the proposal > > Best, > Shawn Huang > > > Hongshun Wang 于2024年3月26日周二 11:56写道: > > > +1 for the proposal > > > > Best Regards, > > Hongshun Wang > > > > On Tue, Mar 26, 2024 at

Re: [ANNOUNCE] Donation Flink CDC into Apache Flink has Completed

2024-03-21 Thread Hangxiang Yu
Congratulations! Thanks for the efforts. On Fri, Mar 22, 2024 at 10:00 AM Yanfei Lei wrote: > Congratulations! > > Best regards, > Yanfei > > Xuannan Su 于2024年3月22日周五 09:21写道: > > > > Congratulations! > > > > Best regards, > > Xuannan > > > > On Fri, Mar 22, 2024 at 9:17 AM Charles Zhang >

Re: [VOTE] FLIP-433: State Access on DataStream API V2

2024-03-20 Thread Hangxiang Yu
+1 (binding) On Thu, Mar 21, 2024 at 10:04 AM Xintong Song wrote: > +1 (binding) > > Best, > > Xintong > > > > On Wed, Mar 20, 2024 at 8:30 PM weijie guo > wrote: > > > Hi everyone, > > > > > > Thanks for all the feedback about the FLIP-433: State Access on > > DataStream API V2 [1]. The

Re: [DISCUSS] FLIP-427: Disaggregated State Store

2024-03-19 Thread Hangxiang Yu
Hi, Yue. Thanks for the reply. If we use proposal1, we can easily reuse these optimizations .It is even > possible to discuss and review the solution together in the Rocksdb > community. We also saw these useful optimizations which could be applied to ForSt in the future. But IIUC, it's not

Re: [DISCUSS] FLIP-427: Disaggregated State Store

2024-03-19 Thread Hangxiang Yu
quot; among multiple workers. >> In these cases, the engine spawns huge amount of scan range requests to >> the >> file system to retrieve different parts of a file. >> How the proposed solution would work with these workloads? >> >> - The similar question r

Re: [ANNOUNCE] Apache Flink 1.19.0 released

2024-03-18 Thread Hangxiang Yu
Congratulations! Thanks release managers and all involved! On Mon, Mar 18, 2024 at 5:23 PM Hang Ruan wrote: > Congratulations! > > Best, > Hang > > Paul Lam 于2024年3月18日周一 17:18写道: > > > Congrats! Thanks to everyone involved! > > > > Best, > > Paul Lam > > > > > 2024年3月18日 16:37,Samrat Deb 写道:

[jira] [Created] (FLINK-34660) AutoRescalingITCase#testCheckpointRescalingInKeyedState AssertionError

2024-03-13 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-34660: Summary: AutoRescalingITCase#testCheckpointRescalingInKeyedState AssertionError Key: FLINK-34660 URL: https://issues.apache.org/jira/browse/FLINK-34660 Project

[jira] [Created] (FLINK-34652) Use CheckpointStreamFactory for StateChangeFsUploader

2024-03-12 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-34652: Summary: Use CheckpointStreamFactory for StateChangeFsUploader Key: FLINK-34652 URL: https://issues.apache.org/jira/browse/FLINK-34652 Project: Flink Issue

Re: [DISCUSS] FLIP-433: State Access on DataStream API V2

2024-03-11 Thread Hangxiang Yu
Graph is generated), so it can't rely on > computations > > > > > that are executed after deploy to TM. > > > > > >> > > > > > >> > > > > > >> > > > > > > >> Currently state access is pretty dynamic in Flink and I would

Re: [VOTE] Release 1.19.0, release candidate #2

2024-03-11 Thread Hangxiang Yu
+1 (non-binding) - Verified signatures and checksums - Reviewed Web PR - Built from source successfully - Ran a wordcount job which worked well On Tue, Mar 12, 2024 at 1:00 AM Jeyhun Karimov wrote: > +1 (non binding) > > - verified that source distribution does not contain binaries > -

Re: [DISCUSS] FLIP-427: Disaggregated State Store

2024-03-10 Thread Hangxiang Yu
above applies also for caching ( I > know caching is subject of FLIP-429, asking here becasue of the related > section in this FLIP). > > Regards, > Jeyhun > > [1] https://blog.min.io/challenge-big-data-small-files/ > > > > On Thu, Mar 7, 2024 at 10:09 AM Hangxiang Y

[DISCUSS] FLIP-427: Disaggregated State Store

2024-03-07 Thread Hangxiang Yu
Hi devs, I'd like to start a discussion on a sub-FLIP of FLIP-423: Disaggregated State Storage and Management[1], which is a joint work of Yuan Mei, Zakelly Lan, Jinzhong Li, Hangxiang Yu, Yanfei Lei and Feng Wang: - FLIP-427: Disaggregated State Store This FLIP introduces the initial version

Re: [DISCUSS] FLIP-433: State Access on DataStream API V2

2024-03-06 Thread Hangxiang Yu
Hi, Weijie. Thanks for your proposal. I'd like to start the discussion with some questions: 1. We have also discussed in FLIP-359/FLINK-32658 about limiting the user operation to avoid creating state when processElement. Could current interfaces also help this? 2. Could you provide more examples

Re: [VOTE] FLIP-420: Add API annotations for RocksDB StateBackend user-facing classes

2024-03-06 Thread Hangxiang Yu
+1 (binding) On Thu, Mar 7, 2024 at 9:34 AM Yun Tang wrote: > > +1 for this FLIP. > Sorry for not being clear in my previous reply, it's a binding vote. > > Best > Yun Tang > > From: Jeyhun Karimov > Sent: Thursday, March 7, 2024 4:40 > To: dev@flink.apache.org

Re: [DISCUSS] Move CheckpointingMode to flink-core

2024-02-26 Thread Hangxiang Yu
Hi, Zakelly. Thanks for driving this. Moving this class to flink-core makes sense to me which could make the code path and configs clearer. It's marked as @Public from 1.0 and 1.20 should be the next long-term version, so 1.19 should have been a suitable version to do it. And also look forward to

[jira] [Created] (FLINK-34512) Thrown root cause for  HandlerRequestException

2024-02-25 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-34512: Summary: Thrown root cause for  HandlerRequestException Key: FLINK-34512 URL: https://issues.apache.org/jira/browse/FLINK-34512 Project: Flink Issue Type

[jira] [Created] (FLINK-34479) Fix missed changelog configs in the documentation

2024-02-20 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-34479: Summary: Fix missed changelog configs in the documentation Key: FLINK-34479 URL: https://issues.apache.org/jira/browse/FLINK-34479 Project: Flink Issue Type

[jira] [Created] (FLINK-34352) Improve the documentation of allowNonRestoredState

2024-02-04 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-34352: Summary: Improve the documentation of allowNonRestoredState Key: FLINK-34352 URL: https://issues.apache.org/jira/browse/FLINK-34352 Project: Flink Issue

Re: [DISCUSS]FLIP-420: Add API annotations for RocksDB StateBackend user-facing classes

2024-01-24 Thread Hangxiang Yu
Hi Jinzhong. Thanks for driving this! Some suggestions: 1. As RocksDBStateBackend marked as Deprecated, We should also mark RocksDBStateBackendFactory as Deprecated 2. Since 1.19 will be freezed in 1.26. Let's adjust the target version to 1.20 On Wed, Jan 24, 2024 at 11:50 PM Zakelly Lan wrote:

Re: [VOTE] FLIP-406: Reorganize State & Checkpointing & Recovery Configuration

2024-01-24 Thread Hangxiang Yu
+1 (binding) On Thu, Jan 25, 2024 at 8:49 AM Rui Fan <1996fan...@gmail.com> wrote: > +1(binding) > > Best, > Rui > > On Wed, 24 Jan 2024 at 21:50, Zakelly Lan wrote: > > > Hi everyone, > > > > I'd like to start a vote on the FLIP-406: Reorganize State & > Checkpointing > > & Recovery

Re: [VOTE] FLIP-416: Deprecate and remove the RestoreMode#LEGACY

2024-01-18 Thread Hangxiang Yu
+1 (binding) On Fri, Jan 19, 2024 at 12:10 PM Zakelly Lan wrote: > Hi everyone, > > I'd like to start a vote on the FLIP-416: Deprecate and remove the > RestoreMode#LEGACY [1]. The discussion thread is here [2]. > > The vote will be open for at least 72 hours unless there is an objection or >

[jira] [Created] (FLINK-34119) Improve description about changelog in document

2024-01-16 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-34119: Summary: Improve description about changelog in document Key: FLINK-34119 URL: https://issues.apache.org/jira/browse/FLINK-34119 Project: Flink Issue Type

Re: [DISCUSS] FLIP-416: Deprecate and remove the RestoreMode#LEGACY

2024-01-14 Thread Hangxiang Yu
Hi, Zakelly. Thanks for the quick feedback and driving this. +1 for removing LEGACY mode in Flink 2.0. On Mon, Jan 15, 2024 at 3:23 AM Danny Cranmer wrote: > +1 to removing LEGACY mode in Flink 2.0. Thanks for driving. > > Danny, > > On Sat, 13 Jan 2024, 08:20 Yanfei Lei, wrote: > > > Thanks

Re: [DISCUSS] FLIP-414: Support Retry Mechanism in RocksDBStateDataTransfer

2024-01-11 Thread Hangxiang Yu
Thanks for driving this. Retry mechanism is common when we want to get or put data by network. So I think it will help when checkpoint failure due to temporary network problems, of course it may increase a bit overhead for some other reasons. Some comments and suggestions: 1. Since Flink has a

Re: Re: Re: [VOTE] Accept Flink CDC into Apache Flink

2024-01-10 Thread Hangxiang Yu
+1 (non-binding) On Thu, Jan 11, 2024 at 11:19 AM Xuannan Su wrote: > +1 (non-binding) > > Best, > Xuannan > > On Thu, Jan 11, 2024 at 10:28 AM Xuyang wrote: > > > > +1 (non-binding)-- > > > > Best! > > Xuyang > > > > > > > > > > > > 在 2024-01-11 10:00:11,"Yang Wang" 写道: > > >+1

Re: [DISCUSS] FLIP-406: Reorganize State & Checkpointing & Recovery Configuration

2024-01-10 Thread Hangxiang Yu
> >> if execution.checkpointing.local-copy.enabled is true and > >> execution.recovery.from-local is false ? I found it's also an issue if > >> current local-recovery from enabled to disabled. Maybe another ticket is > >> needed. > > > > > > IIUC, the

[jira] [Created] (FLINK-34051) Fix equals/hashCode/toString for SavepointRestoreSettings

2024-01-10 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-34051: Summary: Fix equals/hashCode/toString for SavepointRestoreSettings Key: FLINK-34051 URL: https://issues.apache.org/jira/browse/FLINK-34051 Project: Flink

[jira] [Created] (FLINK-34032) Cleanup local=recovery dir when switching local-recovery from enabled to disabled

2024-01-08 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-34032: Summary: Cleanup local=recovery dir when switching local-recovery from enabled to disabled Key: FLINK-34032 URL: https://issues.apache.org/jira/browse/FLINK-34032

Re: [DISCUSS] FLIP-406: Reorganize State & Checkpointing & Recovery Configuration

2024-01-08 Thread Hangxiang Yu
Hi, Zakelly. Thanks for driving this. Overall LGTM as we discussed offline. Some comments/suggestions just came to mind: 1. Could execution.recovery also contain some other behaviors about recovery ? e.g. restart-strategy. 2. Could we also remove some legacy configuration value ? e.g. LEGACY Mode

[jira] [Created] (FLINK-34030) Avoid using negative value for periodic-materialize.interval

2024-01-08 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-34030: Summary: Avoid using negative value for periodic-materialize.interval Key: FLINK-34030 URL: https://issues.apache.org/jira/browse/FLINK-34030 Project: Flink

Re: [VOTE] FLIP-385: Add OpenTelemetryTraceReporter and OpenTelemetryMetricReporter

2023-11-21 Thread Hangxiang Yu
+1(binding) On Wed, Nov 22, 2023 at 10:29 AM Rui Fan <1996fan...@gmail.com> wrote: > +1(binding) > > Best, > Rui > > On Wed, Nov 22, 2023 at 1:20 AM Piotr Nowojski > wrote: > > > Hi All, > > > > I'd like to start a vote on the FLIP-385: Add OpenTelemetryTraceReporter > > and

Re: [VOTE] FLIP-390: Support System out and err to be redirected to LOG or discarded

2023-11-21 Thread Hangxiang Yu
+1 (binding) Thanks for your efforts! On Mon, Nov 20, 2023 at 11:53 AM Rui Fan <1996fan...@gmail.com> wrote: > Hi everyone, > > Thank you to everyone for the feedback on FLIP-390: Support > System out and err to be redirected to LOG or discarded[1] > which has been discussed in this thread [2].

Re: [VOTE] FLIP-384: Introduce TraceReporter and use it to create checkpointing and recovery traces

2023-11-21 Thread Hangxiang Yu
+1 (binding) Thanks for driving this again! On Wed, Nov 22, 2023 at 10:30 AM Rui Fan <1996fan...@gmail.com> wrote: > +1(binding) > > Best, > Rui > > On Wed, Nov 22, 2023 at 6:43 AM Jing Ge > wrote: > > > +1(binding) Thanks! > > > > Best regards, > > Jing > > > > On Tue, Nov 21, 2023 at 6:17 PM

Re: [DISCUSS] FLIP-390: Support System out and err to be redirected to LOG or discarded

2023-11-14 Thread Hangxiang Yu
> because the thread name of the default Task Thread contains the > task name and subtask id. And this solution can support all threads, > because some non-task threads may also call println. > > [1] https://cwiki.apache.org/confluence/x/4guZE > > Best, > Rui > > On W

Re: [DISCUSS] FLIP-390: Support System out and err to be redirected to LOG or discarded

2023-11-14 Thread Hangxiang Yu
t; > > > Redirect it to LOG.info may be a reasonable and easy choice. > > > > The user didn't really want to log into taskmanager.out, it just > > > > happened by accident. > > > > > > > > > > > > > > > > To Hangxiang: > >

Re: [DISCUSS] FLIP-384: Introduce TraceReporter and use it to create checkpointing and recovery traces

2023-11-08 Thread Hangxiang Yu
Hi, Piotr. Thanks for the proposal. Just as we discussed in FLINK-23411, +1 for supporting trace/span to monitor metrics like checkpoint and recovery. We could also do many things based on this mechanism: 1. more fine-grained metrics about checkpoint and recovery. For example, some stage info

Re: [DISCUSS] FLIP-390: Support System out and err to be redirected to LOG or discarded

2023-11-08 Thread Hangxiang Yu
Hi, Rui. Thanks for the proposal. It sounds reasonable. I have some questions, PTAL: 1. I have a similar concern as Feng. Will we redirect to another log file not taskManager.log ? taskManager.log contains lots of important information like init log. It will be rolled quickly if we redirect out

Re: [ANNOUNCE] The Flink Speed Center and benchmark daily run are back online

2023-10-20 Thread Hangxiang Yu
Thanks Zakelly for the great work! On Fri, Oct 20, 2023 at 5:05 PM Rui Fan <1996fan...@gmail.com> wrote: > Thanks for your effort! It's very useful when some new commits affect > performance. > > Best, > Rui > > On Fri, Oct 20, 2023 at 4:42 PM Yanfei Lei wrote: > > > Thanks for your hard work!

Re: [DISCUSS] FLIP-368 Reorganize the exceptions thrown in state interfaces

2023-09-19 Thread Hangxiang Yu
Hi, Zakelly. Thanks for the proposal. +1 for reorganizing exceptions of state interfaces which indeed confuses me currently. >From my experience, users usually omit these exceptions because they cannot do much even if they catch the exceptions. I have some problems and suggestions, PTAL:

[jira] [Created] (FLINK-33055) Correct the error value about 'state.backend.type' in the document

2023-09-07 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-33055: Summary: Correct the error value about 'state.backend.type' in the document Key: FLINK-33055 URL: https://issues.apache.org/jira/browse/FLINK-33055 Project: Flink

Re: [DISCUSS] Add config to enable job stop with savepoint on exceeding tolerable checkpoint Failures

2023-09-06 Thread Hangxiang Yu
Hi, Dongwoo. IIUC, you mean using savepoint to store a snapshot to other storage if checkpoints fail multiple times due to some long lasting exceptions of external storage, right ? I think it's better to achieve this by an external tool instead of introducing a config like that: 1. it's not so

[jira] [Created] (FLINK-32601) Unstable RemoteChannelThroughputBenchmark_remoteRebalance_jmhTest

2023-07-17 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-32601: Summary: Unstable RemoteChannelThroughputBenchmark_remoteRebalance_jmhTest Key: FLINK-32601 URL: https://issues.apache.org/jira/browse/FLINK-32601 Project: Flink

[jira] [Created] (FLINK-32364) Add Rescaling benchmark for ChangelogStateBackend

2023-06-15 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-32364: Summary: Add Rescaling benchmark for ChangelogStateBackend Key: FLINK-32364 URL: https://issues.apache.org/jira/browse/FLINK-32364 Project: Flink Issue Type

Re: [VOTE] FLIP-306: Unified File Merging Mechanism for Checkpoints

2023-05-09 Thread Hangxiang Yu
Hi Zakelly. Thanks for driving this. +1 (no-binding) On Wed, May 10, 2023 at 10:52 AM Yuan Mei wrote: > Thanks for driving this, Zakelly. > > As discussed in the thread, > > +1 for the proposal (binding) > > Best, > > Yuan > > > > On Wed, May 10, 2023 at 10:39 AM Zakelly Lan > wrote: > > > Hi

[jira] [Created] (FLINK-31875) OSS throwns NoClassDefFoundError due to old hadoop-common version

2023-04-20 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-31875: Summary: OSS throwns NoClassDefFoundError due to old hadoop-common version Key: FLINK-31875 URL: https://issues.apache.org/jira/browse/FLINK-31875 Project: Flink

[jira] [Created] (FLINK-31366) Exception is thrown when s3a and s3p are used together

2023-03-08 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-31366: Summary: Exception is thrown when s3a and s3p are used together Key: FLINK-31366 URL: https://issues.apache.org/jira/browse/FLINK-31366 Project: Flink Issue

[jira] [Created] (FLINK-30854) Expose periodic compaction to RocksdbCompactFilterCleanupStrategy

2023-01-31 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-30854: Summary: Expose periodic compaction to RocksdbCompactFilterCleanupStrategy Key: FLINK-30854 URL: https://issues.apache.org/jira/browse/FLINK-30854 Project: Flink

[jira] [Created] (FLINK-30656) Provide more logs for schema compatibility check

2023-01-12 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-30656: Summary: Provide more logs for schema compatibility check Key: FLINK-30656 URL: https://issues.apache.org/jira/browse/FLINK-30656 Project: Flink Issue Type

Re: [ANNOUNCE] New Apache Flink Committer - Lincoln Lee

2023-01-09 Thread Hangxiang Yu
Congratulations, Lincoln! On Tue, Jan 10, 2023 at 1:34 PM Dian Fu wrote: > Congratulations, Lincoln! > > Regards, > Dian > > On Tue, Jan 10, 2023 at 1:31 PM weijie guo > wrote: > > > Congratulations, Lincoln! > > > > Best regards, > > > > Weijie > > > > > > Lijie Wang 于2023年1月10日周二 12:24写道: >

[jira] [Created] (FLINK-30614) Improve resolving schema compatibility -- Milestone two

2023-01-09 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-30614: Summary: Improve resolving schema compatibility -- Milestone two Key: FLINK-30614 URL: https://issues.apache.org/jira/browse/FLINK-30614 Project: Flink

[jira] [Created] (FLINK-30613) Improve resolving schema compatibility -- Milestone one

2023-01-09 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-30613: Summary: Improve resolving schema compatibility -- Milestone one Key: FLINK-30613 URL: https://issues.apache.org/jira/browse/FLINK-30613 Project: Flink

[jira] [Created] (FLINK-30450) FileSystem supports exporting client-side metrics

2022-12-18 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-30450: Summary: FileSystem supports exporting client-side metrics Key: FLINK-30450 URL: https://issues.apache.org/jira/browse/FLINK-30450 Project: Flink Issue Type

[jira] [Created] (FLINK-30345) Improve the serializer performace of state change of changelog

2022-12-08 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-30345: Summary: Improve the serializer performace of state change of changelog Key: FLINK-30345 URL: https://issues.apache.org/jira/browse/FLINK-30345 Project: Flink

Re: [ANNOUNCE] New Apache Flink PMC Members - Godfrey He, Xingbo Huang

2022-11-24 Thread Hangxiang Yu
Congratulations, Godfrey and Xingbo! On Thu, Nov 24, 2022 at 4:12 PM Jingsong Li wrote: > Congrats, Godfrey and Xingbo! > > Best, > Jingsong > > On Thu, Nov 24, 2022 at 4:03 PM Sergey Nuyanzin > wrote: > > > > Congrats to both! > > > > On Thu, Nov 24, 2022 at 8:59 AM Yun Gao > > wrote: > > >

[jira] [Created] (FLINK-29844) FLIP-263: Improve resolving schema compatibility

2022-11-01 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-29844: Summary: FLIP-263: Improve resolving schema compatibility Key: FLINK-29844 URL: https://issues.apache.org/jira/browse/FLINK-29844 Project: Flink Issue Type

[RESULT][VOTE] FLIP-263: Improve resolving schema compatibility

2022-11-01 Thread Hangxiang Yu
Hi everyone, Happy to announce that FLIP-263 [1] has been accepted. >From vote thread [2], There are 9 approving votes, 7 of which are binding: - Yanfei Lei (non-binding) - Yuan Mei (binding) - Zakelly Lan (non-binding) - Yun Gao (binding) - Godfrey He (binding) - Dawid Wysakowicz (binding)

[jira] [Created] (FLINK-29802) ChangelogStateBackend supports native savepoint

2022-10-31 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-29802: Summary: ChangelogStateBackend supports native savepoint Key: FLINK-29802 URL: https://issues.apache.org/jira/browse/FLINK-29802 Project: Flink Issue Type

Re: [ANNOUNCE] Apache Flink 1.16.0 released

2022-10-30 Thread Hangxiang Yu
Congratulations! Thanks Chesnay, Martijn, Godfrey & Xingbo for managing the release. On Fri, Oct 28, 2022 at 7:35 PM Jing Ge wrote: > Congrats! > > On Fri, Oct 28, 2022 at 1:22 PM 任庆盛 wrote: > >> Congratulations and a big thanks to Chesnay, Martijn, Godfrey and Xingbo >> for the awesome work

Re: [VOTE] Drop TypeSerializerConfigSnapshot and savepoint support from Flink versions < 1.8.0

2022-10-30 Thread Hangxiang Yu
+1 (non-binding) On Sat, Oct 29, 2022 at 2:21 AM Tzu-Li (Gordon) Tai wrote: > +1 > > On Fri, Oct 28, 2022 at 10:21 AM Konstantin Knauf > wrote: > > > +1 (binding) > > > > Am Fr., 28. Okt. 2022 um 16:58 Uhr schrieb Piotr Nowojski < > > pnowoj...@apache.org>: > > > > > Hi, > > > > > > As

Re: [DISCUSS] FLIP-263: Improve resolving schema compatibility

2022-10-30 Thread Hangxiang Yu
:Re: [DISCUSS] FLIP-263: Improve resolving schema compatibility > > Hey Huangxiang, > > The section of `Rejected Alternatives` may also need an update. > > Current plan sounds like a reasonable one. I am fine with it. > > Thanks for driving this. > > Best >

[VOTE] FLIP-263: Improve resolving schema compatibility

2022-10-27 Thread Hangxiang Yu
Hi everyone, I'd like to start the vote for FLIP-263 [1]. Thanks for your feedback and the discussion in [2][3]. The vote will be open for at least 72 hours. Best regards, Hangxiang. [1] https://cwiki.apache.org/confluence/display/FLINK/FLIP-263%3A+Improve+resolving+schema+compatibility [2]

[jira] [Created] (FLINK-29777) [JUnit5 Migration] Module: flink-dstl

2022-10-27 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-29777: Summary: [JUnit5 Migration] Module: flink-dstl Key: FLINK-29777 URL: https://issues.apache.org/jira/browse/FLINK-29777 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-29776) [JUnit5 Migration] Module: flink-statebackend-changelog

2022-10-27 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-29776: Summary: [JUnit5 Migration] Module: flink-statebackend-changelog Key: FLINK-29776 URL: https://issues.apache.org/jira/browse/FLINK-29776 Project: Flink

[jira] [Created] (FLINK-29775) [JUnit5 Migration] Module: flink-statebackend-rocksdb

2022-10-27 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-29775: Summary: [JUnit5 Migration] Module: flink-statebackend-rocksdb Key: FLINK-29775 URL: https://issues.apache.org/jira/browse/FLINK-29775 Project: Flink Issue

Re: [ANNOUNCE] Performance Daily Monitoring Moved from Ververica to Apache Flink Slack Channel

2022-10-26 Thread Hangxiang Yu
Hi, Yanfei. Thanks for driving this. It could help us to detect and resolve the regression problem quickly and officially. I'd like to join as a maintainer. Looking forward to the workflow. On Wed, Oct 26, 2022 at 5:18 PM Yuan Mei wrote: > Thanks, Yanfei, to drive this and make the performance

Re: [DISCUSS] FLIP-263: Improve resolving schema compatibility

2022-10-25 Thread Hangxiang Yu
? Look forward to your reply! On Tue, Oct 25, 2022 at 1:31 PM Hangxiang Yu wrote: > Hi, everyone. > > Thanks for your suggestions! > > Let me summarize the remaining questions in the thread and share my ideas > based on your suggestions: > >1. Should we put th

Re: Limiting backpressure during checkpoints

2022-10-25 Thread Hangxiang Yu
Hi Robin. Could you share how you got the metric of CPU usage ? By summing all used CPU cores of TMs or evaluating it by the busy metric in Flink UI ? I think it's the first thing we need to align. > network (async) part of the checkpoint should in theory not cause backpressure since resources

  1   2   >