Re: [DISCUSS] Should Configuration support getting value based on String key?

2023-12-13 Thread Gyula Fóra
Hi Xintong, I don’t really see the actual practical benefit from removing the getstring and setstring low level methods. I understand that ConfigOptions are nicer for 95% of the cases but from a technical point of view there will always be cases where a more flexible dynamic handling is

Re: [VOTE] FLIP-383: Support Job Recovery for Batch Jobs

2023-12-13 Thread Xintong Song
+1 (binding) Best, Xintong On Thu, Dec 14, 2023 at 3:15 PM Lijie Wang wrote: > Hi devs, Thanks for all feedback about the FLIP-383: Support Job Recovery > for Batch Jobs[1]. This FLIP was discussed in [2]. > > I'd like to start a vote for it. The vote will be open for at least 72 > hours

Re:Re: [DISCUSS] FLIP-392: Deprecate the Legacy Group Window Aggregation

2023-12-13 Thread Xuyang
Hi, Timo, Sergey and Lincoln Lee. Thanks for your feedback. > In my opinion the FLIP touches too many > topics at the same time and should be split into multiple FLIPs. We > should > stay focused on what is needed for Flink 2.0. The main goal and topic of this Flip is to align the abilities

[VOTE] FLIP-383: Support Job Recovery for Batch Jobs

2023-12-13 Thread Lijie Wang
Hi devs, Thanks for all feedback about the FLIP-383: Support Job Recovery for Batch Jobs[1]. This FLIP was discussed in [2]. I'd like to start a vote for it. The vote will be open for at least 72 hours (until December 19th 12:00 GMT) unless there is an objection or insufficient votes. [1]

Re: [DISCUSS] Should Configuration support getting value based on String key?

2023-12-13 Thread Xintong Song
> > IIUC, you both prefer using ConfigOption instead of string keys for > all use cases, even internal ones. We can even forcefully delete > these @Depreated methods in Flink-2.0 to guide users or > developers to use ConfigOption. > Yes, at least from my side. I noticed that Configuration is

Re: [DISCUSS] Should Configuration support getting value based on String key?

2023-12-13 Thread Rui Fan
Thanks Xintong and Xuannan for the feedback! IIUC, you both prefer using ConfigOption instead of string keys for all use cases, even internal ones. We can even forcefully delete these @Depreated methods in Flink-2.0 to guide users or developers to use ConfigOption. Using ConfigOption is feasible

Re: [VOTE] Release flink-connector-pulsar 4.1.0, release candidate #1

2023-12-13 Thread tison
Thanks Leonard for driving this release! +1 (non-binding) * Download link valid * Maven staging artifacts look good. * Checksum and gpg matches * LICENSE and NOTICE exist * Can build from source. Best, tison. Rui Fan <1996fan...@gmail.com> 于2023年12月14日周四 09:23写道: > > Thanks Leonard for driving

Re: [VOTE] FLIP-396: Trial to test GitHub Actions as an alternative for Flink's current Azure CI infrastructure

2023-12-13 Thread Xuannan Su
Thanks for driving this! +1(non-binding) Best, Xuannan On Thu, Dec 14, 2023 at 11:04 AM Xintong Song wrote: > > +1 (binding) > > Best, > > Xintong > > > > On Thu, Dec 14, 2023 at 10:05 AM Jiabao Sun > wrote: > > > Thanks Matthias for this hard work! > > > > +1(non-binding) > > > > Best, > >

Re: [DISCUSS] Should Configuration support getting value based on String key?

2023-12-13 Thread Xintong Song
I'm leaning towards not allowing string-key based configuration access in the long term. I see the Configuration being used in two different ways: 1. Writing / reading a specific option. In such cases, ConfigOption has many advantages compared to string-key, such as explicit value type,

[jira] [Created] (FLINK-33819) Suppor setting CompressType in RocksDBStateBackend

2023-12-13 Thread Yue Ma (Jira)
Yue Ma created FLINK-33819: -- Summary: Suppor setting CompressType in RocksDBStateBackend Key: FLINK-33819 URL: https://issues.apache.org/jira/browse/FLINK-33819 Project: Flink Issue Type:

Re: [VOTE] FLIP-396: Trial to test GitHub Actions as an alternative for Flink's current Azure CI infrastructure

2023-12-13 Thread Xintong Song
+1 (binding) Best, Xintong On Thu, Dec 14, 2023 at 10:05 AM Jiabao Sun wrote: > Thanks Matthias for this hard work! > > +1(non-binding) > > Best, > Jiabao > > > > 2023年12月14日 09:57,Leonard Xu 写道: > > > > +1(binding) > > > > > > Best, > > Leonard > > > >> 2023年12月13日 下午10:59,Benchao Li 写道:

Re:Re: [DISCUSS][2.0] FLIP-382: Unify the Provision of Diverse Metadata for Context-like APIs

2023-12-13 Thread Wencong Liu
Hi Lijie, Thank you for the response and for supporting the proposal. Regarding your suggestions: 1. While the getTaskNameWithIndexAndAttemptNumber may seem redundant initially, this concatenated string, for example "MyTask (3/6)#1", serves as an identifier used to denote a specific Task across

Re: [DISCUSS] Should Configuration support getting value based on String key?

2023-12-13 Thread Xuannan Su
Hi Rui, We are currently revisiting all the configurations for Flink 2.0, and it turns out that many string-based configurations in `ConfigConstants` are deprecated and have been replaced by `ConfigOptions`. Since `ConfigOptions` offers many advantages over string-based configurations for the end

Re: [VOTE] FLIP-396: Trial to test GitHub Actions as an alternative for Flink's current Azure CI infrastructure

2023-12-13 Thread Jiabao Sun
Thanks Matthias for this hard work! +1(non-binding) Best, Jiabao > 2023年12月14日 09:57,Leonard Xu 写道: > > +1(binding) > > > Best, > Leonard > >> 2023年12月13日 下午10:59,Benchao Li 写道: >> >> +1 (binding) >> >> Thanks Matthias for driving it! >> >> Etienne Chauchot 于2023年12月13日周三 21:35写道:

Re: [VOTE] FLIP-396: Trial to test GitHub Actions as an alternative for Flink's current Azure CI infrastructure

2023-12-13 Thread Leonard Xu
+1(binding) Best, Leonard > 2023年12月13日 下午10:59,Benchao Li 写道: > > +1 (binding) > > Thanks Matthias for driving it! > > Etienne Chauchot 于2023年12月13日周三 21:35写道: >> >> Thanks Matthias for your hard work ! >> >> +1 (binding) >> >> Best >> >> Etienne >> >> Le 12/12/2023 à 11:23, Lincoln

Re: [VOTE] Release flink-connector-pulsar 4.1.0, release candidate #1

2023-12-13 Thread Rui Fan
Thanks Leonard for driving this release! +1 (non-binding) - Validated checksum hash - Verified signature - Verified that no binaries exist in the source archive - Build the source with Maven and jdk8 - Verified licenses - Verified web PRs, left a minor comment Best, Rui On Wed, Dec 13, 2023 at

[jira] [Created] (FLINK-33818) Implement restore tests for WindowDeduplicate node

2023-12-13 Thread Bonnie Varghese (Jira)
Bonnie Varghese created FLINK-33818: --- Summary: Implement restore tests for WindowDeduplicate node Key: FLINK-33818 URL: https://issues.apache.org/jira/browse/FLINK-33818 Project: Flink

[jira] [Created] (FLINK-33817) Allow ReadDefaultValues = False for non primitive types on Proto3

2023-12-13 Thread Sai Sharath Dandi (Jira)
Sai Sharath Dandi created FLINK-33817: - Summary: Allow ReadDefaultValues = False for non primitive types on Proto3 Key: FLINK-33817 URL: https://issues.apache.org/jira/browse/FLINK-33817 Project:

Edit permissions request

2023-12-13 Thread Alex Nitavsky
Hello, I would love to initiate FLIP in order to extend Apache Curator configuration: https://issues.apache.org/jira/browse/FLINK-33376. Would it be please possible to get edit permissions for Apache Flink confluence space please? Thanks

Re: [DISCUSS] Allow TwoPhaseCommittingSink WithPreCommitTopology to alter the type of the Committable

2023-12-13 Thread Gyula Fóra
Thank you Peter Vary for updating the FLIP based on the discussions. I really like the improvements introduced by the mixin interfaces which now aligns much better with the source and table connectors. While this introduces some breaking changes to the existing connectors, this is a technical

Re: [DISCUSS] Allow TwoPhaseCommittingSink WithPreCommitTopology to alter the type of the Committable

2023-12-13 Thread Péter Váry
I have updated the FLIP-372 [1] based on the comments from multiple sources. Moved to the mixin approach as this seems to be the consensus based on this thread [2] Also created a draft implementation [3] PR, so I can test the changes and default implementations (no new tests yet) Please provide

Re: [DISCUSS] Resolve diamond inheritance of Sink.createWriter

2023-12-13 Thread Péter Váry
The PR is merged, so everything should be back to normal. Thanks everyone involved! Also based on the discussion here, I have updated the FLIP-372 [1]. I tried to incorporate every comment, but in some cases they were contradictory, so please share your opinion on the corresponding thread [2].

Re: [VOTE] FLIP-396: Trial to test GitHub Actions as an alternative for Flink's current Azure CI infrastructure

2023-12-13 Thread Benchao Li
+1 (binding) Thanks Matthias for driving it! Etienne Chauchot 于2023年12月13日周三 21:35写道: > > Thanks Matthias for your hard work ! > > +1 (binding) > > Best > > Etienne > > Le 12/12/2023 à 11:23, Lincoln Lee a écrit : > > +1 (binding) > > > > Thanks for driving this! > > > > Best, > > Lincoln Lee >

Re: [VOTE] FLIP-401: REST API JSON response deserialization unknown field tolerance

2023-12-13 Thread Márton Balassi
+1 (binding) On Tue, Dec 12, 2023 at 4:16 PM Rodrigo Meneses wrote: > +1 > > On Tue, Dec 12, 2023 at 6:58 AM Maximilian Michels wrote: > > > +1 (binding) > > > > On Tue, Dec 12, 2023 at 2:23 PM Peter Huang > > wrote: > > > > > > +1 Non-binding > > > > > > > > > Peter Huang > > > > > > Őrhidi

Re: [VOTE] FLIP-396: Trial to test GitHub Actions as an alternative for Flink's current Azure CI infrastructure

2023-12-13 Thread Etienne Chauchot
Thanks Matthias for your hard work ! +1 (binding) Best Etienne Le 12/12/2023 à 11:23, Lincoln Lee a écrit : +1 (binding) Thanks for driving this! Best, Lincoln Lee Yun Tang 于2023年12月12日周二 17:52写道: Thanks for Matthias driving this work! +1 (binding) Best Yun Tang

Re: [DISCUSS] Should Configuration support getting value based on String key?

2023-12-13 Thread Rui Fan
Thanks Zhu for the quick response! > It is not a blocker of the deprecation, epsecially given that they are not standard > configuration and are just using Configuration class for convenience. Yes, it's not a blocker of deprecation. > These are internal usages and we can have an internal getter

[SUMMARY] Flink 1.19 Release Sync 12/12/2023

2023-12-13 Thread Lincoln Lee
Hi devs, This is the summary of the 3rd release sync of Flink 1.19: - *Features & issues tracking* There're 7 weeks until the feature freeze date, so far we've had 23 flips/features and the status looks good. It is encouraged to continuously updating the 1.19 wiki page[1] for contributors. -

Re: join flink slack

2023-12-13 Thread Romit Mahanta
Second that On Wed, 13 Dec, 2023, 5:58 pm yd c, wrote: > can u invite flink slack >

join flink slack

2023-12-13 Thread yd c
can u invite flink slack

Re: [DISCUSS] Should Configuration support getting value based on String key?

2023-12-13 Thread Zhu Zhu
Hi Rui, I'd like to understand why there is a strong requirement for these deprecated methods. The ConfigOption param methods help to do the type conversion so that users do not need to do it by themselves. For the 2 reasons to keep these methods mentioned above: > 1. A lot of scenarios don't

Re: [PROPOSAL] Contribute Flink CDC Connectors project to Apache Flink

2023-12-13 Thread Leonard Xu
Hey, Zarzycki Thanks for your reply and recognization ! > As I understand you're thinking with Flink CDC 3.0 to make the project > broader than CDC, as more "Streaming Integration" framework. This will > resemble Kafka Connect, what about renaming the project to something like > "Flink

Re: [DISCUSS] Should Configuration support getting value based on String key?

2023-12-13 Thread Rui Fan
Thanks Martijn for the quick clarification! I see Zhu Zhu and Junrui Lee are working on configuration related work of Flink-2.0. I would cc them, and hear some thoughts from them. Best, Rui On Wed, Dec 13, 2023 at 7:17 PM Martijn Visser wrote: > Hi Rui, > > I'm more wondering if part of the

Re: [DISCUSS] Should Configuration support getting value based on String key?

2023-12-13 Thread Martijn Visser
Hi Rui, I'm more wondering if part of the configuration layer changes would mean that these APIs would be removed in 2.0. Because if so, then I don't think we should remove the Deprecate annotation. But I have very little visibility on the plans for the configuration layer. Thanks, Martijn On

[VOTE] Release flink-connector-pulsar 4.1.0, release candidate #1

2023-12-13 Thread Leonard Xu
Hey all, Please review and vote on the release candidate #1 for the version 4.1.0 of the Apache Flink Pulsar Connector as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) The complete staging area is available for your review, which

Re: [DISCUSS] Should Configuration support getting value based on String key?

2023-12-13 Thread Rui Fan
Hi Martijn, Thanks for your reply! I noticed the 2.0 is doing some work related to clean configuration. But this discussion is different from other work. Most of other work are deprecate some Apis in Flink-1.19 and remove them in Flink-2.0. This discussion is a series of methods have been

Re: [DISCUSS] Should Configuration support getting value based on String key?

2023-12-13 Thread Martijn Visser
Hi Rui, Are you thinking about this as part of Flink 2.0, since that has the goal to introduce a completely clean configuration layer? [1] Best regards, Martijn [1] https://cwiki.apache.org/confluence/display/FLINK/2.0+Release On Wed, Dec 13, 2023 at 11:28 AM Maximilian Michels wrote: > > Hi

Re: [DISCUSS] Should Configuration support getting value based on String key?

2023-12-13 Thread Maximilian Michels
Hi Rui, +1 for removing the @Deprecated annotation from `getString(String key, String defaultValue)`. I would remove the other typed variants with default values but I'm ok with keeping them if they are still used. -Max On Wed, Dec 13, 2023 at 4:59 AM Rui Fan <1996fan...@gmail.com> wrote: > >

[jira] [Created] (FLINK-33816) SourceStreamTaskTest.testTriggeringStopWithSavepointWithDrain failed due async checkpoint triggering not being completed

2023-12-13 Thread Matthias Pohl (Jira)
Matthias Pohl created FLINK-33816: - Summary: SourceStreamTaskTest.testTriggeringStopWithSavepointWithDrain failed due async checkpoint triggering not being completed Key: FLINK-33816 URL:

[jira] [Created] (FLINK-33815) Add tests against jdk17 for pulsar connector

2023-12-13 Thread Sergey Nuyanzin (Jira)
Sergey Nuyanzin created FLINK-33815: --- Summary: Add tests against jdk17 for pulsar connector Key: FLINK-33815 URL: https://issues.apache.org/jira/browse/FLINK-33815 Project: Flink Issue

Re: [DISCUSS] Release Flink 1.18.1

2023-12-13 Thread Jing Ge
Hi Martijn, Thanks for the heads-up! I already started 1.18.1-rc1[1]. But please feel free to ping me. I will either redo rc1 or start rc2. Thanks! Best regards, Jing [1] https://dist.apache.org/repos/dist/dev/flink/flink-1.18.1-rc1/ On Tue, Dec 12, 2023 at 9:49 PM Martijn Visser wrote: > Hi

[jira] [Created] (FLINK-33814) Autoscaler Standalone control loop supports multiple thread

2023-12-13 Thread Rui Fan (Jira)
Rui Fan created FLINK-33814: --- Summary: Autoscaler Standalone control loop supports multiple thread Key: FLINK-33814 URL: https://issues.apache.org/jira/browse/FLINK-33814 Project: Flink Issue