Re: [DISCUSS] FLIP-309: Enable operators to trigger checkpoints dynamically

2023-05-24 Thread Dong Lin
Hi Jing, Thanks for your comments! Regarding the idea of using the existing "boundedness" attribute of sources, that is indeed something that we might find intuitive initially. I have thought about this idea, but could not find a good way to make it work. I will try to explain my thoughts and

Re: Questions on checkpointing mechanism for FLIP-27 Source API

2023-05-24 Thread Hongshun Wang
Hi Hong, The checkpoint is triggered by the timer executor of CheckpointCoordinator. It triggers the checkpoint in SourceCoordinator (which is passed to SplitEnumerator) and then in SourceOperator. The checkpoint event is put in SplitEnumerator's event loop to be executed. You can see the details

Re: Questions on checkpointing mechanism for FLIP-27 Source API

2023-05-24 Thread Mason Chen
Hi Hong, In addition, I guess the inconsistency could also be handled by the reader.start() method such that the operator event is re-sent during restore. Best, Mason On Tue, May 23, 2023 at 7:58 AM Piotr Nowojski wrote: > Hi, > > I vaguely remember someone implementing a mechanism to deal

[jira] [Created] (FLINK-32185) Remove M2_HOME usages

2023-05-24 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-32185: Summary: Remove M2_HOME usages Key: FLINK-32185 URL: https://issues.apache.org/jira/browse/FLINK-32185 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-32184) Use revision version property

2023-05-24 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-32184: Summary: Use revision version property Key: FLINK-32184 URL: https://issues.apache.org/jira/browse/FLINK-32184 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-32183) Use maven.multiModuleProjectDirectory property instead of rootDir plugin

2023-05-24 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-32183: Summary: Use maven.multiModuleProjectDirectory property instead of rootDir plugin Key: FLINK-32183 URL: https://issues.apache.org/jira/browse/FLINK-32183

[jira] [Created] (FLINK-32182) Use original japicmp plugin

2023-05-24 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-32182: Summary: Use original japicmp plugin Key: FLINK-32182 URL: https://issues.apache.org/jira/browse/FLINK-32182 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-32181) Drop support for Maven 3.2.5

2023-05-24 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-32181: Summary: Drop support for Maven 3.2.5 Key: FLINK-32181 URL: https://issues.apache.org/jira/browse/FLINK-32181 Project: Flink Issue Type: Technical

[jira] [Created] (FLINK-32180) Move error handling into MultipleComponentLeaderElectionDriverFactory

2023-05-24 Thread Matthias Pohl (Jira)
Matthias Pohl created FLINK-32180: - Summary: Move error handling into MultipleComponentLeaderElectionDriverFactory Key: FLINK-32180 URL: https://issues.apache.org/jira/browse/FLINK-32180 Project:

Re: [DISCUSS] FLIP-229: Prometheus Sink Connector

2023-05-24 Thread Lorenzo Nicora
Hi everybody I am working with Karthi on this connector. I think I can address some of the concerns raised by Danny et al. I do not have write access to the FLIP yet, but I will get it and do some edits. Also regarding the motivation, as suggested by Martijn. Q1) The fact we are using the

[jira] [Created] (FLINK-32179) Handle more repo names for automatic dist discovery

2023-05-24 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-32179: Summary: Handle more repo names for automatic dist discovery Key: FLINK-32179 URL: https://issues.apache.org/jira/browse/FLINK-32179 Project: Flink

[VOTE] Release flink-connector-jdbc v3.1.1, release candidate #1

2023-05-24 Thread Martijn Visser
Hi everyone, Please review and vote on the release candidate #1 for the version 3.1.1, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) The complete staging area is available for your review, which includes: * JIRA release notes [1], *

Re: Zero-Downtime Deployments with Flink Operator

2023-05-24 Thread Jing Ge
Hi Kevin, Thanks for reaching out. This question is not new and has been asked many times previously. I have the same feeling as Gyula. There are ways to provide zero downtime for Flink jobs, but commonly, I doubt if it is really necessary. If there might be some special use cases that really

Re: Zero-Downtime Deployments with Flink Operator

2023-05-24 Thread Kevin Lam
Thanks for your response Gyula! >5. This is possible to build on top of the current operator logic externally Do you have any recommendations on how best to approach this? A Custom Operator Plugin? On Wed, May 24, 2023 at 1:38 PM Gyula Fóra wrote: > Hey Kevin! > > I am not aware of

Re: Zero-Downtime Deployments with Flink Operator

2023-05-24 Thread Gyula Fóra
Hey Kevin! I am not aware of anyone currently working on this for the Flink Operator. Here are my current thoughts on the topic: 1. It's not impossible to build this into the operator but it would require some considerable changes to the logic, both in terms of resource mapping and

Zero-Downtime Deployments with Flink Operator

2023-05-24 Thread Kevin Lam
Hi, Is there any interest or ongoing work around supporting zero-downtime deployments with Flink using the Flink Operator? I saw that https://issues.apache.org/jira/browse/FLINK-24257 existed, but it looks a little stale. I'm interested in learning more about the current state of things. There

[jira] [Created] (FLINK-32178) AWS ALB INGRESS Support

2023-05-24 Thread Tan Kim (Jira)
Tan Kim created FLINK-32178: --- Summary: AWS ALB INGRESS Support Key: FLINK-32178 URL: https://issues.apache.org/jira/browse/FLINK-32178 Project: Flink Issue Type: Improvement Components:

Create new release of flink-connector-jdbc, v3.1.1

2023-05-24 Thread Martijn Visser
Hi all, While working on the upgrade of Flink Shaded [1], I encountered an issue with the Python test for the flink-connector-jdbc. It turns out that Python refers to v3.0.0 of that connector. Unfortunately, that connector version, as well as v3.1.0, use flink-shaded themselves. It specifically

[jira] [Created] (FLINK-32177) Refactor MultipleComponentLeaderElectionDriver.Listener.notifyAllKnownLeaderInformation(Collection)

2023-05-24 Thread Matthias Pohl (Jira)
Matthias Pohl created FLINK-32177: - Summary: Refactor MultipleComponentLeaderElectionDriver.Listener.notifyAllKnownLeaderInformation(Collection) Key: FLINK-32177 URL:

Re: [RESULT][VOTE] Release 1.16.2, release candidate #1

2023-05-24 Thread Jing Ge
Thanks Weijie for your effort! Looking forward to it! Best regards, Jing On Wed, May 24, 2023 at 11:08 AM weijie guo wrote: > I'm happy to announce that we have unanimously approved this release. > > > > There are 8 approving votes, 3 of which are binding: > > > * Yuxin Tan > > * Yun Tang > >

Re: [DISCUSS] FLIP-313 Add support of User Defined AsyncTableFunction

2023-05-24 Thread Aitozi
May I ask for some feedback :D Thanks, Aitozi Aitozi 于2023年5月23日周二 19:14写道: > > Just catch an user case report from Giannis Polyzos for this usage: > > https://lists.apache.org/thread/qljwd40v5ntz6733cwcdr8s4z97b343b > > Aitozi 于2023年5月23日周二 17:45写道: > > > > Hi guys, > > I want to bring

[jira] [Created] (FLINK-32176) [CVE-2022-1471] Upgrade snake yaml to 2.0

2023-05-24 Thread Samrat Deb (Jira)
Samrat Deb created FLINK-32176: -- Summary: [CVE-2022-1471] Upgrade snake yaml to 2.0 Key: FLINK-32176 URL: https://issues.apache.org/jira/browse/FLINK-32176 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-32175) translate doc(datagen.md) to chinese

2023-05-24 Thread wangliang (Jira)
wangliang created FLINK-32175: - Summary: translate doc(datagen.md) to chinese Key: FLINK-32175 URL: https://issues.apache.org/jira/browse/FLINK-32175 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-32174) Update Cloudera product and link in doc page

2023-05-24 Thread Ferenc Csaky (Jira)
Ferenc Csaky created FLINK-32174: Summary: Update Cloudera product and link in doc page Key: FLINK-32174 URL: https://issues.apache.org/jira/browse/FLINK-32174 Project: Flink Issue Type:

Re: How to figure out what's the size of ListState?

2023-05-24 Thread Yun Tang
Hi Amir, For the current Flink, you have to iterator the returned Iterable of ListState#get(). Why Flink lacks an API to get the size of listState directly? This is because Flink leverages RocksDB's merge operator [1] for high-performance listState#add [2], that is to say, we would not even

Re: [VOTE] Release 1.16.2, release candidate #1

2023-05-24 Thread weijie guo
Thanks all, This vote is now closed. I will announce the results in a separate thread. Best regards, Weijie Hang Ruan 于2023年5月23日周二 16:53写道: > +1(non-binding) > > - reviewed Jira release notes > - built from source > - verified signatures > - verified hashes > - verified that no binary

[RESULT][VOTE] Release 1.16.2, release candidate #1

2023-05-24 Thread weijie guo
I'm happy to announce that we have unanimously approved this release. There are 8 approving votes, 3 of which are binding: * Yuxin Tan * Yun Tang * Jing Ge * Xintong Song(binding) * Xingbo Huang(binding) * Qingsheng Ren(binding) * Samrat Deb * Hang Ruan There are no disapproving

Re: [VOTE] Release flink-connector-cassandra 3.1.0, release candidate #2

2023-05-24 Thread Danny Cranmer
Bumping up the list. Can the PMC please take a look? This is the final connector without support for Flink 1.17. Thanks, Danny On Thu, May 18, 2023 at 2:42 PM Danny Cranmer wrote: > +1 (binding) > > - Release notes look good > - Verified signatures and hashes > - Verified there are no binaries