Re: [DISCUSS] The performance of serializerHeavyString starts regress since April 3

2024-05-22 Thread Sam Barker
It looks like the most recent run of JDK 11 saw a big improvement[1] of the performance of the test. That improvement seems related to [2] which is a fix for FLINK-35215 [3]. That suggests to me that the test isn't as isolated to the performance of the code its trying to test as would be ideal.

[DISCUSS] Flink CDC Upgrade Debezium version to 2.x

2024-05-22 Thread gongzhongqiang
Hi all, I would like to start a discussion about upgrading Debezium to version 2.x. Background: Currently, the Debezium community no longer maintains versions prior to 2.0, and the website has taken down the documentation for versions before 2.0. However, Flink CDC depends on Debezium version

[jira] [Created] (FLINK-35427) Support fail-on-unknown-field config in json format

2024-05-22 Thread huhuan (Jira)
huhuan created FLINK-35427: -- Summary: Support fail-on-unknown-field config in json format Key: FLINK-35427 URL: https://issues.apache.org/jira/browse/FLINK-35427 Project: Flink Issue Type:

[jira] [Created] (FLINK-35426) Change the distribution of DynamicFilteringDataCollector to Broadcast

2024-05-22 Thread xingbe (Jira)
xingbe created FLINK-35426: -- Summary: Change the distribution of DynamicFilteringDataCollector to Broadcast Key: FLINK-35426 URL: https://issues.apache.org/jira/browse/FLINK-35426 Project: Flink

[jira] [Created] (FLINK-35425) Support convert Freshness to cron expression for full refresh mode

2024-05-22 Thread dalongliu (Jira)
dalongliu created FLINK-35425: - Summary: Support convert Freshness to cron expression for full refresh mode Key: FLINK-35425 URL: https://issues.apache.org/jira/browse/FLINK-35425 Project: Flink

[jira] [Created] (FLINK-35424) Elasticsearch connector 8 supports SSL context

2024-05-22 Thread Mingliang Liu (Jira)
Mingliang Liu created FLINK-35424: - Summary: Elasticsearch connector 8 supports SSL context Key: FLINK-35424 URL: https://issues.apache.org/jira/browse/FLINK-35424 Project: Flink Issue Type:

[RESULT][VOTE] FLIP-449: Reorganization of flink-connector-jdbc

2024-05-22 Thread Joao Boto
Hi, I am happy to say that FLIP-449: Reorganization of flink-connector-jdbc [1] has been accepted. The proposal has been accepted with 7 approving votes (3 binding) and there are no disapproval (voted on this thread [2]): Ahmed Hamdy (non-binding) Aleksandr Pilipenko (non-binding) Jeyhun

[RESULT][VOTE] FLIP-451: Introduce timeout configuration to AsyncSink

2024-05-22 Thread Ahmed Hamdy
Hi all I'm happy to announce that we have unanimously approved FLIP-451[1] There were 9 votes, of which 4 were binding on the vote thread[2]. - Muhammet Orazov (non-binding) - Jeyhun Karimov (non-binding) - Lorenzo Affetti (non-binding) - Aleksandr Pilipenko (non-binding) - David Radley

Re: [VOTE] FLIP-451: Introduce timeout configuration to AsyncSink

2024-05-22 Thread Ahmed Hamdy
Hi all, Thanks for all your votes, I hereby close the vote and I'll announce the results in a separate email. Best Regards Ahmed Hamdy On Wed, 22 May 2024 at 11:30, weijie guo wrote: > Thanks Ahmed for introducing this FLIP, nice improvement. > > +1(binding) > > Best regards, > > Weijie > >

[jira] [Created] (FLINK-35423) ARRAY_EXCEPT should support set semantics

2024-05-22 Thread Sergey Nuyanzin (Jira)
Sergey Nuyanzin created FLINK-35423: --- Summary: ARRAY_EXCEPT should support set semantics Key: FLINK-35423 URL: https://issues.apache.org/jira/browse/FLINK-35423 Project: Flink Issue Type:

Re: [DISCUSS] FLIP-XXX Apicurio-avro format

2024-05-22 Thread David Radley
Hi Danny, Did you have a chance you have a look at my responses to your feedback? I am hoping to keep the momentum going on this one, kind regards, David. From: David Radley Date: Tuesday, 14 May 2024 at 17:21 To: dev@flink.apache.org Subject: [EXTERNAL] [DISCUSS] FLIP-XXX Apicurio-avro

[jira] [Created] (FLINK-35422) Flink dynamically adjusts operator-level resource usage without stopping the job

2024-05-22 Thread Jira
尹博星 created FLINK-35422: --- Summary: Flink dynamically adjusts operator-level resource usage without stopping the job Key: FLINK-35422 URL: https://issues.apache.org/jira/browse/FLINK-35422 Project: Flink

[jira] [Created] (FLINK-35421) Schema Operator blocking forever when Akka Rpc timeout

2024-05-22 Thread hk__lrzy (Jira)
hk__lrzy created FLINK-35421: Summary: Schema Operator blocking forever when Akka Rpc timeout Key: FLINK-35421 URL: https://issues.apache.org/jira/browse/FLINK-35421 Project: Flink Issue Type:

Re: [DISCUSSION] FLIP-457: Improve Table/SQL Configuration for Flink 2.0

2024-05-22 Thread Leonard Xu
Thanks Jane for the refine work, +1 from my side. I adjusted the table format of FLIP so that it can display all content in one page. Best, Leonard > 2024年5月22日 下午3:42,Jane Chan 写道: > > Hi Lincoln, > > Thanks for your suggestion. I've reviewed the comments from the previous PR > review[1],

[jira] [Created] (FLINK-35420) WordCountMapredITCase fails to compile in IntelliJ

2024-05-22 Thread Piotr Nowojski (Jira)
Piotr Nowojski created FLINK-35420: -- Summary: WordCountMapredITCase fails to compile in IntelliJ Key: FLINK-35420 URL: https://issues.apache.org/jira/browse/FLINK-35420 Project: Flink Issue

Re: [DISCUSS] Add a JDBC Sink Plugin to Flink-CDC-Pipeline

2024-05-22 Thread Yanquan Lv
Thanks jerry for driving this, JDBC sink for CDC pipeline is indeed a high demand in the community. I have one concern: Some databases that use jdbc, such as mysql, may be time-consuming to perform table structure changes, but FlinkCDC will not send DataChangeEvent during this period, which can

Re: [VOTE] Release flink-connector-opensearch v1.2.0, release candidate #1

2024-05-22 Thread Leonard Xu
+1 (binding) - verified signatures - verified hashsums - built from source code with JDK 1.8 succeeded - checked Github release tag - checked release notes - reviewed the web PR Best, Leonard > 2024年5月16日 上午6:58,Andrey Redko 写道: > > +1 (non-binding), thanks Sergey! > > On Wed, May 15, 2024,

Re: [VOTE] Release flink-connector-opensearch v2.0.0, release candidate #1

2024-05-22 Thread Leonard Xu
> +1 (binding) > > - verified signatures > - verified hashsums > - built from source code with JDK 1.8 succeeded > - checked Github release tag > - checked release notes > - reviewed the web PR Supply more information about build from source code with JDK 1.8 > - built from source code with

[jira] [Created] (FLINK-35419) scan.bounded.latest-offset makes queries never finish if the latest message is a EndTxn Kafka marker

2024-05-22 Thread Fabian Paul (Jira)
Fabian Paul created FLINK-35419: --- Summary: scan.bounded.latest-offset makes queries never finish if the latest message is a EndTxn Kafka marker Key: FLINK-35419 URL:

[jira] [Created] (FLINK-35418) EventTimeWindowCheckpointingITCase fails with an NPE

2024-05-22 Thread Ryan Skraba (Jira)
Ryan Skraba created FLINK-35418: --- Summary: EventTimeWindowCheckpointingITCase fails with an NPE Key: FLINK-35418 URL: https://issues.apache.org/jira/browse/FLINK-35418 Project: Flink Issue

Re: [VOTE] Release flink-connector-opensearch v2.0.0, release candidate #1

2024-05-22 Thread Leonard Xu
+1 (binding) - verified signatures - verified hashsums - built from source code with JDK 1.8 succeeded - checked Github release tag - checked release notes - reviewed the web PR Best, Leonard > 2024年5月16日 上午6:58,Andrey Redko 写道: > > +1 (non-binding), thanks Sergey! > > On Wed, May 15, 2024,

Re: [VOTE] Release flink-connector-mongodb v1.2.0, release candidate #2

2024-05-22 Thread Leonard Xu
+1 (binding) - verified signatures - verified hashsums - built from source code with java 1.8 succeeded - checked Github release tag - checked release notes - reviewed the web PR Best, Leonard > 2024年5月22日 下午4:16,Qingsheng Ren 写道: > > +1 (binding) > > - Verified checksum and signature > -

Re: [VOTE] Release flink-connector-kafka v3.2.0, release candidate #1

2024-05-22 Thread Leonard Xu
+1 (binding) - verified signatures - verified hashsums - built from source code with java 1.8 succeeded - checked Github release tag - reviewed the web PR - checked the CI result, minor: the link [7] you post should be [1] - checked release notes, minor: the issue

Re: [VOTE] Release flink-connector-jdbc v3.2.0, release candidate #2

2024-05-22 Thread Leonard Xu
+1 (binding) - verified signatures - verified hashsums - built from source code with java 1.8 succeeded - checked Github release tag - checked release notes - reviewed the web PR Best, Leonard > 2024年4月21日 下午9:42,Hang Ruan 写道: > > +1 (non-binding) > > - Validated checksum hash > - Verified

Re: [VOTE] Release flink-connector-gcp-pubsub v3.1.0, release candidate #1

2024-05-22 Thread Leonard Xu
+1 (binding) - verified signatures - verified hashsums - built from source code with java 1.8 succeeded - checked Github release tag - checked release notes - reviewed the web PR Best, Leonard > 2024年4月21日 下午9:52,Hang Ruan 写道: > > +1 (non-binding) > > - Validated checksum hash > -

Re: [VOTE] Release flink-connector-cassandra v3.2.0, release candidate #1

2024-05-22 Thread Leonard Xu
+1 (binding) - verified signatures - verified hashsums - built from source code with java 1.8 succeeded - checked Github release tag - checked release notes status which only left one issue is used for release tracking - reviewed the web PR Best, Leonard > 2024年5月22日 下午6:10,weijie guo 写道: >

Re: [VOTE] Release flink-connector-aws v4.3.0, release candidate #2

2024-05-22 Thread Leonard Xu
+1 (binding) - verified signatures - verified hashsums - built from source code with java 1.8 succeeded - checked Github release tag - checked release notes - reviewed the web PR Best, Leonard > 2024年4月28日 下午11:56,Aleksandr Pilipenko 写道: > > +1 (non-binding) > > - Verified checksums > -

Re: [VOTE] FLIP-451: Introduce timeout configuration to AsyncSink

2024-05-22 Thread weijie guo
Thanks Ahmed for introducing this FLIP, nice improvement. +1(binding) Best regards, Weijie Leonard Xu 于2024年5月22日周三 17:25写道: > > After discuss with Ahmed, the updated FLIP looks good to me. > > +1(binding) > > > Best, > Leonard > > > 2024年5月21日 下午6:12,Hong Liang 写道: > > > > +1 (binding) >

Re: [VOTE] Release flink-connector-cassandra v3.2.0, release candidate #1

2024-05-22 Thread weijie guo
+1(non-binding) -Validated checksum hash -Verified signature -Build from source Best regards, Weijie Hang Ruan 于2024年5月22日周三 10:12写道: > +1 (non-binding) > > - Validated checksum hash > - Verified signature > - Verified that no binaries exist in the source archive > - Build the source with

Re: [VOTE] FLIP-451: Introduce timeout configuration to AsyncSink

2024-05-22 Thread Leonard Xu
After discuss with Ahmed, the updated FLIP looks good to me. +1(binding) Best, Leonard > 2024年5月21日 下午6:12,Hong Liang 写道: > > +1 (binding) > > Thanks Ahmed > > On Tue, May 14, 2024 at 11:51 AM David Radley > wrote: > >> Thanks for the clarification Ahmed >> >> +1 (non-binding) >> >>

Re: [DISCUSS] FLIP-451: Refactor Async sink API

2024-05-22 Thread Leonard Xu
Thanks Ahmed for the update, the FLIP looks good to me now. Best, Leonard > 2024年5月22日 下午4:34,Ahmed Hamdy 写道: > >> >> (1) Implicitly point a public API change is not enough, Could you add a >> section Public Interfaces to enumerate all Public APIs that you proposed >> and you changed? >> It’s

Re: [DISCUSS] FLIP-443: Interruptible watermark processing

2024-05-22 Thread Piotr Nowojski
Hi Zakelly, > I suggest not doubling the existing methods. Only providing the following one is enough In that case I would prefer to have a complete set of the methods for the sake of completeness. If the number of variants is/would be getting too much, we could convert the class into a builder?

Re: [DISCUSS] Proposing an LTS Release for the 1.x Line

2024-05-22 Thread Xintong Song
Thanks, Alex. I see one task that needs to be done once the FLIP is approved, which I'd suggest to also mention in the: To explain the LTS policy to users on website / documentation (because FLIP is developer-facing) before / upon releasing 1.20. Other than that, the FLIP LGTM. Best, Xintong

Re: [DISCUSS] FLIP-451: Refactor Async sink API

2024-05-22 Thread Ahmed Hamdy
> > (1) Implicitly point a public API change is not enough, Could you add a > section Public Interfaces to enumerate all Public APIs that you proposed > and you changed? > It’s a standard part of a FLIP template[1]. > yes this is updated in the FLIP now. > (2) About the proposed public

Re: [VOTE] Release flink-connector-mongodb v1.2.0, release candidate #2

2024-05-22 Thread Qingsheng Ren
+1 (binding) - Verified checksum and signature - Built from source with Java 8 - Verified source release contains no binaries - Verified tag exists on GitHub - Verified JARs on Maven repo is built by Java 8 - Reviewed web PR Thanks for the awesome work, Danny! Best, Qingsheng On Mon, May 20,

Re: [DISCUSSION] FLIP-457: Improve Table/SQL Configuration for Flink 2.0

2024-05-22 Thread Jane Chan
Hi Lincoln, Thanks for your suggestion. I've reviewed the comments from the previous PR review[1], and the agreement at the time was that any configuration options not included in ExecutionConfigOptions and OptimizerConfigOptions should have the Experimental annotation explicitly added. Since

Re: [DISCUSS] The performance of serializerHeavyString starts regress since April 3

2024-05-22 Thread weijie guo
Thanks Piotr and Zakelly for your advice. I have also tested this in my local machine, and I can not reproduce it even if switch to JDK11. This must have something to do with the benchmark's testing environment. > I will update FLINK-35040[1] to "Won't fix" and update the Priority from Blocker

Re: [SUMMARY] Flink 1.20 Release Sync 05/21/2024

2024-05-22 Thread Rui Fan
Thanks Weijie for the summary! Best, Rui On Wed, May 22, 2024 at 2:52 PM weijie guo wrote: > Dear devs, > > > This is the fourth meeting for Flink 1.20 release cycle. > > > I'd like to share the information synced in the meeting. > > > > - Feature Freeze > > > It is worth noting that there

Re: [DISCUSS] The performance of serializerHeavyString starts regress since April 3

2024-05-22 Thread Rui Fan
Hi Piotr and Zakelly, Thanks a lot for providing these historical JIRAs related to serializerHeavyString performance. It seems this similar issue has happened multiple times. And thanks for clarifying the principle of benchmark. I have encountered the semi-stable state for other benchmarks as

[SUMMARY] Flink 1.20 Release Sync 05/21/2024

2024-05-22 Thread weijie guo
Dear devs, This is the fourth meeting for Flink 1.20 release cycle. I'd like to share the information synced in the meeting. - Feature Freeze It is worth noting that there are only 4 weeks left until the feature freeze time(June 15, 2024), and developers need to pay attention to the

Re: [DISCUSS] FLIP-443: Interruptible watermark processing

2024-05-22 Thread Zakelly Lan
Hi Piotrek, `MailOptions` looks good to me. I suggest not doubling the existing methods. Only providing the following one is enough: void execute( > MailOptions mailOptions, > ThrowingRunnable command, > String descriptionFormat, > Object... descriptionArgs); WDYT? Best,