[jira] [Created] (FLINK-35648) Pipeline job doesn't support multiple routing

2024-06-18 Thread yux (Jira)
yux created FLINK-35648: --- Summary: Pipeline job doesn't support multiple routing Key: FLINK-35648 URL: https://issues.apache.org/jira/browse/FLINK-35648 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-35647) Pipeline route rule supports placeholder replacement

2024-06-18 Thread yux (Jira)
yux created FLINK-35647: --- Summary: Pipeline route rule supports placeholder replacement Key: FLINK-35647 URL: https://issues.apache.org/jira/browse/FLINK-35647 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-35646) Add gateway refresh rest api doc

2024-06-18 Thread dalongliu (Jira)
dalongliu created FLINK-35646: - Summary: Add gateway refresh rest api doc Key: FLINK-35646 URL: https://issues.apache.org/jira/browse/FLINK-35646 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-35645) Add materialized table quickstart doc

2024-06-18 Thread dalongliu (Jira)
dalongliu created FLINK-35645: - Summary: Add materialized table quickstart doc Key: FLINK-35645 URL: https://issues.apache.org/jira/browse/FLINK-35645 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-35643) Add materialized table statement doc

2024-06-18 Thread dalongliu (Jira)
dalongliu created FLINK-35643: - Summary: Add materialized table statement doc Key: FLINK-35643 URL: https://issues.apache.org/jira/browse/FLINK-35643 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-35644) Add workflow scheduler doc

2024-06-18 Thread dalongliu (Jira)
dalongliu created FLINK-35644: - Summary: Add workflow scheduler doc Key: FLINK-35644 URL: https://issues.apache.org/jira/browse/FLINK-35644 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-35642) parsing the configuration file in flink, when configuring the s3 key, if there is contain # char, the correct key will be split

2024-06-18 Thread blackpighe (Jira)
blackpighe created FLINK-35642: -- Summary: parsing the configuration file in flink, when configuring the s3 key, if there is contain # char, the correct key will be split Key: FLINK-35642 URL:

Re: [ANNOUNCE] Apache Flink CDC 3.1.1 released

2024-06-18 Thread gongzhongqiang
Well done! Thanks Qingsheng for the release work and all contributors involved. Best, Zhongqiang Gong Qingsheng Ren 于2024年6月18日周二 23:51写道: > The Apache Flink community is very happy to announce the release of Apache > Flink CDC 3.1.1. > > Apache Flink CDC is a distributed data integration tool

Re: [VOTE] FLIP-462: Support Custom Data Distribution for Input Stream of Lookup Join

2024-06-18 Thread Jingsong Li
+1 binding On Tue, Jun 18, 2024 at 11:54 AM Feng Jin wrote: > > +1 (non-binding) > > > Best, > Feng Jin > > > On Tue, Jun 18, 2024 at 10:24 AM Lincoln Lee wrote: > > > +1 (binding) > > > > > > Best, > > Lincoln Lee > > > > > > Xintong Song 于2024年6月17日周一 13:39写道: > > > > > +1 (binding) > > > >

Re: [ANNOUNCE] Apache Flink CDC 3.1.1 released

2024-06-18 Thread Paul Lam
Well done! Thanks a lot for your hard work! Best, Paul Lam > 2024年6月19日 09:47,Leonard Xu 写道: > > Congratulations! Thanks Qingsheng for the release work and all contributors > involved. > > Best, > Leonard > >> 2024年6月18日 下午11:50,Qingsheng Ren 写道: >> >> The Apache Flink community is very

Re: [ANNOUNCE] Apache Flink 1.19.1 released

2024-06-18 Thread Rui Fan
Thanks Hong for driving this release! I clicked the library/flink docker images link[1] from 1.19.1-release-announcement[2]. And I cannot find the 1.19.1 images there, I'm not sure if it's caused by delay or some steps are missed. [1] https://hub.docker.com/_/flink/tags?page=1=1.19.1 [2]

Re: [ANNOUNCE] Apache Flink 1.19.1 released

2024-06-18 Thread Leonard Xu
Congratulations! Thanks Hong for the release work and all involved! Best, Leonard > 2024年6月19日 上午4:20,Hong Liang 写道: > > The Apache Flink community is very happy to announce the release of Apache > Flink 1.19.1, which is the first bugfix release for the Apache Flink 1.19 > series. > > Apache

Re: [ANNOUNCE] Apache Flink CDC 3.1.1 released

2024-06-18 Thread Leonard Xu
Congratulations! Thanks Qingsheng for the release work and all contributors involved. Best, Leonard > 2024年6月18日 下午11:50,Qingsheng Ren 写道: > > The Apache Flink community is very happy to announce the release of Apache > Flink CDC 3.1.1. > > Apache Flink CDC is a distributed data integration

[jira] [Created] (FLINK-35641) ParquetSchemaConverter should correctly handle field optionality

2024-06-18 Thread Alex Sorokoumov (Jira)
Alex Sorokoumov created FLINK-35641: --- Summary: ParquetSchemaConverter should correctly handle field optionality Key: FLINK-35641 URL: https://issues.apache.org/jira/browse/FLINK-35641 Project:

[ANNOUNCE] Apache Flink 1.19.1 released

2024-06-18 Thread Hong Liang
The Apache Flink community is very happy to announce the release of Apache Flink 1.19.1, which is the first bugfix release for the Apache Flink 1.19 series. Apache Flink® is an open-source stream processing framework for distributed, high-performing, always-available, and accurate data streaming

Re: [VOTE] FLIP-461: Synchronize rescaling with checkpoint creation to minimize reprocessing for the AdaptiveScheduler

2024-06-18 Thread Thomas Weise
+1 (binding) On Tue, Jun 18, 2024 at 11:38 AM Gabor Somogyi wrote: > +1 (binding) > > G > > > On Mon, Jun 17, 2024 at 10:24 AM Matthias Pohl wrote: > > > Hi everyone, > > the discussion in [1] about FLIP-461 [2] is kind of concluded. I am > > starting a vote on this one here. > > > > The vote

[jira] [Created] (FLINK-35640) Drop Flink 1.15 support for the operator

2024-06-18 Thread Mate Czagany (Jira)
Mate Czagany created FLINK-35640: Summary: Drop Flink 1.15 support for the operator Key: FLINK-35640 URL: https://issues.apache.org/jira/browse/FLINK-35640 Project: Flink Issue Type:

Re: [DISCUSS] FLIP-463: Schema Definition in CREATE TABLE AS Statement

2024-06-18 Thread Sergio Pena
Hi Jeyhun, Timo, I updated the FLIP to include the option to reorder the columns defined in the SELECT part. As Timo mentioned, this is equivalent to the Flink INSERT INTO statement and also to Azure CTAS statement. For example: > CREATE TABLE t1(a INT, b INT, c INT); > CREATE TABLE t1(c, b, a)

Re: [VOTE] Apache Flink Kubernetes Operator Release 1.9.0, release candidate #1

2024-06-18 Thread Mate Czagany
Hi, +1 (non-binding) Note: Using the Apache Flink KEYS file [1] to verify the signatures your key seems to be expired, so that file should be updated as well. - Verified checksums and signatures - Built source distribution - Verified all pom.xml versions are the same - Verified install from RC

Re: [ANNOUNCE] Flink 1.20 feature freeze

2024-06-18 Thread Ferenc Csaky
Hi Robert, Rui, Ufuk, and Weijie, I would like to raise the PR about merging `flink run` and `flink run-application` functionality [1] to get considered as part of the 1.20 release. The reason IMO is that the `run-application` CLI command should be removed in the same release when Per-Job mode

[ANNOUNCE] Apache Flink CDC 3.1.1 released

2024-06-18 Thread Qingsheng Ren
The Apache Flink community is very happy to announce the release of Apache Flink CDC 3.1.1. Apache Flink CDC is a distributed data integration tool for real time data and batch data, bringing the simplicity and elegance of data integration via YAML to describe the data movement and transformation

Re: [VOTE] FLIP-461: Synchronize rescaling with checkpoint creation to minimize reprocessing for the AdaptiveScheduler

2024-06-18 Thread Gabor Somogyi
+1 (binding) G On Mon, Jun 17, 2024 at 10:24 AM Matthias Pohl wrote: > Hi everyone, > the discussion in [1] about FLIP-461 [2] is kind of concluded. I am > starting a vote on this one here. > > The vote will be open for at least 72 hours (i.e. until June 20, 2024; > 8:30am UTC) unless there

[RESULT][VOTE] Apache Flink CDC Release 3.1.1, release candidate #0

2024-06-18 Thread Qingsheng Ren
Hi everyone, I'm happy to announce that we have unanimously approved this release. There are 5 approving votes, 3 of which are binding: - Xiqian Yu (non-binding) - Yanquan Lv (non-binding) - Leonard Xu (binding) - Jark Wu (binding) - Lincoln Lee (binding) There are no disapproving votes.

[jira] [Created] (FLINK-35639) upgrading to 1.19 with job in HA state with restart strategy crashes job manager

2024-06-18 Thread yazgoo (Jira)
yazgoo created FLINK-35639: -- Summary: upgrading to 1.19 with job in HA state with restart strategy crashes job manager Key: FLINK-35639 URL: https://issues.apache.org/jira/browse/FLINK-35639 Project: Flink

Re: [ANNOUNCE] New Apache Flink Committer - Zhongqiang Gong

2024-06-18 Thread Swapnal Varma
Congratulations Zhongqiang! Best, Swapnal On Tue, 18 Jun 2024, 07:05 Jacky Lau, wrote: > Congratulations Zhongqiang! > > Best Regards > Jacky Lau > > Yubin Li 于2024年6月18日周二 00:28写道: > > > Congratulations Zhongqiang! > > > > Best Regards > > Yubin > > > > On Tue, Jun 18, 2024 at 12:21 AM

[jira] [Created] (FLINK-35638) Modify OceanBase Docker container to make the test cases runnable on non-Linux systems

2024-06-18 Thread He Wang (Jira)
He Wang created FLINK-35638: --- Summary: Modify OceanBase Docker container to make the test cases runnable on non-Linux systems Key: FLINK-35638 URL: https://issues.apache.org/jira/browse/FLINK-35638

Re: [VOTE] Apache Flink CDC Release 3.1.1, release candidate #0

2024-06-18 Thread Lincoln Lee
+1 (binding) - verified hashes - verified signatures - build from source with JDK 8 & Maven 3.8.6 - the source distributions do not contain any binaries - all POM files point to the same version - checked release notes - reviewed the web PR Best, Lincoln Lee Jark Wu 于2024年6月18日周二 19:58写道: >

Re: [DISCUSS] FLIP-463: Schema Definition in CREATE TABLE AS Statement

2024-06-18 Thread Sergio Pena
Hi Ron, I think the primary key and unique values are handled by the engine. I found this document [1] that explains how Flink handles those records. When using CTAS, you can specify in the table options how you want the records to be merged if you define a primary key. Regarding CTAS, this is

[jira] [Created] (FLINK-35637) ScalarFunctionCallGen does not handle complex argument type properly

2024-06-18 Thread Qinghui Xu (Jira)
Qinghui Xu created FLINK-35637: -- Summary: ScalarFunctionCallGen does not handle complex argument type properly Key: FLINK-35637 URL: https://issues.apache.org/jira/browse/FLINK-35637 Project: Flink

Re: [VOTE] Apache Flink CDC Release 3.1.1, release candidate #0

2024-06-18 Thread Jark Wu
+1 (binding) - Build and compile the source code locally: *OK* - Verified signatures: *OK* - Verified hashes: *OK* - Checked no missing artifacts in the staging area: *OK* - Reviewed the website release PR: *OK* - Checked the licenses: *OK* Best, Jark On Tue, 18 Jun 2024 at 18:14, Leonard Xu

[jira] [Created] (FLINK-35636) Streaming File Sink s3 end-to-end test did not finish after 900 seconds

2024-06-18 Thread Weijie Guo (Jira)
Weijie Guo created FLINK-35636: -- Summary: Streaming File Sink s3 end-to-end test did not finish after 900 seconds Key: FLINK-35636 URL: https://issues.apache.org/jira/browse/FLINK-35636 Project: Flink

[jira] [Created] (FLINK-35635) Release Testing: Verify FLIP-445: Support dynamic parallelism inference for HiveSource

2024-06-18 Thread xingbe (Jira)
xingbe created FLINK-35635: -- Summary: Release Testing: Verify FLIP-445: Support dynamic parallelism inference for HiveSource Key: FLINK-35635 URL: https://issues.apache.org/jira/browse/FLINK-35635 Project:

[jira] [Created] (FLINK-35634) Add a CDC quickstart utility

2024-06-18 Thread yux (Jira)
yux created FLINK-35634: --- Summary: Add a CDC quickstart utility Key: FLINK-35634 URL: https://issues.apache.org/jira/browse/FLINK-35634 Project: Flink Issue Type: New Feature Components:

[jira] [Created] (FLINK-35633) Early verification and clearer error message of pipeline YAML definition

2024-06-18 Thread yux (Jira)
yux created FLINK-35633: --- Summary: Early verification and clearer error message of pipeline YAML definition Key: FLINK-35633 URL: https://issues.apache.org/jira/browse/FLINK-35633 Project: Flink

[SUMMARY] Flink 1.20 Release Sync 06/18/2024

2024-06-18 Thread weijie guo
Dear devs, This is the first meeting after feature freeze of Flink 1.20. I'd like to share the information synced in the meeting. - Feature Freeze We have announced the feature freeze of 1.20 at June 15. Finally, we have 13 completed features. Meanwhile, 3 features were moved to the

Re: [VOTE] Apache Flink CDC Release 3.1.1, release candidate #0

2024-06-18 Thread Leonard Xu
+1 (binding) - verified signatures - verified hashsums - checked release notes - reviewed the web PR - tested Flink CDC works with Flink 1.19 - tested route、transform in MySQL to Doris Pipeline Best, Leonard

[jira] [Created] (FLINK-35632) The example provided in the kafkaSource documentation for topic regex subscription is incorrect

2024-06-18 Thread elon_X (Jira)
elon_X created FLINK-35632: -- Summary: The example provided in the kafkaSource documentation for topic regex subscription is incorrect Key: FLINK-35632 URL: https://issues.apache.org/jira/browse/FLINK-35632

Re: [ANNOUNCE] Flink 1.20 feature freeze

2024-06-18 Thread weijie guo
Hi Zakelly, Thank you for informing us! After discussion, all RMs agreed that this was an important fix that should be merged into 1.20. So feel free to merge it. Best regards, Weijie Zakelly Lan 于2024年6月15日周六 16:29写道: > Hi Robert, Rui, Ufuk and Weijie, > > Thanks for the update! > > FYI:

[jira] [Created] (FLINK-35631) KafkaSource parameter partition.discovery.interval.ms with a default value of 5 minutes does not take effect

2024-06-18 Thread elon_X (Jira)
elon_X created FLINK-35631: -- Summary: KafkaSource parameter partition.discovery.interval.ms with a default value of 5 minutes does not take effect Key: FLINK-35631 URL: https://issues.apache.org/jira/browse/FLINK-35631

Re: [DISCUSS] FLIP-444: Native file copy support

2024-06-18 Thread Keith Lee
Hi Lorenzo, > As I mentioned above, in the end I think hardcoding in Flink support for the `cpulimit` doesn't seem appropriate and at the same time it is not actually necessary. I agree. What's suggested is similar to the way where Flink currently allow users to limit resource usage on s3a

Re: [DISCUSS] FLIP-444: Native file copy support

2024-06-18 Thread Keith Lee
I meant to address Piotr on the above, apologies! Best regards Keith Lee On Tue, Jun 18, 2024 at 8:06 AM Keith Lee wrote: > Hi Lorenzo, > > > As I mentioned above, in the end I think hardcoding in Flink support for > the `cpulimit` doesn't seem appropriate and at the same time it is not >

[jira] [Created] (FLINK-35630) Kafka source may reset the consume offset to earliest when the partition leader changes

2024-06-18 Thread tanjialiang (Jira)
tanjialiang created FLINK-35630: --- Summary: Kafka source may reset the consume offset to earliest when the partition leader changes Key: FLINK-35630 URL: https://issues.apache.org/jira/browse/FLINK-35630

[jira] [Created] (FLINK-35629) Performance regression in stringRead and stringWrite

2024-06-18 Thread Rui Fan (Jira)
Rui Fan created FLINK-35629: --- Summary: Performance regression in stringRead and stringWrite Key: FLINK-35629 URL: https://issues.apache.org/jira/browse/FLINK-35629 Project: Flink Issue Type: Bug

[VOTE] Apache Flink Kubernetes Operator Release 1.9.0, release candidate #1

2024-06-18 Thread Gyula Fóra
Hi Everyone, Please review and vote on the release candidate #1 for the version 1.9.0 of Apache Flink Kubernetes Operator, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) **Release Overview** As an overview, the release consists of

[jira] [Created] (FLINK-35628) Translate “DataGen Connector" of "DataStream Connectors" into Chinese

2024-06-18 Thread Hao Yu (Jira)
Hao Yu created FLINK-35628: -- Summary: Translate “DataGen Connector" of "DataStream Connectors" into Chinese Key: FLINK-35628 URL: https://issues.apache.org/jira/browse/FLINK-35628 Project: Flink