[jira] [Created] (FLINK-33475) yarn parameters cannot be set using flink.yarn.
zhengzhili created FLINK-33475: -- Summary: yarn parameters cannot be set using flink.yarn. Key: FLINK-33475 URL: https://issues.apache.org/jira/browse/FLINK-33475 Project: Flink Issue Type: Bug Components: Command Line Client Reporter: zhengzhili Attachments: image-2023-11-07-15-22-46-793.png yarn parameters cannot be set using flink.yarn. !image-2023-11-07-15-22-46-793.png|width=1104,height=225! -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [VOTE] FLIP-376: Add DISTRIBUTED BY clause
+1 Best, Dawid On Mon, 6 Nov 2023 at 12:38, Timo Walther wrote: > Hi everyone, > > I'd like to start a vote on FLIP-376: Add DISTRIBUTED BY clause[1] which > has been discussed in this thread [2]. > > The vote will be open for at least 72 hours unless there is an objection > or not enough votes. > > [1] > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-376%3A+Add+DISTRIBUTED+BY+clause > [2] https://lists.apache.org/thread/z1p4f38x9ohv29y4nlq8g1wdxwfjwxd1 > > Cheers, > Timo >
[jira] [Created] (FLINK-33474) ShowPlan throws undefined exception In Flink Web Submit Page
Yu Chen created FLINK-33474: --- Summary: ShowPlan throws undefined exception In Flink Web Submit Page Key: FLINK-33474 URL: https://issues.apache.org/jira/browse/FLINK-33474 Project: Flink Issue Type: Bug Components: Runtime / Web Frontend Affects Versions: 1.19.0 Reporter: Yu Chen Attachments: image-2023-11-07-13-53-08-216.png The exception as shown in the figure below, meanwhile, the job plan cannot be displayed properly. The root cause is that the dagreComponent is located in the nz-drawer and is only loaded when the drawer is visible, so we need to wait for the drawer to finish loading and then render the job plan. !image-2023-11-07-13-53-08-216.png|width=400,height=190! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-33473) Update Flink client to 1.18.0
Prabhu Joseph created FLINK-33473: - Summary: Update Flink client to 1.18.0 Key: FLINK-33473 URL: https://issues.apache.org/jira/browse/FLINK-33473 Project: Flink Issue Type: Improvement Components: Kubernetes Operator Reporter: Prabhu Joseph Update the operator flink dependency to Flink-1.18. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [VOTE] Release flink-connector-gcp-pubsub v3.0.2, release candidate #1
+1(non-binding) - Checked release notes - Verified checksums and signatures - Verified no binaries in release - Build connector from source Bests, Samrat On Mon, 6 Nov 2023 at 8:20 PM, Ryan Skraba wrote: > Hello! +1 (non-binding) > > One note: the parent pom still has 1.16.0 for the Maven property of > flink.version for both 1.17 and 1.18 releases. > > I've validated the source for the RC1: > flink-connector-gcp-pubsub-3.0.2-src.tgz at r65060 > * The sha512 checksum is OK. > * The source file is signed correctly. > * The signature 0F79F2AFB2351BC29678544591F9C1EC125FD8DB is found in the > KEYS file, and on https://keyserver.ubuntu.com/ > * The source file is consistent with the GitHub tag v3.0.2-rc1, which > corresponds to commit 4c6be836e6c0f36ef5711f12d7b935254e7d248d > - The files explicitly excluded by create_pristine_sources (such as > .gitignore and the submodule tools/releasing/shared) are not present. > * Has a LICENSE file and a NOTICE file > * Does not contain any compiled binaries. > > * The sources can be compiled and unit tests pass with flink.version 1.17.1 > and flink.version 1.18.0 > > * Nexus has two staged artifact ids for 3.0.2-1.17 and 3.0.2-1.18 > - flink-connector-gcp-pubsub (.jar, -javadoc.jar, -sources.jar and .pom) > - flink-connector-gcp-pubsub-parent (only .pom) > > I did a simple smoke test on an emulated Pub/Sub with the 1.18 version. > > All my best, Ryan Skraba >
[jira] [Created] (FLINK-33472) Solve the problem that the temporary file of flink-conf.yaml in S3AFileSystem cannot be uploaded
zhengzhili created FLINK-33472: -- Summary: Solve the problem that the temporary file of flink-conf.yaml in S3AFileSystem cannot be uploaded Key: FLINK-33472 URL: https://issues.apache.org/jira/browse/FLINK-33472 Project: Flink Issue Type: Bug Components: Client / Job Submission Affects Versions: 1.17.1 Reporter: zhengzhili Attachments: image-2023-11-07-11-34-01-823.png, image-2023-11-07-11-34-29-548.png Solve the problem that the temporary file of flink-conf.yaml in S3AFileSystem cannot be uploaded。 !image-2023-11-07-11-34-29-548.png|width=595,height=175! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[VOTE] FLIP-377: Support configuration to disable filter push down for Table/SQL Sources
Hi Devs, I'd like to start a vote on FLIP-377: Support configuration to disable filter pushdown for Table/SQL Sources[1] which has been discussed in this thread [2]. The vote will be open for at least 72 hours unless there is an objection or not enough votes. [1] https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=276105768 [2] https://lists.apache.org/thread/nvxx8sp9jm009yywm075hoffr632tm7j Best, Jiabao
Re: [DISCUSS] Release 1.17.2
+1 Best, Ron Jing Ge 于2023年11月6日周一 22:07写道: > +1 > Thanks for your effort! > > Best regards, > Jing > > On Mon, Nov 6, 2023 at 1:15 AM Konstantin Knauf wrote: > > > Thank you for picking it up! +1 > > > > Cheers, > > > > Konstantin > > > > Am Mo., 6. Nov. 2023 um 03:48 Uhr schrieb Yun Tang : > > > > > Hi all, > > > > > > I would like to discuss creating a new 1.17 patch release (1.17.2). The > > > last 1.17 release is near half a year old, and since then, 79 tickets > > have > > > been closed [1], of which 15 are blocker/critical [2]. Some > > > of them are quite important, such as FLINK-32758 [3], FLINK-32296 [4], > > > FLINK-32548 [5] > > > and FLINK-33010[6]. > > > > > > In addition to this, FLINK-33149 [7] is important to bump snappy-java > to > > > 1.1.10.4. > > > Although FLINK-33149 is unresolved, it was done in 1.17.2. > > > > > > I am not aware of any unresolved blockers and there are no in-progress > > > tickets [8]. Please let me know if there are any issues you'd like to > be > > > included in this release but still not merged. > > > > > > If the community agrees to create this new patch release, I could > > > volunteer as the release manager with Yu Chen. > > > > > > Since there will be another flink-1.16.3 release request during the > same > > > time, we will work with Rui Fan since many issues will be fixed in both > > > releases. > > > > > > [1] > > > > > > https://issues.apache.org/jira/issues/?jql=project%20%3D%20FLINK%20AND%20fixVersion%20%3D%201.17.2%20%20and%20resolution%20%20!%3D%20%20Unresolved%20order%20by%20priority%20DESC > > > [2] > > > > > > https://issues.apache.org/jira/issues/?jql=project%20%3D%20FLINK%20AND%20fixVersion%20%3D%201.17.2%20and%20resolution%20%20!%3D%20Unresolved%20%20and%20priority%20in%20(Blocker%2C%20Critical)%20ORDER%20by%20priority%20%20DESC > > > [3] https://issues.apache.org/jira/browse/FLINK-32758 > > > [4] https://issues.apache.org/jira/browse/FLINK-32296 > > > [5] https://issues.apache.org/jira/browse/FLINK-32548 > > > [6] https://issues.apache.org/jira/browse/FLINK-33010 > > > [7] https://issues.apache.org/jira/browse/FLINK-33149 > > > [8] https://issues.apache.org/jira/projects/FLINK/versions/12353260 > > > > > > Best > > > Yun Tang > > > > > > > > > -- > > https://twitter.com/snntrable > > https://github.com/knaufk > > >
Re: [DISCUSS] Connector releases for Flink 1.18
Hi, I looked into both connectors, they both are failing at archunit. I tried a few things but with my limited experience couldn't make much progress. Don't want to block the release. If anyone knows what needs to be done, please move forward with the fixes. Hopefully, shall be able to help more in the future. Thanks Ash On Fri, 3 Nov 2023 at 08:35, mystic lama wrote: > Hi, > > I can look into pulsar and Cassandra fixes. I agree it's archunit issue > based on the build logs. > I don't have much experience with it, but I was able to reproduce it > locally. > > I can send out PR's over the weekend. Do we have JIRA's for this? If not I > can create to track. > > Thanks > @sh > > On Fri, 3 Nov 2023 at 02:20, Martijn Visser > wrote: > >> Hi Danny, >> >> Thanks a lot for starting the discussion on this topic! I know that >> Pulsar is failing because of Archunit, which I expect the same issue >> to be for Cassandra (I know that Etienne was working on this). Happy >> to help. >> >> Best regards, >> >> Martijn >> >> On Thu, Nov 2, 2023 at 9:08 PM Danny Cranmer >> wrote: >> > >> > Hey all. >> > >> > Now Flink 1.18 is released we need to do some connector releases for >> > integration parity. We can use this thread to start the discussions for >> > each connector release and spawn separate vote threads. Kafka is done >> [1] >> > (thanks Gordon) and AWS connectors are in process [2], I appreciate help >> > with votes on that one. >> > >> > Opensearch: Flink 1.18 nightly build passing [3]. I volunteer to be >> release >> > manager for this one. I will consolidate 1.0.2 [4] and 1.1.0 [5] into a >> > single release as 1.1.0. >> > MongoDB: Flink 1.18 nightly build passing [6]. I volunteer to be release >> > manager for this one. I will work with Jiabao to get FLINK-33257 merged >> > into 1.1.0 and release that [7]. >> > GCP Pub Sub: Flink 1.18 nightly build passing [8]. I volunteer to be >> > release manager for this one. Looks like 3.0.2 is ready to go [9], we >> > should proceed with this. >> > >> > ElasticSearch: Flink 1.18 nightly build passing [10]. There are a good >> > stack of changes ready for 3.1.0 [11], suggest we release that. >> > JDBC: Flink 1.18 nightly build passing [12]. There are a good stack of >> > changes ready for 3.2.0 [13], suggest we release that. >> > RabbitMQ: Flink 1.18 nightly build passing [14]. There are no changes >> ready >> > for 3.0.2 [15], recommend we do a minimal 3.0.1-1.18. >> > >> > Pulsar: The nightly CI is failing [16], needs a deeper look >> > Cassandra: The nightly CI is failing [17], needs a deeper look >> > >> > Once I have completed Opensearch/MongoDB/GCP I will pick up others, but >> > hope others can help out. >> > >> > Thanks, >> > Danny >> > >> > [1] https://lists.apache.org/thread/0lvrm9hl3hnn1fpr74k68lsm22my8xh7 >> > [2] https://lists.apache.org/thread/ko6nrtfsykkz9c9k9392jfj4l9f7qg11 >> > [3] https://github.com/apache/flink-connector-opensearch/actions >> > [4] https://issues.apache.org/jira/projects/FLINK/versions/12353142 >> > [5] https://issues.apache.org/jira/projects/FLINK/versions/12353141 >> > [6] https://github.com/apache/flink-connector-mongodb/actions >> > [7] https://issues.apache.org/jira/projects/FLINK/versions/12353483 >> > [8] https://github.com/apache/flink-connector-gcp-pubsub/actions >> > [9] https://issues.apache.org/jira/projects/FLINK/versions/12353144 >> > [10] https://github.com/apache/flink-connector-elasticsearch/actions >> > [11] https://issues.apache.org/jira/projects/FLINK/versions/12352520 >> > [12] https://github.com/apache/flink-connector-jdbc/actions >> > [13] https://issues.apache.org/jira/projects/FLINK/versions/12353143 >> > [14] https://github.com/apache/flink-connector-rabbitmq/actions >> > [15] https://issues.apache.org/jira/projects/FLINK/versions/12353145 >> > [16] https://github.com/apache/flink-connector-pulsar/actions >> > [17] https://github.com/apache/flink-connector-cassandra/actions >> >
Re: [VOTE] FLIP-376: Add DISTRIBUTED BY clause
+1 (binding) Lincoln Lee 于2023年11月7日周二 08:50写道: > > +1 (binding) > > Best, > Lincoln Lee > > > Martijn Visser 于2023年11月7日周二 01:02写道: > > > +1 (binding) > > > > Thanks for driving this Timo > > > > On Mon, Nov 6, 2023 at 12:40 PM Timo Walther wrote: > > > > > > Hi everyone, > > > > > > I'd like to start a vote on FLIP-376: Add DISTRIBUTED BY clause[1] which > > > has been discussed in this thread [2]. > > > > > > The vote will be open for at least 72 hours unless there is an objection > > > or not enough votes. > > > > > > [1] > > > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-376%3A+Add+DISTRIBUTED+BY+clause > > > [2] https://lists.apache.org/thread/z1p4f38x9ohv29y4nlq8g1wdxwfjwxd1 > > > > > > Cheers, > > > Timo > > -- Best, Benchao Li
Re: [VOTE] FLIP-376: Add DISTRIBUTED BY clause
+1 (binding) Best, Lincoln Lee Martijn Visser 于2023年11月7日周二 01:02写道: > +1 (binding) > > Thanks for driving this Timo > > On Mon, Nov 6, 2023 at 12:40 PM Timo Walther wrote: > > > > Hi everyone, > > > > I'd like to start a vote on FLIP-376: Add DISTRIBUTED BY clause[1] which > > has been discussed in this thread [2]. > > > > The vote will be open for at least 72 hours unless there is an objection > > or not enough votes. > > > > [1] > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-376%3A+Add+DISTRIBUTED+BY+clause > > [2] https://lists.apache.org/thread/z1p4f38x9ohv29y4nlq8g1wdxwfjwxd1 > > > > Cheers, > > Timo >
[jira] [Created] (FLINK-33471) Kubernetes operator supports compiling with Java 21
Sergey Nuyanzin created FLINK-33471: --- Summary: Kubernetes operator supports compiling with Java 21 Key: FLINK-33471 URL: https://issues.apache.org/jira/browse/FLINK-33471 Project: Flink Issue Type: Improvement Components: Kubernetes Operator Reporter: Sergey Nuyanzin Since there is a new Java LTS version available (21) it would make sense to support it -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-33470) Implement restore tests for Join node
Jim Hughes created FLINK-33470: -- Summary: Implement restore tests for Join node Key: FLINK-33470 URL: https://issues.apache.org/jira/browse/FLINK-33470 Project: Flink Issue Type: Sub-task Reporter: Jim Hughes -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-33469) Implement restore tests for Limit node
Jim Hughes created FLINK-33469: -- Summary: Implement restore tests for Limit node Key: FLINK-33469 URL: https://issues.apache.org/jira/browse/FLINK-33469 Project: Flink Issue Type: Sub-task Reporter: Jim Hughes -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [VOTE] FLIP-376: Add DISTRIBUTED BY clause
+1 (binding) Thanks for driving this Timo On Mon, Nov 6, 2023 at 12:40 PM Timo Walther wrote: > > Hi everyone, > > I'd like to start a vote on FLIP-376: Add DISTRIBUTED BY clause[1] which > has been discussed in this thread [2]. > > The vote will be open for at least 72 hours unless there is an objection > or not enough votes. > > [1] > https://cwiki.apache.org/confluence/display/FLINK/FLIP-376%3A+Add+DISTRIBUTED+BY+clause > [2] https://lists.apache.org/thread/z1p4f38x9ohv29y4nlq8g1wdxwfjwxd1 > > Cheers, > Timo
[jira] [Created] (FLINK-33468) Run Kubernetes IT test fails due to wrong Maven version
Matthias Pohl created FLINK-33468: - Summary: Run Kubernetes IT test fails due to wrong Maven version Key: FLINK-33468 URL: https://issues.apache.org/jira/browse/FLINK-33468 Project: Flink Issue Type: Sub-task Components: Build System / CI Reporter: Matthias Pohl https://github.com/XComp/flink/actions/runs/6772388387/job/18405319623#step:15:4841 {code} [...] Error: 15:52:19 15:52:19.837 [ERROR] Rule 0: org.apache.maven.plugins.enforcer.RequireMavenVersion failed with message: Nov 06 15:52:19 Detected Maven Version: 3.8.8 is not in the allowed range [3.8.6,3.8.6]. Nov 06 15:52:19 15:52:19.837 [INFO] Nov 06 15:52:19 15:52:19.837 [INFO] BUILD FAILURE Nov 06 15:52:19 15:52:19.838 [INFO] Nov 06 15:52:19 15:52:19.839 [INFO] Total time: 9.755 s Nov 06 15:52:19 15:52:19.839 [INFO] Finished at: 2023-11-06T15:52:19Z Nov 06 15:52:19 15:52:19.839 [INFO] Error: 15:52:19 15:52:19.840 [ERROR] Failed to execute goal org.apache.maven.plugins:maven-enforcer-plugin:3.1.0:enforce (enforce-maven) on project flink-kubernetes: Some Enforcer rules have failed. Look above for specific messages explaining why the rule failed. -> [Help 1] Error: 15:52:19 15:52:19.840 [ERROR] Error: 15:52:19 15:52:19.840 [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. Error: 15:52:19 15:52:19.840 [ERROR] Re-run Maven using the -X switch to enable full debug logging. Error: 15:52:19 15:52:19.840 [ERROR] Error: 15:52:19 15:52:19.840 [ERROR] For more information about the errors and possible solutions, please read the following articles: Error: 15:52:19 15:52:19.840 [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException {code} -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [VOTE] Release flink-connector-opensearch v1.1.0, release candidate #1
Hello! +1 (non-binding) Thanks for the release! I've validated the source for the RC1: * flink-connector-opensearch-1.1.0-src.tgz at r64995 * The sha512 checksum is OK. * The source file is signed correctly. * The signature 0F79F2AFB2351BC29678544591F9C1EC125FD8DB is found in the KEYS file, and on https://keyserver.ubuntu.com/ * The source file is consistent with the GitHub tag v1.1.0-rc1, which corresponds to commit 0f659cc65131c9ff7c8c35eb91f5189e80414ea1 - The files explicitly excluded by create_pristine_sources (such as .gitignore and the submodule tools/releasing/shared) are not present. * Has a LICENSE file and a NOTICE file * Does not contain any compiled binaries. * The sources can be compiled and unit tests pass with flink.version 1.17.1 and flink.version 1.18.0 * Nexus has three staged artifact ids for 1.1.0-1.17 and 1.1.0-1.18 - flink-connector-opensearch (.jar, -javadoc.jar, -sources.jar, -tests.jar and .pom) - flink-sql-connector-opensearch (.jar, -sources.jar and .pom) - flink-connector-gcp-pubsub-parent (only .pom) All my best, Ryan On Fri, Nov 3, 2023 at 10:29 AM Danny Cranmer wrote: > > Hi everyone, > > Please review and vote on the release candidate #1 for the version 1.1.0 of > flink-connector-opensearch, as follows: > [ ] +1, Approve the release > [ ] -1, Do not approve the release (please provide specific comments) > > > The complete staging area is available for your review, which includes: > * JIRA release notes [1], > * the official Apache source release to be deployed to dist.apache.org [2], > which are signed with the key with fingerprint > 0F79F2AFB2351BC29678544591F9C1EC125FD8DB [3], > * all artifacts to be deployed to the Maven Central Repository [4], > * source code tag v1.1.0-rc1 [5], > * website pull request listing the new release [6]. > > The vote will be open for at least 72 hours. It is adopted by majority > approval, with at least 3 PMC affirmative votes. > > Thanks, > Danny > > [1] > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12353141 > [2] > https://dist.apache.org/repos/dist/dev/flink/flink-connector-opensearch-1.1.0-rc1/ > [3] https://dist.apache.org/repos/dist/release/flink/KEYS > [4] https://repository.apache.org/content/repositories/orgapacheflink-1666/ > [5] https://github.com/apache/flink-connector-opensearch/tree/v1.1.0-rc1 > [6] https://github.com/apache/flink-web/pull/694
Re: [VOTE] Release flink-connector-gcp-pubsub v3.0.2, release candidate #1
Hello! +1 (non-binding) One note: the parent pom still has 1.16.0 for the Maven property of flink.version for both 1.17 and 1.18 releases. I've validated the source for the RC1: flink-connector-gcp-pubsub-3.0.2-src.tgz at r65060 * The sha512 checksum is OK. * The source file is signed correctly. * The signature 0F79F2AFB2351BC29678544591F9C1EC125FD8DB is found in the KEYS file, and on https://keyserver.ubuntu.com/ * The source file is consistent with the GitHub tag v3.0.2-rc1, which corresponds to commit 4c6be836e6c0f36ef5711f12d7b935254e7d248d - The files explicitly excluded by create_pristine_sources (such as .gitignore and the submodule tools/releasing/shared) are not present. * Has a LICENSE file and a NOTICE file * Does not contain any compiled binaries. * The sources can be compiled and unit tests pass with flink.version 1.17.1 and flink.version 1.18.0 * Nexus has two staged artifact ids for 3.0.2-1.17 and 3.0.2-1.18 - flink-connector-gcp-pubsub (.jar, -javadoc.jar, -sources.jar and .pom) - flink-connector-gcp-pubsub-parent (only .pom) I did a simple smoke test on an emulated Pub/Sub with the 1.18 version. All my best, Ryan Skraba
Re: [DISCUSS] Release 1.17.2
+1 Thanks for your effort! Best regards, Jing On Mon, Nov 6, 2023 at 1:15 AM Konstantin Knauf wrote: > Thank you for picking it up! +1 > > Cheers, > > Konstantin > > Am Mo., 6. Nov. 2023 um 03:48 Uhr schrieb Yun Tang : > > > Hi all, > > > > I would like to discuss creating a new 1.17 patch release (1.17.2). The > > last 1.17 release is near half a year old, and since then, 79 tickets > have > > been closed [1], of which 15 are blocker/critical [2]. Some > > of them are quite important, such as FLINK-32758 [3], FLINK-32296 [4], > > FLINK-32548 [5] > > and FLINK-33010[6]. > > > > In addition to this, FLINK-33149 [7] is important to bump snappy-java to > > 1.1.10.4. > > Although FLINK-33149 is unresolved, it was done in 1.17.2. > > > > I am not aware of any unresolved blockers and there are no in-progress > > tickets [8]. Please let me know if there are any issues you'd like to be > > included in this release but still not merged. > > > > If the community agrees to create this new patch release, I could > > volunteer as the release manager with Yu Chen. > > > > Since there will be another flink-1.16.3 release request during the same > > time, we will work with Rui Fan since many issues will be fixed in both > > releases. > > > > [1] > > > https://issues.apache.org/jira/issues/?jql=project%20%3D%20FLINK%20AND%20fixVersion%20%3D%201.17.2%20%20and%20resolution%20%20!%3D%20%20Unresolved%20order%20by%20priority%20DESC > > [2] > > > https://issues.apache.org/jira/issues/?jql=project%20%3D%20FLINK%20AND%20fixVersion%20%3D%201.17.2%20and%20resolution%20%20!%3D%20Unresolved%20%20and%20priority%20in%20(Blocker%2C%20Critical)%20ORDER%20by%20priority%20%20DESC > > [3] https://issues.apache.org/jira/browse/FLINK-32758 > > [4] https://issues.apache.org/jira/browse/FLINK-32296 > > [5] https://issues.apache.org/jira/browse/FLINK-32548 > > [6] https://issues.apache.org/jira/browse/FLINK-33010 > > [7] https://issues.apache.org/jira/browse/FLINK-33149 > > [8] https://issues.apache.org/jira/projects/FLINK/versions/12353260 > > > > Best > > Yun Tang > > > > > -- > https://twitter.com/snntrable > https://github.com/knaufk >
Re: [DISCUSS] Release Flink 1.16.3
+1 for the final bug fix for Flink 1.16. Thanks for your effort! Best regards, Jing On Mon, Nov 6, 2023 at 1:14 AM Konstantin Knauf wrote: > +1. Thank you for picking it up. Yes, this will be the final bug fix for > Flink 1.16. > > Am Mo., 6. Nov. 2023 um 03:47 Uhr schrieb Rui Fan <1996fan...@gmail.com>: > > > Hi all, > > > > I would like to discuss creating a new 1.16 patch release (1.16.3). The > > last 1.16 release is over five months old, and since then, 50 tickets > have > > been closed [1], of which 10 are blocker/critical [2]. Some > > of them are quite important, such as FLINK-32296 [3], FLINK-32548 [4] > > and FLINK-33010[5]. > > > > In addition to this, FLINK-33149 [6] is important to bump snappy-java to > > 1.1.10.4. > > Although FLINK-33149 is unresolved, it was done in 1.16.3. > > > > I am not aware of any unresolved blockers and there are no in-progress > > tickets [7]. Please let me know if there are any issues you'd like to be > > included in this release but still not merged. > > > > Since 1.18.0 has been released, I'd suggest that we vote to make 1.16.3 > > the final bugix release of 1.16, looking forward to any feedback from > you. > > Background info could be found at [8], and thanks Jing for the > information. > > > > If the community agrees to create this new patch release, I could > > volunteer as the release manager. > > > > Since there will be another flink-1.17.2 release request during the same > > time, > > I will work with Yun and Yu since many issues will be fixed in both > > releases. > > > > [1] > > > > > https://issues.apache.org/jira/browse/FLINK-32231?jql=project%20%3D%20FLINK%20AND%20fixVersion%20%3D%201.16.3%20%20and%20resolution%20%20!%3D%20%20Unresolved%20order%20by%20priority%20DESC > > [2] > > > > > https://issues.apache.org/jira/browse/FLINK-32231?jql=project%20%3D%20FLINK%20AND%20fixVersion%20%3D%201.16.3%20and%20resolution%20%20!%3D%20Unresolved%20%20and%20priority%20in%20(Blocker%2C%20Critical)%20ORDER%20by%20priority%20%20DESC > > [3] https://issues.apache.org/jira/browse/FLINK-32296 > > [4] https://issues.apache.org/jira/browse/FLINK-32548 > > [5] https://issues.apache.org/jira/browse/FLINK-33010 > > [6] https://issues.apache.org/jira/browse/FLINK-33149 > > [7] https://issues.apache.org/jira/projects/FLINK/versions/12353259 > > [8] https://lists.apache.org/thread/szq23kr3rlkm80rw7k9n95js5vqpsnbv > > > > Best, > > Rui > > > > > -- > https://twitter.com/snntrable > https://github.com/knaufk >
Re: [DISCUSS] FLIP-381: Deprecate configuration getters/setters that return/set complex Java objects
Hi Junrui, Thanks for updating the FLIP! It looks good and clear now. Appreciate it! Best regards, Jing On Sun, Nov 5, 2023 at 7:35 PM Junrui Lee wrote: > Hi Jing, > > Thank you for your feedback on the proposal. I appreciate your input and > understand your concerns. > > Regarding your questions, the complex Java class that will be deprecated is > org.apache.flink.api.common.restartstrategy.RestartStrategies and and its > internal classes, which include: > > - RestartStrategyConfiguration > - FallbackRestartStrategyConfiguration > - FailureRateRestartStrategyConfiguration > - ExponentialDelayRestartStrategyConfiguration > - FixedDelayRestartStrategyConfiguration > - NoRestartStrategyConfiguration > > And based on your suggestion, I have restructured the content of the FLIP > to clearly indicate which entire classes will be deprecated and which > methods and fields will be deprecated. Please take a look at the updated > version. > > Best regards, > Junrui > > Jing Ge 于2023年11月5日周日 00:31写道: > > > Hi Junrui, > > > > Thanks for bringing the proposal to our attention. It looks overall > great. > > > > I am just a little bit confused with the content of Flip with the > following > > questions: > > > > 1. How many complex java classes will be deprecated? > > 2. After reading the section title "Deprecate following classes, fields > and > > methods", I thought that e.g. > > org.apache.flink.streaming.api.environment.StreamExecutionEnvironment > will > > be deprecated. I would suggest restructure the content and point out > > clearly which entire classes will be deprecated and which methods and > > fields and not the class itself will be deprecated instead of mixing them > > up. WDYT? > > > > Best regards, > > Jing > > > > > > On Fri, Nov 3, 2023 at 8:59 AM Wencong Liu wrote: > > > > > Thanks Junrui for your effort! > > > > > > Making all configuration code paths lead to ConfigOption is a more > > > standardized > > > approach to configuring Flink applications. > > > > > > +1 for this proposal. > > > > > > Best, > > > Wencong Liu > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > At 2023-11-02 10:10:14, "Junrui Lee" wrote: > > > >Hi devs, > > > > > > > >I would like to start a discussion on FLIP-381: Deprecate > configuration > > > >getters/setters that return/set complex Java objects[1]. > > > > > > > >Currently, the job configuration in FLINK is spread out across > different > > > >components, which leads to inconsistencies and confusion. To address > > this > > > >issue, it is necessary to migrate non-ConfigOption complex Java > objects > > to > > > >use ConfigOption and adopt a single Configuration object to host all > the > > > >configuration. > > > >However, there is a significant blocker in implementing this solution. > > > >These complex Java objects in StreamExecutionEnvironment, > > > CheckpointConfig, > > > >and ExecutionConfig have already been exposed through the public API, > > > >making it challenging to modify the existing implementation. > > > > > > > >Therefore, I propose to deprecate these Java objects and their > > > >corresponding getter/setter interfaces, ultimately removing them in > > > >FLINK-2.0. > > > > > > > >Your feedback and thoughts on this proposal are highly appreciated. > > > > > > > >Best regards, > > > >Junrui Lee > > > > > > > >[1] > > > > > > > > > > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=278464992 > > > > > >
Review request
Hi everyone, Is there anyone available to review this PR (1) that I opened 1,5 month ago ? People I've pinged seem to be unavailable at the moment. Thanks [1] https://github.com/apache/flink/pull/23443 Best Etienne
Re: [VOTE] FLIP-376: Add DISTRIBUTED BY clause
+1 (binding) Thanks! Best regards, Jing On Mon, Nov 6, 2023 at 3:40 AM Timo Walther wrote: > Hi everyone, > > I'd like to start a vote on FLIP-376: Add DISTRIBUTED BY clause[1] which > has been discussed in this thread [2]. > > The vote will be open for at least 72 hours unless there is an objection > or not enough votes. > > [1] > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-376%3A+Add+DISTRIBUTED+BY+clause > [2] https://lists.apache.org/thread/z1p4f38x9ohv29y4nlq8g1wdxwfjwxd1 > > Cheers, > Timo >
[jira] [Created] (FLINK-33467) Support concurrent serialization in akka
dizhou cao created FLINK-33467: -- Summary: Support concurrent serialization in akka Key: FLINK-33467 URL: https://issues.apache.org/jira/browse/FLINK-33467 Project: Flink Issue Type: Sub-task Components: Runtime / RPC Reporter: dizhou cao In OLAP high QPS scenarios, there are a large number of RPC requests for deploy task and update task status. At this time, the serialization and deserialization performed by AKKA threads become a bottleneck. By supporting parallelization of akka serialization operations, the performance of akka in accepting and processing a large number of RPC requests can be improved. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[VOTE] FLIP-376: Add DISTRIBUTED BY clause
Hi everyone, I'd like to start a vote on FLIP-376: Add DISTRIBUTED BY clause[1] which has been discussed in this thread [2]. The vote will be open for at least 72 hours unless there is an objection or not enough votes. [1] https://cwiki.apache.org/confluence/display/FLINK/FLIP-376%3A+Add+DISTRIBUTED+BY+clause [2] https://lists.apache.org/thread/z1p4f38x9ohv29y4nlq8g1wdxwfjwxd1 Cheers, Timo
[jira] [Created] (FLINK-33466) Bounded Kafka source never finishes after restore from savepoint
Jonas Weile created FLINK-33466: --- Summary: Bounded Kafka source never finishes after restore from savepoint Key: FLINK-33466 URL: https://issues.apache.org/jira/browse/FLINK-33466 Project: Flink Issue Type: Bug Components: Connectors / Kafka, Runtime / Checkpointing Affects Versions: 1.17.1 Reporter: Jonas Weile When setting up a bounded Kafka source with the OffsetsInitializer.latest(), if the job is restored from a savepoint before the source has finished, then the Kafka source will never transition to a finished state. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [VOTE] Release flink-connector-gcp-pubsub v3.0.2, release candidate #1
+1(non-binding) -Checked release notes - Verified checksums and signatures - Verified no binaries in release - Build connector from source - Reviewed web PR. Best Regards Ahmed Hamdy On Fri, 3 Nov 2023 at 11:23, Danny Cranmer wrote: > Hi everyone, > > Please review and vote on the release candidate #1 for the version 3.0.2 of > flink-connector-pubsub, as follows: > [ ] +1, Approve the release > [ ] -1, Do not approve the release (please provide specific comments) > > The complete staging area is available for your review, which includes: > * JIRA release notes [1], > * the official Apache source release to be deployed to dist.apache.org > [2], > which are signed with the key with fingerprint > 0F79F2AFB2351BC29678544591F9C1EC125FD8DB [3], > * all artifacts to be deployed to the Maven Central Repository [4], > * source code tag v3.0.2-rc1 [5], > * website pull request listing the new release [6]. > > The vote will be open for at least 72 hours. It is adopted by majority > approval, with at least 3 PMC affirmative votes. > > Thanks, > Danny > > [1] > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12353144 > [2] > > https://dist.apache.org/repos/dist/dev/flink/flink-connector-gcp-pubsub-3.0.2-rc1 > [3] https://dist.apache.org/repos/dist/release/flink/KEYS > [4] > https://repository.apache.org/content/repositories/orgapacheflink-1667/ > [5] https://github.com/apache/flink-connector-gcp-pubsub/tree/v3.0.2-rc1 > [6] https://github.com/apache/flink-web/pull/695 >
Re: [VOTE] Release flink-connector-opensearch v1.1.0, release candidate #1
Hi Danny, +1 (non-binding) - Checked release notes - Verified Checksums and Signature - Checked no binaries in release. - Reviewed web PR - Build Connector from source. Best Regards Ahmed Hamdy On Fri, 3 Nov 2023 at 13:05, Andriy Redko wrote: > +1 (non-binding), thanks a lot Danny! > > Best Regards, > Andriy Redko > > > Hi everyone, > > > Please review and vote on the release candidate #1 for the version 1.1.0 > of > > flink-connector-opensearch, as follows: > > [ ] +1, Approve the release > > [ ] -1, Do not approve the release (please provide specific comments) > > > > The complete staging area is available for your review, which includes: > > * JIRA release notes [1], > > * the official Apache source release to be deployed to dist.apache.org > [2], > > which are signed with the key with fingerprint > > 0F79F2AFB2351BC29678544591F9C1EC125FD8DB [3], > > * all artifacts to be deployed to the Maven Central Repository [4], > > * source code tag v1.1.0-rc1 [5], > > * website pull request listing the new release [6]. > > > The vote will be open for at least 72 hours. It is adopted by majority > > approval, with at least 3 PMC affirmative votes. > > > Thanks, > > Danny > > > [1] > > > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12353141 > > [2] > > > https://dist.apache.org/repos/dist/dev/flink/flink-connector-opensearch-1.1.0-rc1/ > > [3] https://dist.apache.org/repos/dist/release/flink/KEYS > > [4] > https://repository.apache.org/content/repositories/orgapacheflink-1666/ > > [5] https://github.com/apache/flink-connector-opensearch/tree/v1.1.0-rc1 > > [6] https://github.com/apache/flink-web/pull/694 > >
Re: [DISCUSS] Release Flink 1.16.3
+1. Thank you for picking it up. Yes, this will be the final bug fix for Flink 1.16. Am Mo., 6. Nov. 2023 um 03:47 Uhr schrieb Rui Fan <1996fan...@gmail.com>: > Hi all, > > I would like to discuss creating a new 1.16 patch release (1.16.3). The > last 1.16 release is over five months old, and since then, 50 tickets have > been closed [1], of which 10 are blocker/critical [2]. Some > of them are quite important, such as FLINK-32296 [3], FLINK-32548 [4] > and FLINK-33010[5]. > > In addition to this, FLINK-33149 [6] is important to bump snappy-java to > 1.1.10.4. > Although FLINK-33149 is unresolved, it was done in 1.16.3. > > I am not aware of any unresolved blockers and there are no in-progress > tickets [7]. Please let me know if there are any issues you'd like to be > included in this release but still not merged. > > Since 1.18.0 has been released, I'd suggest that we vote to make 1.16.3 > the final bugix release of 1.16, looking forward to any feedback from you. > Background info could be found at [8], and thanks Jing for the information. > > If the community agrees to create this new patch release, I could > volunteer as the release manager. > > Since there will be another flink-1.17.2 release request during the same > time, > I will work with Yun and Yu since many issues will be fixed in both > releases. > > [1] > > https://issues.apache.org/jira/browse/FLINK-32231?jql=project%20%3D%20FLINK%20AND%20fixVersion%20%3D%201.16.3%20%20and%20resolution%20%20!%3D%20%20Unresolved%20order%20by%20priority%20DESC > [2] > > https://issues.apache.org/jira/browse/FLINK-32231?jql=project%20%3D%20FLINK%20AND%20fixVersion%20%3D%201.16.3%20and%20resolution%20%20!%3D%20Unresolved%20%20and%20priority%20in%20(Blocker%2C%20Critical)%20ORDER%20by%20priority%20%20DESC > [3] https://issues.apache.org/jira/browse/FLINK-32296 > [4] https://issues.apache.org/jira/browse/FLINK-32548 > [5] https://issues.apache.org/jira/browse/FLINK-33010 > [6] https://issues.apache.org/jira/browse/FLINK-33149 > [7] https://issues.apache.org/jira/projects/FLINK/versions/12353259 > [8] https://lists.apache.org/thread/szq23kr3rlkm80rw7k9n95js5vqpsnbv > > Best, > Rui > -- https://twitter.com/snntrable https://github.com/knaufk
Re: [DISCUSS] Release 1.17.2
Thank you for picking it up! +1 Cheers, Konstantin Am Mo., 6. Nov. 2023 um 03:48 Uhr schrieb Yun Tang : > Hi all, > > I would like to discuss creating a new 1.17 patch release (1.17.2). The > last 1.17 release is near half a year old, and since then, 79 tickets have > been closed [1], of which 15 are blocker/critical [2]. Some > of them are quite important, such as FLINK-32758 [3], FLINK-32296 [4], > FLINK-32548 [5] > and FLINK-33010[6]. > > In addition to this, FLINK-33149 [7] is important to bump snappy-java to > 1.1.10.4. > Although FLINK-33149 is unresolved, it was done in 1.17.2. > > I am not aware of any unresolved blockers and there are no in-progress > tickets [8]. Please let me know if there are any issues you'd like to be > included in this release but still not merged. > > If the community agrees to create this new patch release, I could > volunteer as the release manager with Yu Chen. > > Since there will be another flink-1.16.3 release request during the same > time, we will work with Rui Fan since many issues will be fixed in both > releases. > > [1] > https://issues.apache.org/jira/issues/?jql=project%20%3D%20FLINK%20AND%20fixVersion%20%3D%201.17.2%20%20and%20resolution%20%20!%3D%20%20Unresolved%20order%20by%20priority%20DESC > [2] > https://issues.apache.org/jira/issues/?jql=project%20%3D%20FLINK%20AND%20fixVersion%20%3D%201.17.2%20and%20resolution%20%20!%3D%20Unresolved%20%20and%20priority%20in%20(Blocker%2C%20Critical)%20ORDER%20by%20priority%20%20DESC > [3] https://issues.apache.org/jira/browse/FLINK-32758 > [4] https://issues.apache.org/jira/browse/FLINK-32296 > [5] https://issues.apache.org/jira/browse/FLINK-32548 > [6] https://issues.apache.org/jira/browse/FLINK-33010 > [7] https://issues.apache.org/jira/browse/FLINK-33149 > [8] https://issues.apache.org/jira/projects/FLINK/versions/12353260 > > Best > Yun Tang > -- https://twitter.com/snntrable https://github.com/knaufk
[jira] [Created] (FLINK-33465) Make SingleThreadFetcherManager and FutureCompletingBlockingQueue as PublicEvolving.
Hongshun Wang created FLINK-33465: - Summary: Make SingleThreadFetcherManager and FutureCompletingBlockingQueue as PublicEvolving. Key: FLINK-33465 URL: https://issues.apache.org/jira/browse/FLINK-33465 Project: Flink Issue Type: Improvement Components: Connectors / Parent Affects Versions: 1.18.0 Reporter: Hongshun Wang Fix For: 1.19.0 As discussed in FLINK-31324, though the {{SingleThreadFetcherManager}} is annotated as {{{}Internal{}}}, it actually acts as some-degree public API, which is widely used in many connector projects: [flink-cdc-connector|https://github.com/ververica/flink-cdc-connectors/blob/release-2.3.0/flink-connector-mysql-cdc/src/main/java/com/ververica/cdc/connectors/mysql/source/reader/MySqlSourceReader.java#L93], [flink-connector-mongodb|https://github.com/apache/flink-connector-mongodb/blob/main/flink-connector-mongodb/src/main/java/org/apache/flink/connector/mongodb/source/reader/MongoSourceReader.java#L58] and so on. More over, even the constructor of `SingleThreadMultiplexSourceReaderBase` (which is PublicEvolving) includes the params of `SingleThreadFetcherManager` and `FutureCompletingBlockingQueue` .That means that the `SingleThreadFetcherManager` and `FutureCompletingBlockingQueue`have already been exposed to users for a long time and are widely used. ```java public SingleThreadMultiplexSourceReaderBase( FutureCompletingBlockingQueue> elementsQueue, SingleThreadFetcherManager splitFetcherManager, RecordEmitter recordEmitter, Configuration config, SourceReaderContext context) { super(elementsQueue, splitFetcherManager, recordEmitter, config, context); } ``` Thus, why not make SingleThreadFetcherManager and FutureCompletingBlockingQueue PublicEvolving? -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-33464) JoinITCase.testRightOuterJoin failed due to heartbeat timeout
Matthias Pohl created FLINK-33464: - Summary: JoinITCase.testRightOuterJoin failed due to heartbeat timeout Key: FLINK-33464 URL: https://issues.apache.org/jira/browse/FLINK-33464 Project: Flink Issue Type: Sub-task Components: Runtime / Coordination Reporter: Matthias Pohl https://github.com/XComp/flink/actions/runs/6756936036/job/18367079822#step:12:11525 {code} Error: 21:46:20 21:46:20.936 [ERROR] Tests run: 196, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 788.192 s <<< FAILURE! - in org.apache.flink.table.planner.runtime.batch.sql.join.JoinITCase Error: 21:46:20 21:46:20.936 [ERROR] org.apache.flink.table.planner.runtime.batch.sql.join.JoinITCase.testRightOuterJoin Time elapsed: 68.118 s <<< ERROR! Nov 04 21:46:20 java.lang.RuntimeException: Failed to fetch next result Nov 04 21:46:20 at org.apache.flink.streaming.api.operators.collect.CollectResultIterator.nextResultFromFetcher(CollectResultIterator.java:118) Nov 04 21:46:20 at org.apache.flink.streaming.api.operators.collect.CollectResultIterator.hasNext(CollectResultIterator.java:89) Nov 04 21:46:20 at org.apache.flink.table.planner.connectors.CollectDynamicSink$CloseableRowIteratorWrapper.hasNext(CollectDynamicSink.java:230) Nov 04 21:46:20 at java.base/java.util.Iterator.forEachRemaining(Iterator.java:132) Nov 04 21:46:20 at org.apache.flink.util.CollectionUtil.iteratorToList(CollectionUtil.java:122) Nov 04 21:46:20 at org.apache.flink.table.planner.runtime.utils.BatchTestBase.executeQuery(BatchTestBase.scala:309) Nov 04 21:46:20 at org.apache.flink.table.planner.runtime.utils.BatchTestBase.check(BatchTestBase.scala:145) Nov 04 21:46:20 at org.apache.flink.table.planner.runtime.utils.BatchTestBase.checkResult(BatchTestBase.scala:109) Nov 04 21:46:20 at org.apache.flink.table.planner.runtime.batch.sql.join.JoinITCase.testRightOuterJoin(JoinITCase.scala:892) Nov 04 21:46:20 at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) Nov 04 21:46:20 at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) Nov 04 21:46:20 at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) Nov 04 21:46:20 at java.base/java.lang.reflect.Method.invoke(Method.java:566) Nov 04 21:46:20 at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:727) Nov 04 21:46:20 at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60) Nov 04 21:46:20 at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131) Nov 04 21:46:20 at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:156) Nov 04 21:46:20 at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:147) Nov 04 21:46:20 at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestTemplateMethod(TimeoutExtension.java:94) Nov 04 21:46:20 at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(InterceptingExecutableInvoker.java:103) Nov 04 21:46:20 at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.lambda$invoke$0(InterceptingExecutableInvoker.java:93) Nov 04 21:46:20 at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106) Nov 04 21:46:20 at org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64) Nov 04 21:46:20 at org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45) Nov 04 21:46:20 at org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37) Nov 04 21:46:20 at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:92) Nov 04 21:46:20 at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:86) Nov 04 21:46:20 at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$7(TestMethodTestDescriptor.java:217) Nov 04 21:46:20 at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73) Nov 04 21:46:20 at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:213) Nov 04 21:46:20 at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDes