[jira] [Created] (FLINK-32038) OffsetCommitMode.Kafka_periodic with checkpointing enabled
Pritam Agarwala created FLINK-32038:
---
Summary: OffsetCommitMode.Kafka_periodic with checkpointing
enabled
Key: FLINK-32038
URL: https://issues.apache.org/jira/browse/FLINK-32038
Project: Flink
Issue Type: Improvement
Reporter: Pritam Agarwala
I need to get kafka-lag to prepare a graph and its dependent on kafka committed
offset. Flink is updating the offsets only after checkpointing to make it
consistent.
Default Behaviour as per doc :
If checkpoint is enabled, but {{consumer.setCommitOffsetsOnCheckpoints}} set to
false, then offset will not be committed at all even if the
{{enable.auto.commit}} is set to true.
So, when {{consumer.setCommitOffsetsOnCheckpoints}} set to false, *shouldn't it
fall back on the {{enable.auto.commit}} to do offset commit regularly since*
*in any case flink doesn't use consumer committed offsets for recovery.*
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (FLINK-32037) The edge on Web UI is wrong after parallelism changes via parallelism overrides or AdaptiveScheduler
Zhanghao Chen created FLINK-32037: - Summary: The edge on Web UI is wrong after parallelism changes via parallelism overrides or AdaptiveScheduler Key: FLINK-32037 URL: https://issues.apache.org/jira/browse/FLINK-32037 Project: Flink Issue Type: Improvement Components: Runtime / Coordination, Runtime / REST Affects Versions: 1.17.0 Reporter: Zhanghao Chen *Background* After FLINK-30213, in case of parallelism changes to the JobGraph, as done via the AdaptiveScheduler or through providing JobVertexId overrides in PipelineOptions#PARALLELISM_OVERRIDES, when the consumer parallelism doesn't match the local parallelism, the original ForwardPartitioner will be replaced with the RebalancePartitioner. *Problem* Although the actual partitioner changes underneath, the ship strategy seen on the Web UI is still FORWARD. This is because the fix patch applies when we init StreamTask, and the job graph is not touched. Web UI uses the JSON plan generated from the job graph for display, and the ship strategy is get by JobEdge#getShipStrategyName. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-32036) TableEnvironmentTest.test_explain is unstable on azure ci
Sergey Nuyanzin created FLINK-32036:
---
Summary: TableEnvironmentTest.test_explain is unstable on azure ci
Key: FLINK-32036
URL: https://issues.apache.org/jira/browse/FLINK-32036
Project: Flink
Issue Type: Bug
Components: API / Python
Affects Versions: 1.17.1
Reporter: Sergey Nuyanzin
it's failed on ci (1.17 branch so far)
{noformat}
May 07 01:51:35 === FAILURES
===
May 07 01:51:35 __ TableEnvironmentTest.test_explain
___
May 07 01:51:35
May 07 01:51:35 self =
May 07 01:51:35
May 07 01:51:35 def test_explain(self):
May 07 01:51:35 schema = RowType() \
May 07 01:51:35 .add('a', DataTypes.INT()) \
May 07 01:51:35 .add('b', DataTypes.STRING()) \
May 07 01:51:35 .add('c', DataTypes.STRING())
May 07 01:51:35 t_env = self.t_env
May 07 01:51:35 t = t_env.from_elements([], schema)
May 07 01:51:35 result = t.select(t.a + 1, t.b, t.c)
May 07 01:51:35
May 07 01:51:35 > actual = result.explain()
May 07 01:51:35
May 07 01:51:35 pyflink/table/tests/test_table_environment_api.py:66
{noformat}
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=48766=logs=bf5e383b-9fd3-5f02-ca1c-8f788e2e76d3=85189c57-d8a0-5c9c-b61d-fc05cfac62cf=25029
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (FLINK-32035) SQL Client should support HTTPS with built-in JDK certificates
Alexander Fedulov created FLINK-32035: - Summary: SQL Client should support HTTPS with built-in JDK certificates Key: FLINK-32035 URL: https://issues.apache.org/jira/browse/FLINK-32035 Project: Flink Issue Type: Improvement Components: Table SQL / Client, Table SQL / Gateway Reporter: Alexander Fedulov Assignee: Alexander Fedulov -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-32034) Python's DistUtils is deprecated as of 3.10
Colten Pilgreen created FLINK-32034: --- Summary: Python's DistUtils is deprecated as of 3.10 Key: FLINK-32034 URL: https://issues.apache.org/jira/browse/FLINK-32034 Project: Flink Issue Type: Bug Components: API / Python Affects Versions: 1.17.0 Environment: Kubernetes Java 11 Python 3.10.9 Reporter: Colten Pilgreen I have recent just went through an upgrade from 1.13 to 1.17, along with that I upgraded the python version on our Flink Session server. Most everything that is part of our workflow works, except for Python Dependency Management. After doing some digging, I found the reason is due to the DeprecationWarning that is printed when trying to get the site packages path. The script is ` GET_SITE_PACKAGES_PATH_SCRIPT` and it is executed in the `getSitePackagesPath` method in the `PythonEnvironmentManagerUtils` class. The issue is that the DeprecationWarning is included into the `PYTHONPATH` environment variable which is passed to the beam runner. The deprecation warning breaks Python's ability to find the site packages due to characters that are not allowed in filesystem paths. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-32033) Resource lifecycle state shows stable even if application deployment goes missing
Gyula Fora created FLINK-32033: -- Summary: Resource lifecycle state shows stable even if application deployment goes missing Key: FLINK-32033 URL: https://issues.apache.org/jira/browse/FLINK-32033 Project: Flink Issue Type: Bug Components: Kubernetes Operator Affects Versions: kubernetes-operator-1.4.0 Reporter: Gyula Fora Currently the lifecycle state shows STABLE even if an application deployment was deleted and stays in MISSING / reconciling state. We should consider these FAILED as they won't be recovered. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (FLINK-32032) Upgrade to flink-shaded 17.0
Martijn Visser created FLINK-32032: -- Summary: Upgrade to flink-shaded 17.0 Key: FLINK-32032 URL: https://issues.apache.org/jira/browse/FLINK-32032 Project: Flink Issue Type: Technical Debt Reporter: Martijn Visser Assignee: Martijn Visser Fix For: 1.18.0 -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [VOTE] Release flink-shaded 17.0, release candidate #1
Thanks all. This vote is now closed, I'll announce the results in a separate thread. On Mon, May 8, 2023 at 2:26 PM Martijn Visser wrote: > +1 (binding) > > - Validated hashes > - Verified signature > - Verified that no binaries exist in the source archive > - Build the source with Maven > - Verified licenses > - Verified web PRs > > On Thu, May 4, 2023 at 6:37 AM Leonard Xu wrote: > >> +1 (binding) >> >> - built from source code succeeded >> - verified signatures >> - verified hashsums >> - checked Github release tag >> - checked release notes >> - reviewed the web PR >> - filed issue FLINK-31994[1] to update the copyright year, not a blocker >> >> Best, >> Leonard >> [1] https://issues.apache.org/jira/browse/FLINK-31994 >> >> > 2023年5月2日 下午8:48，Danny Cranmer 写道： >> > >> > +1 (binding) >> > >> > - Release notes look good >> > - Source archive signature and checksums good >> > - Verified there are no binaries in the source archive >> > - Contents of Maven repo look good/match source build >> > - Built from sourcepass >> > - Tag is present in Github >> > - Verified NOTICE files >> > >> > Observations >> > - NOTICE file copyright still says 2021, not blocking but needs to be >> > updated >> > >> > Thanks, >> > Danny >> > >> > On Thu, Apr 20, 2023 at 5:02 PM Matthias Pohl >> > wrote: >> > >> >> Thanks Martijn for driving the flink-shaded release. >> >> >> >> +1 (non-binding) >> >> >> >> * Downloaded artifact >> >> * Verified SHA512 checksums >> >> * GPG signatures Compared >> >> * checkout with provided sources >> >> * Verified pom file versions >> >> * Went over NOTICE file/pom files changes once more without finding >> >> anything suspicious >> >> * build Flink 1.17.0 (which should have been probably rather master 8) >> ) >> >> with flink-shaded 17.0 without issues >> >> * run Flink session cluster using the binaries from the previously >> >> mentioned build and deployed WordCount example job without issues >> >> >> >> On Thu, Apr 20, 2023 at 2:26 PM Martijn Visser < >> martijnvis...@apache.org> >> >> wrote: >> >> >> >>> Hi everyone, >> >>> >> >>> Please review and vote on the release candidate #1 for the version >> 17.0, >> >> as >> >>> follows: >> >>> [ ] +1, Approve the release >> >>> [ ] -1, Do not approve the release (please provide specific comments) >> >>> >> >>> The complete staging area is available for your review, which >> includes: >> >>> * JIRA release notes [1], >> >>> * the official Apache source release to be deployed to >> dist.apache.org >> >>> [2], >> >>> which are signed with the key with fingerprint >> >>> A5F3BCE4CBE993573EC5966A65321B8382B219AF [3], >> >>> * all artifacts to be deployed to the Maven Central Repository [4], >> >>> * source code tag "release-17.0-rc1" [5], >> >>> * website pull request listing the new release [6]. >> >>> >> >>> The vote will be open for at least 72 hours. It is adopted by majority >> >>> approval, with at least 3 PMC affirmative votes. >> >>> >> >>> Thanks, >> >>> Release Manager >> >>> >> >>> [1] >> >>> >> >>> >> >> >> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522=12352445 >> >>> [2] >> https://dist.apache.org/repos/dist/dev/flink/flink-shaded-17.0-rc1 >> >>> [3] https://dist.apache.org/repos/dist/release/flink/KEYS >> >>> [4] >> >>> >> https://repository.apache.org/content/repositories/orgapacheflink-1628/ >> >>> [5] >> https://github.com/apache/flink-shaded/releases/tag/release-17.0-rc1 >> >>> [6] https://github.com/apache/flink-web/pull/645 >> >>> >> >> >> >>
Re: [VOTE] Release flink-shaded 17.0, release candidate #1
+1 (binding) - Validated hashes - Verified signature - Verified that no binaries exist in the source archive - Build the source with Maven - Verified licenses - Verified web PRs On Thu, May 4, 2023 at 6:37 AM Leonard Xu wrote: > +1 (binding) > > - built from source code succeeded > - verified signatures > - verified hashsums > - checked Github release tag > - checked release notes > - reviewed the web PR > - filed issue FLINK-31994[1] to update the copyright year, not a blocker > > Best, > Leonard > [1] https://issues.apache.org/jira/browse/FLINK-31994 > > > 2023年5月2日 下午8:48，Danny Cranmer 写道： > > > > +1 (binding) > > > > - Release notes look good > > - Source archive signature and checksums good > > - Verified there are no binaries in the source archive > > - Contents of Maven repo look good/match source build > > - Built from sourcepass > > - Tag is present in Github > > - Verified NOTICE files > > > > Observations > > - NOTICE file copyright still says 2021, not blocking but needs to be > > updated > > > > Thanks, > > Danny > > > > On Thu, Apr 20, 2023 at 5:02 PM Matthias Pohl > > wrote: > > > >> Thanks Martijn for driving the flink-shaded release. > >> > >> +1 (non-binding) > >> > >> * Downloaded artifact > >> * Verified SHA512 checksums > >> * GPG signatures Compared > >> * checkout with provided sources > >> * Verified pom file versions > >> * Went over NOTICE file/pom files changes once more without finding > >> anything suspicious > >> * build Flink 1.17.0 (which should have been probably rather master 8) ) > >> with flink-shaded 17.0 without issues > >> * run Flink session cluster using the binaries from the previously > >> mentioned build and deployed WordCount example job without issues > >> > >> On Thu, Apr 20, 2023 at 2:26 PM Martijn Visser < > martijnvis...@apache.org> > >> wrote: > >> > >>> Hi everyone, > >>> > >>> Please review and vote on the release candidate #1 for the version > 17.0, > >> as > >>> follows: > >>> [ ] +1, Approve the release > >>> [ ] -1, Do not approve the release (please provide specific comments) > >>> > >>> The complete staging area is available for your review, which includes: > >>> * JIRA release notes [1], > >>> * the official Apache source release to be deployed to dist.apache.org > >>> [2], > >>> which are signed with the key with fingerprint > >>> A5F3BCE4CBE993573EC5966A65321B8382B219AF [3], > >>> * all artifacts to be deployed to the Maven Central Repository [4], > >>> * source code tag "release-17.0-rc1" [5], > >>> * website pull request listing the new release [6]. > >>> > >>> The vote will be open for at least 72 hours. It is adopted by majority > >>> approval, with at least 3 PMC affirmative votes. > >>> > >>> Thanks, > >>> Release Manager > >>> > >>> [1] > >>> > >>> > >> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522=12352445 > >>> [2] https://dist.apache.org/repos/dist/dev/flink/flink-shaded-17.0-rc1 > >>> [3] https://dist.apache.org/repos/dist/release/flink/KEYS > >>> [4] > >>> > https://repository.apache.org/content/repositories/orgapacheflink-1628/ > >>> [5] > https://github.com/apache/flink-shaded/releases/tag/release-17.0-rc1 > >>> [6] https://github.com/apache/flink-web/pull/645 > >>> > >> > >
Re: [VOTE] Release flink-connector-pulsar, release candidate #1
(Apologies for the previous email, that's a copy & paste error from the email above where I had already voted) Thanks all. This vote is now closed, I'll announce the results in a separate thread. On Mon, May 8, 2023 at 2:07 PM Martijn Visser wrote: > +1 (binding) > > - Validated hashes > - Verified signature > - Verified that no binaries exist in the source archive > - Build the source with Maven > - Verified licenses > - Verified web PRs > > > On Thu, May 4, 2023 at 5:17 AM Leonard Xu wrote: > >> +1 (binding) >> >> - built from source code succeeded >> - verified signatures >> - verified hashsums >> - checked Github release tag >> - checked release notes >> - reviewed the web PR and left minor comment >> >> Best, >> Leonard >> >> > 2023年4月24日 下午5:13，Danny Cranmer 写道： >> > >> > +1 (binding) >> > >> > - Release notes look ok >> > - Verified signature and hashes of source archive >> > - Tag is present in Github >> > - Reviewed Web PR >> > - Contents of MAven repo look good >> > - No binaries in the source archive >> > - Built source with Maven >> > - Verified NOTICE files >> > >> > Some observations >> > - Nightly CI is failing build for release tag due to 1.18 >> incompatibility, >> > I cut a ticket for that >> https://issues.apache.org/jira/browse/FLINK-31918 >> > - NOTICE files need year updating to 2023 however this is not a release >> > blocker >> > >> > >> > >> > On Wed, Apr 19, 2023 at 7:50 PM Martijn Visser < >> martijnvis...@apache.org> >> > wrote: >> > >> >> @Yufan can you make sure that these changes are properly included in >> the >> >> closed Jira tickets, under "Release Notes" ? >> >> >> >> +1 (binding) >> >> >> >> - Validated hashes >> >> - Verified signature >> >> - Verified that no binaries exist in the source archive >> >> - Build the source with Maven >> >> - Verified licenses >> >> - Verified web PRs >> >> >> >> On Thu, Apr 13, 2023 at 4:32 PM Yufan Sheng wrote: >> >> >> >>> +1 from the developer of flink-connector-pulsar >> >>> >> >>> I just want to add the comments here for the newly introduced breaking >> >>> changes in 4.0.0 >> >>> >> >>> Breaking Changes >> >>> >> >>> Some classes and interfaces which are annotated with @PublicEvolving >> >>> have been changed in this release. And this may affect the end-users. >> >>> >> >>> Pulsar Sink >> >>> >> >>> 1. PulsarMessage should be created by using the >> >>> PulsarMessage.builder() method. The PulsarMessageBuilder can’t be >> >>> created directly like before. >> >>> 2. The route method in the TopicRouter interface should return a >> >>> TopicPartition instead of a topic name. >> >>> 3. All the static methods in PulsarSerializationSchema have been >> >>> removed. You should set them (Schema, SerializationSchema) by directly >> >>> using PulsarSinkBuilder.setSerializationSchema(). >> >>> >> >>> Pulsar Source >> >>> >> >>> 1. The subscription type setting has been removed from the connector. >> >>> All the connector’s subscriptions will be created in Exclusive type. >> >>> The checkpoint for Shared and Key_Shared subscriptions couldn’t be >> >>> used since this release. >> >>> 2. The setSubscriptionType method has been removed from the >> >>> PulsarSourceBuilder. >> >>> 3. The fromMessageTime method has been removed from the StartCursor. >> >>> Pulsar doesn’t support seeking from message time. >> >>> 4. The RangeGenerator interface removed the deprecated open method and >> >>> the keyShareMode method. Because we don’t support the Key_Shared >> >>> subscription now. The RangeGenerator is only used in Exclusive >> >>> subscription to filter the desired keys. >> >>> 5. TopicPartition only exposes two constructors with the >> >>> @PublicEvolving annotation now. >> >>> 6. All the static methods in PulsarDeserializationSchema have been >> >>> removed. You should set them (Schema, SerializationSchema) by directly >> >>> using PulsarSourceBuilder.setDeserializationSchema(). >> >>> 7. The first argument of the open method in >> >>> PulsarDeserializationSchema has been changed from >> >>> InitializationContext to a new PulsarInitializationContext interface. >> >>> >> >>> On Thu, Apr 13, 2023 at 4:58 PM Martijn Visser < >> martijnvis...@apache.org >> >>> >> >>> wrote: >> >> Hi everyone, >> Please review and vote on the release candidate #1 for the version >> >> 4.0.0, >> as follows: >> [ ] +1, Approve the release >> [ ] -1, Do not approve the release (please provide specific comments) >> >> >> The complete staging area is available for your review, which >> includes: >> * JIRA release notes [1], >> * the official Apache source release to be deployed to >> dist.apache.org >> >>> [2], >> which are signed with the key with fingerprint >> A5F3BCE4CBE993573EC5966A65321B8382B219AF [3], >> * all artifacts to be deployed to the Maven Central Repository [4], >> * source code tag v4.0.0-rc1 [5], >> * website pull request listing the new release [6]. >> >> The vote
Re: [VOTE] Release flink-connector-pulsar, release candidate #1
+1 (binding) - Validated hashes - Verified signature - Verified that no binaries exist in the source archive - Build the source with Maven - Verified licenses - Verified web PRs On Thu, May 4, 2023 at 5:17 AM Leonard Xu wrote: > +1 (binding) > > - built from source code succeeded > - verified signatures > - verified hashsums > - checked Github release tag > - checked release notes > - reviewed the web PR and left minor comment > > Best, > Leonard > > > 2023年4月24日 下午5:13，Danny Cranmer 写道： > > > > +1 (binding) > > > > - Release notes look ok > > - Verified signature and hashes of source archive > > - Tag is present in Github > > - Reviewed Web PR > > - Contents of MAven repo look good > > - No binaries in the source archive > > - Built source with Maven > > - Verified NOTICE files > > > > Some observations > > - Nightly CI is failing build for release tag due to 1.18 > incompatibility, > > I cut a ticket for that > https://issues.apache.org/jira/browse/FLINK-31918 > > - NOTICE files need year updating to 2023 however this is not a release > > blocker > > > > > > > > On Wed, Apr 19, 2023 at 7:50 PM Martijn Visser > > > wrote: > > > >> @Yufan can you make sure that these changes are properly included in the > >> closed Jira tickets, under "Release Notes" ? > >> > >> +1 (binding) > >> > >> - Validated hashes > >> - Verified signature > >> - Verified that no binaries exist in the source archive > >> - Build the source with Maven > >> - Verified licenses > >> - Verified web PRs > >> > >> On Thu, Apr 13, 2023 at 4:32 PM Yufan Sheng wrote: > >> > >>> +1 from the developer of flink-connector-pulsar > >>> > >>> I just want to add the comments here for the newly introduced breaking > >>> changes in 4.0.0 > >>> > >>> Breaking Changes > >>> > >>> Some classes and interfaces which are annotated with @PublicEvolving > >>> have been changed in this release. And this may affect the end-users. > >>> > >>> Pulsar Sink > >>> > >>> 1. PulsarMessage should be created by using the > >>> PulsarMessage.builder() method. The PulsarMessageBuilder can’t be > >>> created directly like before. > >>> 2. The route method in the TopicRouter interface should return a > >>> TopicPartition instead of a topic name. > >>> 3. All the static methods in PulsarSerializationSchema have been > >>> removed. You should set them (Schema, SerializationSchema) by directly > >>> using PulsarSinkBuilder.setSerializationSchema(). > >>> > >>> Pulsar Source > >>> > >>> 1. The subscription type setting has been removed from the connector. > >>> All the connector’s subscriptions will be created in Exclusive type. > >>> The checkpoint for Shared and Key_Shared subscriptions couldn’t be > >>> used since this release. > >>> 2. The setSubscriptionType method has been removed from the > >>> PulsarSourceBuilder. > >>> 3. The fromMessageTime method has been removed from the StartCursor. > >>> Pulsar doesn’t support seeking from message time. > >>> 4. The RangeGenerator interface removed the deprecated open method and > >>> the keyShareMode method. Because we don’t support the Key_Shared > >>> subscription now. The RangeGenerator is only used in Exclusive > >>> subscription to filter the desired keys. > >>> 5. TopicPartition only exposes two constructors with the > >>> @PublicEvolving annotation now. > >>> 6. All the static methods in PulsarDeserializationSchema have been > >>> removed. You should set them (Schema, SerializationSchema) by directly > >>> using PulsarSourceBuilder.setDeserializationSchema(). > >>> 7. The first argument of the open method in > >>> PulsarDeserializationSchema has been changed from > >>> InitializationContext to a new PulsarInitializationContext interface. > >>> > >>> On Thu, Apr 13, 2023 at 4:58 PM Martijn Visser < > martijnvis...@apache.org > >>> > >>> wrote: > > Hi everyone, > Please review and vote on the release candidate #1 for the version > >> 4.0.0, > as follows: > [ ] +1, Approve the release > [ ] -1, Do not approve the release (please provide specific comments) > > > The complete staging area is available for your review, which > includes: > * JIRA release notes [1], > * the official Apache source release to be deployed to > dist.apache.org > >>> [2], > which are signed with the key with fingerprint > A5F3BCE4CBE993573EC5966A65321B8382B219AF [3], > * all artifacts to be deployed to the Maven Central Repository [4], > * source code tag v4.0.0-rc1 [5], > * website pull request listing the new release [6]. > > The vote will be open for at least 72 hours. It is adopted by majority > approval, with at least 3 PMC affirmative votes. > > Thanks, > Release Manager > > [1] > > >>> > >> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522=12352653 > [2] > > >>> > >> > https://dist.apache.org/repos/dist/dev/flink/flink-connector-pulsar-4.0.0-rc1 > [3]
Re: [VOTE] Release flink-connector-opensearch, release candidate #1
Thanks all. This vote is now closed, I'll announce the results in a separate thread. On Thu, May 4, 2023 at 5:22 AM Leonard Xu wrote: > +1 (binding) > > - built from source code succeeded > - verified signatures > - verified hashsums > - checked Github release tag > - checked release notes > - reviewed the web PR > > Best, > Leonard > > > 2023年5月2日 下午3:14，Khanh Vu 写道： > > > > +1 (non-binding) > > > > - Verified sha512 checksum matches file archive. > > - Verified file archive is signed and signature is authorized. > > - Verified no binaries exist in the source archive. > > - Verified source archive is consistent with Github source code with > > tag v1.0.1-rc1, at commit c52dbf4fc9c473592479a6c4fc6b2b5227699737 > > - Source built successfully with maven. > > - Verified files contain license header by running apache-rat:check > (except > > 2 files: README.md & .editorconfig) > > - Executed end to end tests successfully for flink versions: 1.16.1, > 1.17.0 > > (failed for 1.15.4, I supposed the connector only works with Flink 1.16.x > > onwards) > > > > Suggestion: it would be nice if in README we're explicitly stating that > > since which Flink version the connector works with. > > > > Best regards, > > *Khanh Vu* > > > > > > On Wed, Apr 19, 2023 at 8:12 PM Martijn Visser > > > wrote: > > > >> +1 (binding) > >> > >> - Validated hashes > >> - Verified signature > >> - Verified that no binaries exist in the source archive > >> - Build the source with Maven > >> - Verified licenses > >> - Verified web PRs > >> > >> Note: the copyright year in the NOTICE files have been updated to 2023 > >> > >> On Wed, Apr 19, 2023 at 9:43 AM Danny Cranmer > >> wrote: > >> > >>> Thanks for driving, Martijn. > >>> > >>> +1 (binding) > >>> > >>> - Release notes look good > >>> - Verified source archive signature/checksums > >>> - Verified binary signature/checksums > >>> - Verified no binaries in the source archive > >>> - Source tag exists on GitHub > >>> - Reviewed web PR > >>> - Verified NOTICE files > >>> > >>> Observations > >>> - NOTICE files need the year updating to 2023 > >>> > >>> Thanks, > >>> Danny > >>> > >>> > >>> On Tue, Apr 18, 2023 at 5:19 PM Ryan Skraba > >>> > >>> wrote: > >>> > Hello! +1 (non-binding) > > I've validated the source for the RC1: > flink-connector-opensearch-1.0.1-src.tgz > * The sha512 checksum is OK. > * The source file is signed correctly. > * The signature A5F3BCE4CBE993573EC5966A65321B8382B219AF is found in > >> the > KEYS file, and on https://keys.openpgp.org > * The source file is consistent with the Github tag v1.0.1-rc1, which > corresponds to commit c52dbf4fc9c473592479a6c4fc6b2b5227699737 > - The files explicitly excluded by create_pristine_sources (such as > .gitignore and the submodule tools/releasing/shared) are not present. > * Has a LICENSE file and a NOTICE file. The sql-connector has a > NOTICE file for bundled artifacts. > * Does not contain any compiled binaries. > > * The sources can be compiled and tests pass with flink.version 1.17.0 > >>> and > flink.version 1.16.1 > > * Nexus has three staged artifact ids for 1.0.1-1.16 and 1.0.1-1.17 > - flink-connector-opensearch-parent (only .pom) > - flink-connector-opensearch (.jar, -sources.jar, -javadoc.jar, > >>> -tests.jar > and .pom) > - flink-sql-connector-opensearch (.jar, -sources.jar and .pom) > * All 18 files have been signed with the same key as above, and have > correct sha1 and md5 checksums. > > I didn't run any additional smoke tests other than the integration > test > cases. > > A couple minor points, but nothing that would block this release. > > - like the other connectors I've checked, flink.version in the parent > >> pom > is set to 1.16.0 even for 1.17 artifacts, which might be confusing. > - the NOTICE files have the wrong year. > - unlike other connectors, flink-connector-opensearch publishes the > -tests.jar classifier to nexus. Is this desired? > - The sql-connector PackagingITCase test fails when using > `-Prelease,docs-and-source`, but otherwise works as intended. > > All my best and thanks for the release. > > Ryan > > On Thu, Apr 13, 2023 at 3:39 PM Andrey Redko > wrote: > > > +1 (non-binding), thanks Martijn! > > > > Best Regards, > >Andriy Redko > > > > On Thu, Apr 13, 2023, 8:54 AM Martijn Visser < > >> martijnvis...@apache.org > > > wrote: > > > >> Hi everyone, > >> Please review and vote on the release candidate #1 for the version > 1.0.1, > >> as follows: > >> [ ] +1, Approve the release > >> [ ] -1, Do not approve the release (please provide specific > >> comments) > >> > >> > >> The complete staging area is available for your review, which > >>> includes: > >> * JIRA release
Re: [VOTE] Release flink-connector-elasticsearch, release candidate #1
Thanks all. This vote is now closed, I'll announce the results in a separate thread. On Thu, May 4, 2023 at 6:06 AM Leonard Xu wrote: > +1 (binding) > > - built from source code succeeded > - verified signatures > - verified hashsums > - checked Github release tag > - checked release notes > - checked the contents of maven repo > - reviewed the web PR > > Best, > Leonard > > > 2023年4月21日 上午1:07，Ahmed Hamdy 写道： > > > > + (non-binding) > > - Verified signature and hashes are correct. > > - Verified there are no binaries in the source archive > > - Verified tag in github > > - reviewed and approved web PR > > - build source successfully > > > > Best Regards > > Ahmed > > > > On Thu, 20 Apr 2023 at 16:50, Danny Cranmer > wrote: > > > >> +1 (binding) > >> > >> - Release notes look ok > >> - Verified signature/hashes of source archive > >> - Verified there are no binaries in the source archive > >> - Contents of Maven repo look good > >> - Verified NOTICE files > >> - Tag exists in Github > >> - Reviewed web PR > >> > >> Not a blocker > >> - Copyright year in NOTICE file needs to be updated to 2023 > >> > >> On Wed, Apr 19, 2023 at 7:28 PM Martijn Visser < > martijnvis...@apache.org> > >> wrote: > >> > >>> +1 (binding) > >>> > >>> - Validated hashes > >>> - Verified signature > >>> - Verified that no binaries exist in the source archive > >>> - Build the source with Maven > >>> - Verified licenses > >>> - Verified web PRs > >>> > >>> On Thu, Apr 13, 2023 at 1:51 PM Andriy Redko wrote: > >>> > +1 (non-binding), thank you Martijn! > > MV> Hi everyone, > MV> Please review and vote on the release candidate #1 for the version > 3.0.1, > MV> as follows: > MV> [ ] +1, Approve the release > MV> [ ] -1, Do not approve the release (please provide specific > >> comments) > > MV> This version supports both Flink 1.16.x and Flink 1.17.x > > MV> The complete staging area is available for your review, which > >>> includes: > MV> * JIRA release notes [1], > MV> * the official Apache source release to be deployed to > >>> dist.apache.org > [2], > MV> which are signed with the key with fingerprint > MV> A5F3BCE4CBE993573EC5966A65321B8382B219AF [3], > MV> * all artifacts to be deployed to the Maven Central Repository > [4], > MV> * source code tag v3.0.1-rc1 [5], > MV> * website pull request listing the new release [6]. > > MV> The vote will be open for at least 72 hours. It is adopted by > >>> majority > MV> approval, with at least 3 PMC affirmative votes. > > MV> Thanks, > MV> Release Manager > > MV> [1] > MV> > > >>> > >> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522=12352521 > MV> [2] > MV> > > >>> > >> > https://dist.apache.org/repos/dist/dev/flink/flink-connector-elasticsearch-3.0.1-rc1 > MV> [3] https://dist.apache.org/repos/dist/release/flink/KEYS > MV> [4] > > >> https://repository.apache.org/content/repositories/orgapacheflink-1609/ > MV> [5] > MV> > > >>> > >> > https://github.com/apache/flink-connector-elasticsearch/releases/tag/v3.0.1-rc1 > MV> [6] https://github.com/apache/flink-web/pull/634 > > > >>> > >> > >
Re: [VOTE] Release flink-connector-rabbitmq v3.0.1, release candidate #1
Thanks all. This vote is now closed, I'll announce the results in a separate thread. On Sat, May 6, 2023 at 4:45 PM Jing Ge wrote: > +1 > > - built the source > - verified signature > - verified hash > - contains no compiled binaries > - checked tag > - web PR looks good > > Best regards, > Jing > > > On Fri, May 5, 2023 at 9:14 AM Khanh Vu wrote: > > > Sorry, the above report is supposed for flink-connector-gpc-pubsub-3.0.1 > > > > - > > Here is the check for this one: > > > > +1 (non-binding) > > > > - Verified sha512 checksum matches file archive. > > - Verified file archive is signed and signature is authenticated. > > - Verified no binaries exist in the source archive. > > - Verified source archive is consistent with Github source code with tag > > v3.0.1-rc1, at commit 9827e71662c8f155cda5efe5ebbac804fd0fd8e2 > > - Source built successfully with maven. > > - (No end to end tests run for this connector) > > > > Best regards, > > Khanh Vu > > > > > > On Fri, May 5, 2023 at 7:55 AM Khanh Vu wrote: > > > > > +1 (non-binding) > > > > > > - Verified sha512 checksum matches file archive. > > > - Verified file archive is signed and signature is authorized. > > > - Verified no binaries exist in the source archive. > > > - Verified source archive is consistent with Github source code with > > > tag v3.0.1-rc1, at commit 73e56edb2aa4513f6a73dc071545fb2508fd2d44 > > > - Source built successfully with maven. > > > - Executed end to end tests successfully for flink versions: 1.15.4, > > > 1.16.1, 1.17.0 > > > > > > Best regards, > > > Khanh Vu > > > > > > > > > On Thu, May 4, 2023 at 3:47 AM Leonard Xu wrote: > > > > > >> > > >> +1 (binding) > > >> > > >> - built from source code succeeded > > >> - verified signatures > > >> - verified hashsums > > >> - checked Github release tag > > >> - reviewed the web PR > > >> > > >> Best, > > >> Leonard > > >> > > >> > > >> 2023年4月20日 上午3:29，Martijn Visser 写道： > > >> > > >> +1 (binding) > > >> > > >> - Validated hashes > > >> - Verified signature > > >> - Verified that no binaries exist in the source archive > > >> - Build the source with Maven > > >> - Verified licenses > > >> - Verified web PRs > > >> > > >> On Mon, Apr 17, 2023 at 7:00 PM Ryan Skraba > > > >> > > > >> wrote: > > >> > > >> Hello! +1 (non-binding) > > >> > > >> I've validated the source for the RC1: > > >> flink-connector-rabbitmq-3.0.1-src.tgz > > >> * The sha512 checksum is OK. > > >> * The source file is signed correctly. > > >> * The signature A5F3BCE4CBE993573EC5966A65321B8382B219AF is found in > the > > >> KEYS file, and on https://keys.openpgp.org > > >> * The source file is consistent with the Github tag v3.0.1-rc1, which > > >> corresponds to commit 9827e71662c8f155cda5efe5ebbac804fd0fd8e2 > > >> - The files explicitly excluded by create_pristine_sources (such as > > >> .gitignore and the submodule tools/releasing/shared) are not present. > > >> * Has a LICENSE file and a NOTICE file. The sql-connector has a > > >> NOTICE file for bundled artifacts. > > >> * Does not contain any compiled binaries. > > >> > > >> * The sources can be compiled and tests pass with flink.version 1.17.0 > > and > > >> flink.version 1.16.1 > > >> > > >> * Nexus has three staged artifact ids for 3.0.1-1.16 and 3.0.1-1.17 > > >> - flink-connector-rabbitmq-parent (only .pom) > > >> - flink-connector-rabbitmq (.jar, -sources.jar, -javadoc.jar and .pom) > > >> - flink-sql-connector-rabbitmq (.jar, -sources.jar and .pom) > > >> * All 16 files have been signed with the same key as above, and have > > >> correct sha1 and md5 checksums. > > >> > > >> I didn't run any additional smoke tests other than the integration > test > > >> cases. > > >> > > >> A couple minor points, but nothing that would block this release. > > >> > > >> - like flink-connector-gcp-pubsub-parent, the > > >> flink-connector-rabbitmq-parent:3.0.1-1.17 pom artifact has the > > >> flink.version set to 1.16.0, which might be confusing. > > >> - the NOTICE file for sql-connector has the wrong year. > > >> > > >> All my best and thanks for the release. > > >> > > >> Ryan > > >> > > >> > > >> On Thu, Apr 13, 2023 at 4:45 PM Martijn Visser < > > martijnvis...@apache.org> > > >> wrote: > > >> > > >> Hi everyone, > > >> Please review and vote on the release candidate #1 for the version > > 3.0.1, > > >> as follows: > > >> [ ] +1, Approve the release > > >> [ ] -1, Do not approve the release (please provide specific comments) > > >> > > >> This version is compatible with Flink 1.16.x and Flink 1.17.x > > >> > > >> The complete staging area is available for your review, which > includes: > > >> * JIRA release notes [1], > > >> * the official Apache source release to be deployed to > dist.apache.org > > >> [2], > > >> which are signed with the key with fingerprint > > >> A5F3BCE4CBE993573EC5966A65321B8382B219AF [3], > > >> * all artifacts to be deployed to the Maven Central Repository [4], > > >> * source code tag v3.0.1-rc1 [5], > > >> *
[jira] [Created] (FLINK-32030) SQL Client gateway mode should accept URLs
Alexander Fedulov created FLINK-32030:
-
Summary: SQL Client gateway mode should accept URLs
Key: FLINK-32030
URL: https://issues.apache.org/jira/browse/FLINK-32030
Project: Flink
Issue Type: Improvement
Components: Table SQL / Client, Table SQL / Gateway
Affects Versions: 1.17.0
Reporter: Alexander Fedulov
Assignee: Alexander Fedulov
Currently, the _--endpoint_ parameter has to be specified in the
_InetSocketAddress_ format, i.e. _hostname:port._ While this works fine for
basic use cases, it does not support the placement of the gateway behind a
proxy or using an Ingress for routing to a specific Flink cluster based on the
URL path. I.e. it expects
_[some.hostname.com:9001|http://some.hostname.com:9001/]_ to directly serve
requests on _[some.hostname.com:9001/v1|http://some.hostname.com:9001/v1]_ .
Mapping to a non-root location, i.e.
_[some.hostname.com:9001/flink-clusters/sql-preview-cluster-1/v1|http://some.hostname.com:9001/flink-clusters/sql-preview-cluster-1/v1]_
is not supported.
Since the client talks to the gateway via its REST endpoint, the right format
for the _--endpoint_ parameter is {_}URL{_}, not _InetSocketAddress_ .
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (FLINK-32031) Flink GCP Connector having issues with Conscrypt library
Jayadeep Jayaraman created FLINK-32031:
--
Summary: Flink GCP Connector having issues with Conscrypt library
Key: FLINK-32031
URL: https://issues.apache.org/jira/browse/FLINK-32031
Project: Flink
Issue Type: Bug
Components: Connectors / Google Cloud PubSub
Reporter: Jayadeep Jayaraman
When using the current pubsub connector it is not using the latest libraries
bom due to which when the connector is used in Cloud Dataproc it is failing
with conscrypt related issues.
{code:java}
Caused by:
repackaged.io.grpc.netty.shaded.io.netty.channel.AbstractChannel$AnnotatedSocketException:
Network is unreachable: pubsub.googleapis.com/2607:f8b0:4001:c23:0:0:0:5f:443
Caused by: java.net.SocketException: Network is unreachable
at java.base/sun.nio.ch.Net.connect0(Native Method)
at java.base/sun.nio.ch.Net.connect(Net.java:483)
at java.base/sun.nio.ch.Net.connect(Net.java:472)
at
java.base/sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:692)
at
repackaged.io.grpc.netty.shaded.io.netty.util.internal.SocketUtils$3.run(SocketUtils.java:91)
at
repackaged.io.grpc.netty.shaded.io.netty.util.internal.SocketUtils$3.run(SocketUtils.java:88)
at java.base/java.security.AccessController.doPrivileged(Native Method)
at
repackaged.io.grpc.netty.shaded.io.netty.util.internal.SocketUtils.connect(SocketUtils.java:88)
at
repackaged.io.grpc.netty.shaded.io.netty.channel.socket.nio.NioSocketChannel.doConnect(NioSocketChannel.java:315)
at
repackaged.io.grpc.netty.shaded.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.connect(AbstractNioChannel.java:248)
at
repackaged.io.grpc.netty.shaded.io.netty.channel.DefaultChannelPipeline$HeadContext.connect(DefaultChannelPipeline.java:1342)
at
repackaged.io.grpc.netty.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeConnect(AbstractChannelHandlerContext.java:548)
at
repackaged.io.grpc.netty.shaded.io.netty.channel.AbstractChannelHandlerContext.connect(AbstractChannelHandlerContext.java:533)
at
repackaged.io.grpc.netty.shaded.io.netty.channel.ChannelDuplexHandler.connect(ChannelDuplexHandler.java:54)
at
repackaged.io.grpc.netty.shaded.io.grpc.netty.WriteBufferingAndExceptionHandler.connect(WriteBufferingAndExceptionHandler.java:150)
at
repackaged.io.grpc.netty.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeConnect(AbstractChannelHandlerContext.java:548)
at
repackaged.io.grpc.netty.shaded.io.netty.channel.AbstractChannelHandlerContext.access$1000(AbstractChannelHandlerContext.java:61)
at
repackaged.io.grpc.netty.shaded.io.netty.channel.AbstractChannelHandlerContext$9.run(AbstractChannelHandlerContext.java:538)
at
repackaged.io.grpc.netty.shaded.io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
at
repackaged.io.grpc.netty.shaded.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
at
repackaged.io.grpc.netty.shaded.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500)
at
repackaged.io.grpc.netty.shaded.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at
repackaged.io.grpc.netty.shaded.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at
repackaged.io.grpc.netty.shaded.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:829){code}
On disabling ipv6 getting the following error:-
{code:java}
Channel Pipeline: [SslHandler#0, ProtocolNegotiators$ClientTlsHandler#0,
WriteBufferingAndExceptionHandler#0, DefaultChannelPipeline$TailContext#0]
at io.grpc.stub.ClientCalls.toStatusRuntimeException(ClientCalls.java:244)
at io.grpc.stub.ClientCalls.getUnchecked(ClientCalls.java:225)
at io.grpc.stub.ClientCalls.blockingUnaryCall(ClientCalls.java:142)
at
com.google.pubsub.v1.SubscriberGrpc$SubscriberBlockingStub.pull(SubscriberGrpc.java:1641)
at
org.apache.flink.streaming.connectors.gcp.pubsub.BlockingGrpcPubSubSubscriber.pull(BlockingGrpcPubSubSubscriber.java:73)
at
org.apache.flink.streaming.connectors.gcp.pubsub.BlockingGrpcPubSubSubscriber.pull(BlockingGrpcPubSubSubscriber.java:77)
at
org.apache.flink.streaming.connectors.gcp.pubsub.BlockingGrpcPubSubSubscriber.pull(BlockingGrpcPubSubSubscriber.java:77)
at
org.apache.flink.streaming.connectors.gcp.pubsub.BlockingGrpcPubSubSubscriber.pull(BlockingGrpcPubSubSubscriber.java:77)
at
org.apache.flink.streaming.connectors.gcp.pubsub.BlockingGrpcPubSubSubscriber.pull(BlockingGrpcPubSubSubscriber.java:67)
at
org.apache.flink.streaming.connectors.gcp.pubsub.PubSubSource.run(PubSubSource.java:128)
at
org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:110)
at
[jira] [Created] (FLINK-32029) FutureUtils.handleUncaughtException swallows exceptions that are caused by the exception handler code
Matthias Pohl created FLINK-32029:
-
Summary: FutureUtils.handleUncaughtException swallows exceptions
that are caused by the exception handler code
Key: FLINK-32029
URL: https://issues.apache.org/jira/browse/FLINK-32029
Project: Flink
Issue Type: Bug
Components: Runtime / Coordination
Affects Versions: 1.16.1, 1.17.0, 1.18.0
Reporter: Matthias Pohl
I ran into an issue in FLINK-31773 where the passed exception handler relies on
the {{leaderContender}} field of the {{DefaultLeaderElectionService}}. This
field can become {{null}} with the changes of FLINK-31773. But the {{null}}
check was missed in the error handling code. This bug wasn't exposed because
{{FutureUtils.handleUncaughtException(..)}} expects the passed error handler
callback to be bug-free.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (FLINK-32028) Error handling for ElasticSearch sink
Tudor Plugaru created FLINK-32028: - Summary: Error handling for ElasticSearch sink Key: FLINK-32028 URL: https://issues.apache.org/jira/browse/FLINK-32028 Project: Flink Issue Type: Improvement Components: Connectors / ElasticSearch Affects Versions: 1.16.1 Reporter: Tudor Plugaru The deprecated ElasticsearchSink supports setting an error handler via a [public method |[https://github.com/apache/flink-connector-elasticsearch/blob/8f75d4e059c09b55cc3a44bab3e64330b1246d27/flink-connector-elasticsearch7/src/main/java/org/apache/flink/streaming/connectors/elasticsearch7/ElasticsearchSink.java#L216],] but the new sink, does not. Ideally there would be a way to handle ES specific exceptions and be able to skip items from being retried on ES indefinitely and not block entirely the pipeline. -- This message was sent by Atlassian Jira (v8.20.10#820010)
