Re: [ANNOUNCE] Apache Flink 1.13.0 released

2021-05-05 Thread Xintong Song
Thanks Dawid & Guowei as the release managers, and everyone who has contributed to this release. Thank you~ Xintong Song On Thu, May 6, 2021 at 9:51 AM Leonard Xu wrote: > Thanks Dawid & Guowei for the great work, thanks everyone involved. > > Best, > Leonard > > 在 2021年5月5日,17:12,Theo

[jira] [Created] (FLINK-22580) web404

2021-05-05 Thread dong (Jira)
dong created FLINK-22580: Summary: web404 Key: FLINK-22580 URL: https://issues.apache.org/jira/browse/FLINK-22580 Project: Flink Issue Type: Improvement Reporter: dong

[jira] [Created] (FLINK-22579) The description content of Kubernetes config Incomplete display

2021-05-05 Thread dong (Jira)
dong created FLINK-22579: Summary: The description content of Kubernetes config Incomplete display Key: FLINK-22579 URL: https://issues.apache.org/jira/browse/FLINK-22579 Project: Flink Issue Type:

Re: Escape # character in containerized.master.env

2021-05-05 Thread Yang Wang
I think you could also use the "flink run-application ... -Dcontainerized.master.env.FOO=foo#bar ..." to work around. Best, Yang Lukáš Drbal 于2021年5月4日周二 下午9:43写道: > I just workaround it by using k8 secret and map it via > kubernetes.env.secretKeyRef into flink containers. > > On Tue, May 4,

Re: Can we use ZK as HA for flink JM cluster(deployed in k8s standalone session mode)

2021-05-05 Thread Yun Gao
Hi, Logically using zk HA on k8s cluster is supported. Attaching files is not support by apache mail server, could you update the files into some file server and paste the connections ? Best, Yun -- Sender:bhagi@R

[jira] [Created] (FLINK-22578) deployment configuration page miss content navigate menu

2021-05-05 Thread Jira
谢波 created FLINK-22578: -- Summary: deployment configuration page miss content navigate menu Key: FLINK-22578 URL: https://issues.apache.org/jira/browse/FLINK-22578 Project: Flink Issue Type: Improvement

Re: [ANNOUNCE] Apache Flink 1.13.0 released

2021-05-05 Thread Leonard Xu
Thanks Dawid & Guowei for the great work, thanks everyone involved. Best, Leonard > 在 2021年5月5日,17:12,Theo Diefenthal 写道: > > Thanks for managing the release. +1. I like the focus on improving operations > with this version. > > Von: "Matthias Pohl" > An: "Etienne Chauchot" > CC: "dev" ,

Configmaps not deleted Kubernetes HA

2021-05-05 Thread Enrique
Hi all, I am deploying a Flink Cluster in session mode using Kubernetes HA and have seen it working with the different config maps for the dispatcher, restserver and resourcemanager. I also have configured storage for checkpointing and HA metadata. When I submit a job, I can see that a config

Re: [DISCUSS] Backport FLIP-27 Kafka source connector fixes with API change to release-1.12.

2021-05-05 Thread Arvid Heise
After looking a bit more into it, I'm also +1. Let's merge it soonish. On Wed, May 5, 2021 at 7:20 PM Thomas Weise wrote: > I opened the PR to backport the changes: > https://github.com/apache/flink/pull/15840 > > Without these fixes the new KafkaSource in 1.12 is near unusable. The most >

Re: [DISCUSS] Backport FLIP-27 Kafka source connector fixes with API change to release-1.12.

2021-05-05 Thread Thomas Weise
I opened the PR to backport the changes: https://github.com/apache/flink/pull/15840 Without these fixes the new KafkaSource in 1.12 is near unusable. The most obvious problem I ran into during testing was that checkpoints fail when consumption has not started for a split (easily reproduced with a

[jira] [Created] (FLINK-22577) KubernetesLeaderElectionAndRetrievalITCase is failing

2021-05-05 Thread Matthias (Jira)
Matthias created FLINK-22577: Summary: KubernetesLeaderElectionAndRetrievalITCase is failing Key: FLINK-22577 URL: https://issues.apache.org/jira/browse/FLINK-22577 Project: Flink Issue Type:

[jira] [Created] (FLINK-22576) Jira bot does not set the status to Open if it unassigns people

2021-05-05 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-22576: - Summary: Jira bot does not set the status to Open if it unassigns people Key: FLINK-22576 URL: https://issues.apache.org/jira/browse/FLINK-22576 Project: Flink

[jira] [Created] (FLINK-22575) Offer a more recent flink-shaded-hadoop-2-uber

2021-05-05 Thread Timo Walther (Jira)
Timo Walther created FLINK-22575: Summary: Offer a more recent flink-shaded-hadoop-2-uber Key: FLINK-22575 URL: https://issues.apache.org/jira/browse/FLINK-22575 Project: Flink Issue Type:

Re: Flink SQL Cdc with schema changing

2021-05-05 Thread Jark Wu
Hi Taher, Could you explain a bit more your use case and what do you expect Flink SQL to support? That could help us to better understand and plan the future roadmap. Best, Jark On Wed, 5 May 2021 at 19:42, Taher Koitawala wrote: > Thank you for the reply Jack Wu, however that does not

[jira] [Created] (FLINK-22574) Adaptive Scheduler: Can not cancel restarting job

2021-05-05 Thread Robert Metzger (Jira)
Robert Metzger created FLINK-22574: -- Summary: Adaptive Scheduler: Can not cancel restarting job Key: FLINK-22574 URL: https://issues.apache.org/jira/browse/FLINK-22574 Project: Flink Issue

Re: [DISCUSS] Add async backpressure support to FlinkKafkaProducer

2021-05-05 Thread Piotr Nowojski
no problem and I'm glad that it's solving your problem :) Piotrek śr., 5 maj 2021 o 08:58 Wenhao Ji napisał(a): > Thanks Piotr for your reply! > It is a nice solution! By restricting the buffer using these properties, I > think maxConcurrentRequests attribute is indeed not necessary anymore. >

Re: Flink SQL Cdc with schema changing

2021-05-05 Thread Taher Koitawala
Thank you for the reply Jack Wu, however that does not satisfy my requirements, my use case is to have something that supports a schema drift over avro format. Column addition and column datatype change both types of variations is what I am trying to solve for. Either way thanks for the help, much

Re: Presence of Jars in Flink reg security

2021-05-05 Thread Chesnay Schepler
One of these (plexus-utils) is afaik used by maven, so the scanner is potentially scanning the wrong thing. Or you are scanning all dependencies downloaded during the build of Flink, including everything used by various plugins of the build process & maven itself. On 5/5/2021 11:08 AM, Till

Re: Flink SQL Cdc with schema changing

2021-05-05 Thread Jark Wu
Hi Taher, Currently, Flink (SQL) CDC doesn't support automatic schema change and doesn't support to consume schema change events in source. But you can upgrade schema manually, for example, if you have a table with columns [a, b, c], you can define a flink table t1 with these 3 columns. When you

[jira] [Created] (FLINK-22573) AsyncIO can timeout elements after completion

2021-05-05 Thread Arvid Heise (Jira)
Arvid Heise created FLINK-22573: --- Summary: AsyncIO can timeout elements after completion Key: FLINK-22573 URL: https://issues.apache.org/jira/browse/FLINK-22573 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-22572) JsonFsStreamSinkITCase.testPart fails running in a timeout

2021-05-05 Thread Matthias (Jira)
Matthias created FLINK-22572: Summary: JsonFsStreamSinkITCase.testPart fails running in a timeout Key: FLINK-22572 URL: https://issues.apache.org/jira/browse/FLINK-22572 Project: Flink Issue

Re: Presence of Jars in Flink reg security

2021-05-05 Thread Till Rohrmann
Hi Prasanna, in the latest Flink version (1.13.0) I couldn't find these dependencies. Which version of Flink are you looking at? What you could check is whether one of these dependencies is contained in one of Flink's shaded dependencies [1]. [1] https://github.com/apache/flink-shaded Cheers,

[jira] [Created] (FLINK-22571) SavepointITCase.testSavepointForJobWithIteration fails with Timeout

2021-05-05 Thread Matthias (Jira)
Matthias created FLINK-22571: Summary: SavepointITCase.testSavepointForJobWithIteration fails with Timeout Key: FLINK-22571 URL: https://issues.apache.org/jira/browse/FLINK-22571 Project: Flink

[jira] [Created] (FLINK-22570) Reduce #notifications of Flink Jira Bot by combining API requests

2021-05-05 Thread Konstantin Knauf (Jira)
Konstantin Knauf created FLINK-22570: Summary: Reduce #notifications of Flink Jira Bot by combining API requests Key: FLINK-22570 URL: https://issues.apache.org/jira/browse/FLINK-22570 Project:

[jira] [Created] (FLINK-22568) RescalingITCase. testSavepointRescalingInPartitionedOperatorStateList fails with TimeoutException

2021-05-05 Thread Matthias (Jira)
Matthias created FLINK-22568: Summary: RescalingITCase. testSavepointRescalingInPartitionedOperatorStateList fails with TimeoutException Key: FLINK-22568 URL: https://issues.apache.org/jira/browse/FLINK-22568

[jira] [Created] (FLINK-22569) Limit #tickets touched by Flink Jira Bot per run

2021-05-05 Thread Konstantin Knauf (Jira)
Konstantin Knauf created FLINK-22569: Summary: Limit #tickets touched by Flink Jira Bot per run Key: FLINK-22569 URL: https://issues.apache.org/jira/browse/FLINK-22569 Project: Flink

[jira] [Created] (FLINK-22567) OrcFsStreamingSinkITCase crashes JVM

2021-05-05 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-22567: Summary: OrcFsStreamingSinkITCase crashes JVM Key: FLINK-22567 URL: https://issues.apache.org/jira/browse/FLINK-22567 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-22566) Running Kerberized YARN application on Docker test (default input) fails with no resources

2021-05-05 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-22566: Summary: Running Kerberized YARN application on Docker test (default input) fails with no resources Key: FLINK-22566 URL:

[jira] [Created] (FLINK-22565) Running Kerberized YARN application on Docker test (custom fs plugin) unstable

2021-05-05 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-22565: Summary: Running Kerberized YARN application on Docker test (custom fs plugin) unstable Key: FLINK-22565 URL: https://issues.apache.org/jira/browse/FLINK-22565

Re: [DISCUSS] Add async backpressure support to FlinkKafkaProducer

2021-05-05 Thread Wenhao Ji
Thanks Piotr for your reply! It is a nice solution! By restricting the buffer using these properties, I think maxConcurrentRequests attribute is indeed not necessary anymore. On Tue, May 4, 2021 at 11:52 PM Piotr Nowojski wrote: > Hi Wenhao, > > As far as I know this is different compared to

[jira] [Created] (FLINK-22564) ITCASE_KUBECONFIG is not exported

2021-05-05 Thread Matthias (Jira)
Matthias created FLINK-22564: Summary: ITCASE_KUBECONFIG is not exported Key: FLINK-22564 URL: https://issues.apache.org/jira/browse/FLINK-22564 Project: Flink Issue Type: Bug