[jira] [Created] (FLINK-30822) KafkaSinkITCase fails in setup and teardown method because the topic couldn't be created/deleted

Matthias Pohl created FLINK-30822: - Summary: KafkaSinkITCase fails in setup and teardown method because the topic couldn't be created/deleted Key: FLINK-30822 URL: https://issues.apache.org/jira/browse/FLINK-30822

Re: [VOTE] Externalize AWS formats to flink-connector-aws

+1 (non-binding) Thanks, Ahmed On Fri, 27 Jan 2023 at 17:24, Jing Ge wrote: > +1 Thanks for driving it! > > Best regards, > Jing > > On Fri, Jan 27, 2023 at 2:36 PM Danny Cranmer > wrote: > > > Hello Hong, thanks for driving this. > > > > +1 on the proposal and the voting schema looks good. > >

Re: [DISCUSS] FLIP-274 : Introduce metric group for OperatorCoordinator

I hadn't looked in detail in to how the task info can be introduced, but given that the coordinates are created by the execution graph, where we only work on tasks, it should be possible and rather simple. The OperatorCoordinatorHolder gets access to the ExecutionJobVertex from which it can ex

unsucscribe

unsucscribe

Re: [VOTE] Externalize AWS formats to flink-connector-aws

+1 (non-binding). Thanks for driving it. Best regards, Yuxia - 原始邮件 - 发件人: "Ahmed Hamdy" 收件人: "dev" 发送时间: 星期一, 2023年 1 月 30日 下午 4:01:14 主题: Re: [VOTE] Externalize AWS formats to flink-connector-aws +1 (non-binding) Thanks, Ahmed On Fri, 27 Jan 2023 at 17:24, Jing Ge wrote: > +1 Than

Re: unsucscribe

To unsubscribe, you can send email to dev-unsubscr...@flink.apache.org with any contents or subject. Please see more in the Flink Doc[1] [1] https://flink.apache.org/community.html#how-to-subscribe-to-a-mailing-list Best regards, Yuxia - 原始邮件 - 发件人: "Liu Haoyang" 收件人: "dev" 发送时间: 星期一,

Re: [VOTE] Externalize AWS formats to flink-connector-aws

+1 (binding) Op ma 30 jan. 2023 om 09:30 schreef yuxia > +1 (non-binding). > Thanks for driving it. > > Best regards, > Yuxia > > - 原始邮件 - > 发件人: "Ahmed Hamdy" > 收件人: "dev" > 发送时间: 星期一, 2023年 1 月 30日 下午 4:01:14 > 主题: Re: [VOTE] Externalize AWS formats to flink-connector-aws > > +1 (non

Re: [Discuss] Conventions on driving FLIPs towards consensus

Thanks Xintong for summarizing the PMC discussion here. I agree that working on trust instead of imposing more rules is the right thing to do. But I see Dong's point on documenting such an agreement in some way to give guidance to new contributors. One thing that I think might be helpful to includ

Re: [VOTE] Externalize AWS formats to flink-connector-aws

Sounds reasonable to me. +1 (binding) On Mon, Jan 30, 2023 at 9:52 AM Martijn Visser wrote: > +1 (binding) > > Op ma 30 jan. 2023 om 09:30 schreef yuxia > > > +1 (non-binding). > > Thanks for driving it. > > > > Best regards, > > Yuxia > > > > - 原始邮件 - > > 发件人: "Ahmed Hamdy" > > 收件人:

Re: [VOTE] Externalize AWS formats to flink-connector-aws

+1 (non-binding) Thanks for driving it. On Mon, Jan 30, 2023 at 2:53 PM Matthias Pohl wrote: > Sounds reasonable to me. > > +1 (binding) > > On Mon, Jan 30, 2023 at 9:52 AM Martijn Visser > wrote: > > > +1 (binding) > > > > Op ma 30 jan. 2023 om 09:30 schreef yuxia > > > > > +1 (non-binding).

Re: [VOTE] Externalize AWS formats to flink-connector-aws

+1 (non-binding) Thanks Hong for driving this. Regards, Daren On 30/01/2023, 09:32, "Samrat Deb" wrote: CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe. +1 (non-b

Re: [DISCUSS] FLIP-274 : Introduce metric group for OperatorCoordinator

Hi Chesnay, IIUC, our only disagreement is whether to support task info in the metric format. But we all agree with the default metric format "metrics.scope.jm-operator=.jobmanager.." which doesn't include task info, right? Regarding "coordinator metrics don't *belong to* any tasks/vertexes", I m

Re: [VOTE] Externalize AWS formats to flink-connector-aws

Sorry for coming in so late. This makes a lot of sense and I'd love to see it happen. However, the avro glue registry relies heavily on flink-avro interfaces which are currently not part of the public API. I think we should sort this out first before externalizing this format. To add on to tha

[jira] [Created] (FLINK-30823) Enable speculative execution for some of the typical built-in sinks

Biao Liu created FLINK-30823: Summary: Enable speculative execution for some of the typical built-in sinks Key: FLINK-30823 URL: https://issues.apache.org/jira/browse/FLINK-30823 Project: Flink

Re: [VOTE] FLIP-285: Refactoring LeaderElection to make Flink support multi-component leader election out-of-the-box

+1 non-binding Best Rui Fan On Thu, Jan 26, 2023 at 19:07 Chesnay Schepler wrote: > +1 > > On 25/01/2023 10:33, Matthias Pohl wrote: > > Hi everyone, > > After the discussion thread [1] on FLIP-285 [2] didn't bring up any new > > items, I want to start voting on FLIP-285. This FLIP will not on

[jira] [Created] (FLINK-30824) Add document for new introduced hive agg option

dalongliu created FLINK-30824: - Summary: Add document for new introduced hive agg option Key: FLINK-30824 URL: https://issues.apache.org/jira/browse/FLINK-30824 Project: Flink Issue Type: Sub-tas

[jira] [Created] (FLINK-30825) Ingress creation failed on K8s with version < 1.19

Shang Yuanchun created FLINK-30825: -- Summary: Ingress creation failed on K8s with version < 1.19 Key: FLINK-30825 URL: https://issues.apache.org/jira/browse/FLINK-30825 Project: Flink Issue

Re: [DISCUSS] FLIP-274 : Introduce metric group for OperatorCoordinator

The new option must default to the configured value of metrics.scope.jm.job because otherwise it isn't backwards-compatible with existing coordinator metrics. According to some earlier response in this thread there are already coordinator metrics. If that weren't the case you'd be correct, and

Re: [Discuss] Conventions on driving FLIPs towards consensus

Thanks all for the valuable information. It is a great step to make the FLIP running more smoothly. I'd like to share two additional suggestions: 1. Veto with care - Voter should present her-/him-self and give a veto with her/his own technical justification. No veto is allowed only based on someo

Re: [DISCUSS] Promote SinkV2 to @Public and deprecate SinkFunction

Hi Martijn, Thanks for driving this effort to clean-up the Flink codebase! I like the idea to cleanup Flink codebase to avoid having two Sinks. On the other hand, I also thing the concern mentioned by Jing makes sense. In addition to thinking in terms of the rule proposed in FLIP-197

Re: [VOTE] Externalize AWS formats to flink-connector-aws

Thanks Hong for driving this effort! +1 (binding) Regarding the concern mentioned by Chesnay, since apache/flink-connector-aws is managed by Flink community (not by Flink users) similar to the apache/flink, it should be OK for apache/flink-connector-aws to use non-public API, similar to how class

[jira] [Created] (FLINK-30826) HashAggITCase.testLeadLag is failing with jdk 11

Sergey Nuyanzin created FLINK-30826: --- Summary: HashAggITCase.testLeadLag is failing with jdk 11 Key: FLINK-30826 URL: https://issues.apache.org/jira/browse/FLINK-30826 Project: Flink Issue

[ANNOUNCE] FRocksDB 6.20.3-ververica-2.0 released

It is very happy to announce the release of FRocksDB 6.20.3-ververica-2.0. Compiled files for Linux x86, Linux arm, Linux ppc64le, MacOS x86, MacOS arm, and Windows are included in FRocksDB 6.20.3-ververica-2.0 jar, and the FRocksDB in Flink 1.17 would be updated to 6.20.3-ververica-2.0. Release

Re: [ANNOUNCE] FRocksDB 6.20.3-ververica-2.0 released

Hi Yanfei, Thanks for your effort. Looking forward to checking it. Best regards, Jing On Mon, Jan 30, 2023 at 1:42 PM Yanfei Lei wrote: > It is very happy to announce the release of FRocksDB 6.20.3-ververica-2.0. > > Compiled files for Linux x86, Linux arm, Linux ppc64le, MacOS x86, > MacOS ar

Re: [VOTE] Externalize AWS formats to flink-connector-aws

On 30/01/2023 13:24, Dong Lin wrote: it should be OK for apache/flink-connector-aws to use non-public API, similar to how classes inside Flink can use APIs marked with @Internal. We can mark related classes with @Internal as appropriate, without requiring any new FLIP. It's not though, because

[jira] [Created] (FLINK-30827) Improve the SQL Client code structure

Shengkai Fang created FLINK-30827: - Summary: Improve the SQL Client code structure Key: FLINK-30827 URL: https://issues.apache.org/jira/browse/FLINK-30827 Project: Flink Issue Type: Sub-task

Re: [VOTE] Release 1.16.1, release candidate #1

Hi everyone, Thank you all for your voting, the vote is now closed. I'll finalize the release and send out the announcement as soon as all the necessary steps have been completed. Best regards, Martijn Op ma 30 jan. 2023 om 07:49 schreef Jing Zhang : > Thanks Martijn for driving this. > +1 (no

[RESULT][VOTE] Release 1.16.1, release candidate #1

Hi all, I'm happy to announce that the release for Flink 1.16.1, release candidate #1 has been unanimously approved. Matthias (non-binding) Sergey (non-binding) Konstantin (binding) Dawid (binding) Gabor (non-binding) Hong (non-binding) Gyula (binding) ConradJam (non-binding) Jing Zhang (non-bind

[jira] [Created] (FLINK-30828) SortAggITCase.testLeadLag failed

Matthias Pohl created FLINK-30828: - Summary: SortAggITCase.testLeadLag failed Key: FLINK-30828 URL: https://issues.apache.org/jira/browse/FLINK-30828 Project: Flink Issue Type: Bug

Re: [RESULT][VOTE] Release 1.16.1, release candidate #1

Hi Martijn, Thanks for driving this! Could you roughly estimate when 1.16.1 could be officially released? Best regards, Jing On Mon, Jan 30, 2023 at 2:19 PM Martijn Visser wrote: > Hi all, > > I'm happy to announce that the release for Flink 1.16.1, release candidate > #1 has been unanimously

Re: [DISCUSS] Incorporate performance regression monitoring into routine process

Hi Yanfei, Thanks for driving the benchmark monitoring effort! The Google doc and the community wiki looks pretty good. According to Yuan's comment, it seems that we currently manually watch the benchmark results to detect regression. Have we considered automating this process by e.g. exporting t

[jira] [Created] (FLINK-30829) Make the backpressure tab could be sort by the backpressure level

Zhanghao Chen created FLINK-30829: - Summary: Make the backpressure tab could be sort by the backpressure level Key: FLINK-30829 URL: https://issues.apache.org/jira/browse/FLINK-30829 Project: Flink

Re: [DISCUSS] Incorporate performance regression monitoring into routine process

Hi Dong, The main issue with an automatic tool at the moment is that some benchmarks are quite noisy and performance regressions are often within the noise of a given benchmark. Our currently existing tooling can not handle those cases. Until we address this issue, I think it will have to remain a

[jira] [Created] (FLINK-30830) Temporal Table Function Samples for Java Do Not Work As-Is, And Some Functions Are Deprecated

Jeremy Ber created FLINK-30830: -- Summary: Temporal Table Function Samples for Java Do Not Work As-Is, And Some Functions Are Deprecated Key: FLINK-30830 URL: https://issues.apache.org/jira/browse/FLINK-30830

[jira] [Created] (FLINK-30831) Improving e2e test output in case errors/exceptions are found

Matthias Pohl created FLINK-30831: - Summary: Improving e2e test output in case errors/exceptions are found Key: FLINK-30831 URL: https://issues.apache.org/jira/browse/FLINK-30831 Project: Flink

[jira] [Created] (FLINK-30832) flink-kubernetes-operator helm chart - allow specifying operator pod resources requests/limits for flink-webhook container

Vincent Chenal created FLINK-30832: -- Summary: flink-kubernetes-operator helm chart - allow specifying operator pod resources requests/limits for flink-webhook container Key: FLINK-30832 URL: https://issues.apache

Re: [VOTE] Externalize AWS formats to flink-connector-aws

Hi @Chesnay @Dong Lin, Thanks for flagging up the concerns. I agree that we should make clear which APIs are public in flink-avro so that we can make the connector code more independent. IMO that moves in the right direction for Flink in general! 1. Public classes/interfaces in flink-avro -

Re: [DISCUSS] FLIP-264 Extract BaseCoordinatorContext

Let me start an initial discussion thread at dev@flink. Like to gauge the interests from the community (including Hudi and Delta Lake) first before spending time on writing up a big FLIP. On Fri, Jan 27, 2023 at 10:45 PM Jark Wu wrote: > Thank Steven for the explanation. > > It sounds good to me

[jira] [Created] (FLINK-30833) MetricListener supports closing metric groups

Mason Chen created FLINK-30833: -- Summary: MetricListener supports closing metric groups Key: FLINK-30833 URL: https://issues.apache.org/jira/browse/FLINK-30833 Project: Flink Issue Type: Improve

Re: [VOTE] Externalize AWS formats to flink-connector-aws

On Mon, Jan 30, 2023 at 8:51 PM Chesnay Schepler wrote: > On 30/01/2023 13:24, Dong Lin wrote: > > it should be OK for > > apache/flink-connector-aws to use non-public API, similar to how classes > > inside Flink can use APIs marked with @Internal. > > > > We can mark related classes with @Intern

Re: [VOTE] Externalize AWS formats to flink-connector-aws

Hi Hong, Thanks for the detailed reply. It seems that there are roughly 32 public classes in flink-avro that are not marked with @PublicEvolving. Some of them are explicitly marked as @Internal. Are you suggesting to mark all these classes as @PublicEvolving? I agree it would be nice to mark mor

[DISCUSS] streaming shuffle to improve data clustering and tame small files problem

Hi, We had a proposal to add a streaming shuffling stage in the Flink Iceberg sink to to improve data clustering and tame the small files problem [1]. Here are a couple of common use cases. * Event time partitioned table where we can get small files problem due to skewed and long-tail distributio

Re: [DISCUSS] streaming shuffle to improve data clustering and tame small files problem

Thank Steven, for starting this discussion. As I suggested in the previous thread, this can be a joint effort beneficial for various projects. I would also like to hear opinions from @Jingsong Li , who is maintaining Flink Table Store. Best, Jark On Tue, 31 Jan 2023 at 08:46, Steven Wu wrote:

Re: [DISCUSS] Incorporate performance regression monitoring into routine process

Hi Piotr, Yes, the challenge of developing such an automatic tool is indeed to handle noise and achieve a balance between false positive and false negative. It is great to know that we already have scripts that can access historical benchmark data and generate alerts. There is some heuristic that

Re: [Discuss] Conventions on driving FLIPs towards consensus

Thanks all for the feedback. @Dong, I fully agree that we should document the agreements on wiki. However, I'd hesitate about the MUST / SHOULD phrasing. I personally would consider these conventions, except for the one quoted from the ASF policy, as empirical inputs for individual judgements, w

Re: [VOTE] FLIP-285: Refactoring LeaderElection to make Flink support multi-component leader election out-of-the-box

+1 non-binding Best regards, Weijie Rui Fan <1996fan...@gmail.com> 于2023年1月30日周一 18:57写道： > +1 non-binding > > Best > > Rui Fan > > On Thu, Jan 26, 2023 at 19:07 Chesnay Schepler wrote: > > > +1 > > > > On 25/01/2023 10:33, Matthias Pohl wrote: > > > Hi everyone, > > > After the discussion th

Re: unsucscribe

Hi Jake, Please email dev-unsubscr...@flink.apache.org as describe here: https://flink.apache.org/community.html Best regards, Weijie Jake 于2023年1月29日周日 10:48写道： >

[jira] [Created] (FLINK-30834) SortAggITCase.testLeadLag is unstable in ALL_EXCHANGES_BLOCKING mode

Zhu Zhu created FLINK-30834: --- Summary: SortAggITCase.testLeadLag is unstable in ALL_EXCHANGES_BLOCKING mode Key: FLINK-30834 URL: https://issues.apache.org/jira/browse/FLINK-30834 Project: Flink I

Re: [VOTE] FLIP-285: Refactoring LeaderElection to make Flink support multi-component leader election out-of-the-box

+1 non-binding Matthias Pohl 于2023年1月25日周三 17:34写道： > Hi everyone, > After the discussion thread [1] on FLIP-285 [2] didn't bring up any new > items, I want to start voting on FLIP-285. This FLIP will not only align > the leader election code base again through FLINK-26522 [3]. I also plan to >

Re: [VOTE] FLIP-285: Refactoring LeaderElection to make Flink support multi-component leader election out-of-the-box

+1 (Binding) Best, Yang ConradJam 于2023年1月31日周二 12:09写道： > +1 non-binding > > Matthias Pohl 于2023年1月25日周三 17:34写道： > > > Hi everyone, > > After the discussion thread [1] on FLIP-285 [2] didn't bring up any new > > items, I want to start voting on FLIP-285. This FLIP will not only align > > the

[jira] [Created] (FLINK-30835) Add smoke tests for flink connector

Weijie Guo created FLINK-30835: -- Summary: Add smoke tests for flink connector Key: FLINK-30835 URL: https://issues.apache.org/jira/browse/FLINK-30835 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-30836) Release FRocksDB 6.20.3-ververica-2.0

Yanfei Lei created FLINK-30836: -- Summary: Release FRocksDB 6.20.3-ververica-2.0 Key: FLINK-30836 URL: https://issues.apache.org/jira/browse/FLINK-30836 Project: Flink Issue Type: Improvement

Re: [VOTE] FLIP-290: Operator state compression (FLINK-30113)

+1 (binding) Thanks for driving this Best regards, Weijie Yuan Mei 于2023年1月28日周六 13:37写道： > +1 binding > > Best > Yuan > > On Sat, Jan 21, 2023 at 12:49 AM Rui Fan <1996fan...@gmail.com> wrote: > > > +1 (no-binding) > > > > Best > > Rui Fan > > > > On Fri, Jan 20, 2023 at 10:46 PM ConradJam

[jira] [Created] (FLINK-30837) Remove dependency of MutableByteArrayInputStream from flink-avro-glue-schema-registry

Hong Liang Teoh created FLINK-30837: --- Summary: Remove dependency of MutableByteArrayInputStream from flink-avro-glue-schema-registry Key: FLINK-30837 URL: https://issues.apache.org/jira/browse/FLINK-30837

[jira] [Created] (FLINK-30838) Update documentation about the AdaptiveBatchScheduler

Junrui Li created FLINK-30838: - Summary: Update documentation about the AdaptiveBatchScheduler Key: FLINK-30838 URL: https://issues.apache.org/jira/browse/FLINK-30838 Project: Flink Issue Type: S

[jira] [Created] (FLINK-30839) Relaxed looping group pattern make the following strict contiguity relaxed too

Juntao Hu created FLINK-30839: - Summary: Relaxed looping group pattern make the following strict contiguity relaxed too Key: FLINK-30839 URL: https://issues.apache.org/jira/browse/FLINK-30839 Project: Fli

[jira] [Created] (FLINK-30840) Fix resource leak when creating Executor

Shengkai Fang created FLINK-30840: - Summary: Fix resource leak when creating Executor Key: FLINK-30840 URL: https://issues.apache.org/jira/browse/FLINK-30840 Project: Flink Issue Type: Sub-ta

[jira] [Created] (FLINK-30841) Incorrect calc merge in streaming

lincoln lee created FLINK-30841: --- Summary: Incorrect calc merge in streaming Key: FLINK-30841 URL: https://issues.apache.org/jira/browse/FLINK-30841 Project: Flink Issue Type: Bug Com

Re: [ANNOUNCE] FRocksDB 6.20.3-ververica-2.0 released

Thanks Yanfei for driving the release! Best Yuan On Mon, Jan 30, 2023 at 8:46 PM Jing Ge via user wrote: > Hi Yanfei, > > Thanks for your effort. Looking forward to checking it. > > Best regards, > Jing > > On Mon, Jan 30, 2023 at 1:42 PM Yanfei Lei wrote: > >> It is very happy to announce the

[jira] [Created] (FLINK-30842) Add document for sink supports speculative execution

Biao Liu created FLINK-30842: Summary: Add document for sink supports speculative execution Key: FLINK-30842 URL: https://issues.apache.org/jira/browse/FLINK-30842 Project: Flink Issue Type: Sub-

Re: [VOTE] FLIP-290: Operator state compression (FLINK-30113)

Thanks for participating in the vote discussion, and I think the vote result has already been announced in [1] [1] https://lists.apache.org/thread/gpc0x1qx7kb5qd48w8p32lxywp0665qr Best Yun Tang From: weijie guo Sent: Tuesday, January 31, 2023 14:17 To: dev@fli

[jira] [Created] (FLINK-30843) Add Notifier For Autoscaler

Gaurav Miglani created FLINK-30843: -- Summary: Add Notifier For Autoscaler Key: FLINK-30843 URL: https://issues.apache.org/jira/browse/FLINK-30843 Project: Flink Issue Type: Improvement

Re: [ANNOUNCE] FRocksDB 6.20.3-ververica-2.0 released

Thanks Yuanfei for driving the frocksdb release! Best Yun Tang From: Yuan Mei Sent: Tuesday, January 31, 2023 15:09 To: Jing Ge Cc: Yanfei Lei ; dev@flink.apache.org ; user ; user...@flink.apache.org Subject: Re: [ANNOUNCE] FRocksDB 6.20.3-ververica-2.0 releas

[jira] [Created] (FLINK-30844) TaskTest failed fatally with exit code 239 (FatalExitExceptionHandler)

Matthias Pohl created FLINK-30844: - Summary: TaskTest failed fatally with exit code 239 (FatalExitExceptionHandler) Key: FLINK-30844 URL: https://issues.apache.org/jira/browse/FLINK-30844 Project: Fli

[jira] [Created] (FLINK-30845) Params in jarURI end up in file name

Fabio Wanner created FLINK-30845: Summary: Params in jarURI end up in file name Key: FLINK-30845 URL: https://issues.apache.org/jira/browse/FLINK-30845 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-30846) SpeculativeSchedulerITCase.testSpeculativeExecutionOfInputFormatSource fails

Matthias Pohl created FLINK-30846: - Summary: SpeculativeSchedulerITCase.testSpeculativeExecutionOfInputFormatSource fails Key: FLINK-30846 URL: https://issues.apache.org/jira/browse/FLINK-30846 Projec