[VOTE] Apache Flink CDC Release 3.1.0, release candidate #1

Hi everyone, Please review and vote on the release candidate #1 for the version 3.1.0 of Apache Flink CDC, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) **Release Overview** As an overview, the release consists of the following: a)

Re: [DISCUSS] FLIP-443: Interruptible watermark processing

Hi Piotrek, Thanks for this proposal. It looks like it will shorten the checkpoint duration, especially in the case of back pressure. +1 for it! I'd like to ask some questions to understand your thoughts more precisely. 1. Currently when AbstractStreamOperator or AbstractStreamOperatorV2

Re: Question around Flink's AdaptiveBatchScheduler

Hi Xia and Jinrui, Filed https://github.com/apache/flink/pull/24736 to address the above described issue. Please take a look whenever you can. Thanks Venkat On Thu, Apr 18, 2024 at 12:16 PM Venkatakrishnan Sowrirajan < vsowr...@asu.edu> wrote: > Filed

[jira] [Created] (FLINK-35272) Pipeline Transform job supports omitting / renaming calculation column

yux created FLINK-35272: --- Summary: Pipeline Transform job supports omitting / renaming calculation column Key: FLINK-35272 URL: https://issues.apache.org/jira/browse/FLINK-35272 Project: Flink Issue

[jira] [Created] (FLINK-35271) Add doc for syntax `describe job 'xxx'`

xuyang created FLINK-35271: -- Summary: Add doc for syntax `describe job 'xxx'` Key: FLINK-35271 URL: https://issues.apache.org/jira/browse/FLINK-35271 Project: Flink Issue Type: Sub-task

[Discuss] FLIP-452: Allow Skipping Invocation of Function Calls While Constant-folding

I'd like to start a discussion of FLIP-452: Allow Skipping Invocation of Function Calls While Constant-folding [1] This feature proposes adding a new method FunctionDefinition.allowConstantFolding() as part of the Flink Table/SQL API. This would be used to determine whether an expression

Re: [DISCUSS] FLIP-451: Refactor Async sink API

Hi Jeyhun, Thanks for your feedback. I agree the phrasing is a bit confusing, the main scope for FLIP-451 is limited to introducing timeout configuration and the new "ResultHandler" to Async Sink API. I will remove reference to FLIP-284 from the FLIP to disambiguate. Best Regards Ahmed Hamdy On

[jira] [Created] (FLINK-35270) Enrich information in logs, making it easier for debugging

Haifei Chen created FLINK-35270: --- Summary: Enrich information in logs, making it easier for debugging Key: FLINK-35270 URL: https://issues.apache.org/jira/browse/FLINK-35270 Project: Flink

Re: [DISCUSS] FLIP-451: Refactor Async sink API

Hi Ahmed, Thanks a lot for the FLIP. +1 for it. My main concern is that the boundary/scope of the two FLIPs (451 and 284) and their differentiation/overlap is unclear for me from the FLIP document. Could you please elaborate more on this? Regards, Jeyhun On Mon, Apr 29, 2024 at 4:13 PM Ahmed

[jira] [Created] (FLINK-35269) Fix logging level for errors in AWS connector sinks

Aleksandr Pilipenko created FLINK-35269: --- Summary: Fix logging level for errors in AWS connector sinks Key: FLINK-35269 URL: https://issues.apache.org/jira/browse/FLINK-35269 Project: Flink

[DISCUSS] FLIP-451: Refactor Async sink API

Hi all, I would like to start a discussion on FLIP-451[1] The proposal comes on encountering a couple of issues while working with implementers for Async Sink. The FLIP mainly proposes a new API similar to AsyncFunction and ResultFuture as well as introducing timeout handling for AsyncSink

[DISCUSS] FLIP-443: Interruptible watermark processing

Hi all, I would like to start a discussion on FLIP-443: Interruptible watermark processing. https://cwiki.apache.org/confluence/x/qgn9EQ This proposal tries to make Flink's subtask thread more responsive when processing watermarks/firing timers, and make those operations interruptible/break

Flink stateful functions and Agentic Architecture

I am a systems architect developing a POC concept for an AI product using Agentic Architecture with generative LLMs. It occurred to me that it could be possible to use Flink stateful functions to provide the event driven communications and execution environment for agents built with a framework

[jira] [Created] (FLINK-35268) Support TTL for Async State API

Hangxiang Yu created FLINK-35268: Summary: Support TTL for Async State API Key: FLINK-35268 URL: https://issues.apache.org/jira/browse/FLINK-35268 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-35267) Create documentation for FlinkStateSnapshot CR

Mate Czagany created FLINK-35267: Summary: Create documentation for FlinkStateSnapshot CR Key: FLINK-35267 URL: https://issues.apache.org/jira/browse/FLINK-35267 Project: Flink Issue Type:

[jira] [Created] (FLINK-35266) Add e2e tests for FlinkStateSnapshot CRs

Mate Czagany created FLINK-35266: Summary: Add e2e tests for FlinkStateSnapshot CRs Key: FLINK-35266 URL: https://issues.apache.org/jira/browse/FLINK-35266 Project: Flink Issue Type:

[jira] [Created] (FLINK-35265) Implement FlinkStateSnapshot custom resource

Mate Czagany created FLINK-35265: Summary: Implement FlinkStateSnapshot custom resource Key: FLINK-35265 URL: https://issues.apache.org/jira/browse/FLINK-35265 Project: Flink Issue Type:

[jira] [Created] (FLINK-35264) Flink CDC Pipeline transform rules do not take effect

yux created FLINK-35264: --- Summary: Flink CDC Pipeline transform rules do not take effect Key: FLINK-35264 URL: https://issues.apache.org/jira/browse/FLINK-35264 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-35263) FLIP-446: Kubernetes Operator State Snapshot CRD

Mate Czagany created FLINK-35263: Summary: FLIP-446: Kubernetes Operator State Snapshot CRD Key: FLINK-35263 URL: https://issues.apache.org/jira/browse/FLINK-35263 Project: Flink Issue Type:

[jira] [Created] (FLINK-35262) Bridge between AsyncKeyedStateBackend and AsyncExecutionController

Hangxiang Yu created FLINK-35262: Summary: Bridge between AsyncKeyedStateBackend and AsyncExecutionController Key: FLINK-35262 URL: https://issues.apache.org/jira/browse/FLINK-35262 Project: Flink

Re: [DISCUSSION] FLIP-450: Improve Runtime Configuration for Flink 2.0

Thanks for driving this effort, Xuannan. +1 for the proposed changes. Just one suggestion: Some of the proposed changes involve not solely changing the configuration options, but are bound to changing / removal of certain features. E.g., the removal of hash-blocking shuffle and legacy hybrid

Re: [DISCUSS] FLIP-XXX Support currentFetchEventTimeLag and processingLag metrics

Thanks Qingsheng for the patient guidance! And sorry to everyone for wasting your time. I suddenly realised that my implementation is wrong. In my implementation the FetchTime and EventTime do not come from the same record and I am making this stupid mistake :( . I need some time to think about