Re: [Re-DISCUSS] FLIP-202: Introduce ClickHouse Connector

2023-08-23 Thread Jing Ge
Hi Conrad, Thanks for driving it! +1 for starting a new round of discussion based on the input you wrote in the comment. Look forward to the new design. Best regards, Jing On Thu, Aug 24, 2023 at 8:00 AM ConradJam wrote: > Hi Community > > I want to re-initiate the discussion related to FLIP-2

[jira] [Created] (FLINK-32950) statsd reporter does not follow spec for counters

2023-08-23 Thread Paymahn (Jira)
Paymahn created FLINK-32950: --- Summary: statsd reporter does not follow spec for counters Key: FLINK-32950 URL: https://issues.apache.org/jira/browse/FLINK-32950 Project: Flink Issue Type: Bug

[Re-DISCUSS] FLIP-202: Introduce ClickHouse Connector

2023-08-23 Thread ConradJam
Hi Community I want to re-initiate the discussion related to FLIP-202. This is a discussion related to the Clickhouse connector. Some previous discussions are summarized in [1] ● We need an external code warehouse to store this part of code, as @Martijn Visser said flink-clickhouse-connector to c

Re: [Discuss] FLIP-355: Add parent dir of files to classpath using yarn.provided.lib.dirs

2023-08-23 Thread Yang Wang
+1 for this FLIP Maybe a FLIP is an overkill for this enhancement. Best, Yang Venkatakrishnan Sowrirajan 于2023年8月23日周三 01:44写道: > Thanks for the FLIP, Archit. > > This is definitely quite a useful addition to the *yarn.provided.lib.dirs* > . +1. > > IMO, except for the fact that *yarn.provide

Re: [DISCUSS] FLIP-334 : Decoupling autoscaler and kubernetes

2023-08-23 Thread Matt Wang
Sorry for the late reply, I still have a small question here: Does the interface function of handlerRecommendedParallelism in AutoScalerEventHandler conflict with handlerScalingFailure/handlerScalingReport (one of the handles the event of scale failure, and the other handles the event of scale succ

[jira] [Created] (FLINK-32949) Allow specifying the ServerSocket port for the collect function when accessing the TaskManager from the client.

2023-08-23 Thread JiaJian He (Jira)
JiaJian He created FLINK-32949: -- Summary: Allow specifying the ServerSocket port for the collect function when accessing the TaskManager from the client. Key: FLINK-32949 URL: https://issues.apache.org/jira/browse/FL

Re: [DISCUSS] FLIP-356: Support Nested Fields Filter Pushdown

2023-08-23 Thread Venkatakrishnan Sowrirajan
Becket and Jark, Deprecate all the other > methods except tryApplyFilters() and tryApplyProjections(). For *SupportsProjectionPushDown*, we still need a *supportsNestedProjections* API on the table source as some of the table sources might not be able to handle nested fields and therefore the Fl

Re: Flink 1.17.2 planned?

2023-08-23 Thread Jing Ge
Hi Christian, Thanks for your understanding. We will take a look at 1.17.2, once the 1.18 release is done. In the meantime, there might be someone in the community who volunteers to be the 1.17.2 release manager. You will see related email threads on the Dev. Stay tuned please :-) Best regards, J

[jira] [Created] (FLINK-32948) Minimize visibility of parameterized tests

2023-08-23 Thread Jiabao Sun (Jira)
Jiabao Sun created FLINK-32948: -- Summary: Minimize visibility of parameterized tests Key: FLINK-32948 URL: https://issues.apache.org/jira/browse/FLINK-32948 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-32947) Autoscaler standalone mode supports the metric report

2023-08-23 Thread Rui Fan (Jira)
Rui Fan created FLINK-32947: --- Summary: Autoscaler standalone mode supports the metric report Key: FLINK-32947 URL: https://issues.apache.org/jira/browse/FLINK-32947 Project: Flink Issue Type: New F

[DISCUSS][2.0] Deprecating Accumulator in favor of MetricsGroup

2023-08-23 Thread Matthias Pohl
Hi everyone, I was looking into serializing the ArchivedExecutionGraph for another FLIP and came across Accumulators [1] (don't mix that one up with the window accumulators of the Table/SQL API). Accumulators were introduced in Flink quite a while ago in Statosphere PR #340 [2]. I had a brief chat

Re: [ANNOUNCE] release-1.18 branch cut

2023-08-23 Thread Rui Fan
Thanks for the quick response! Best, Rui On Wed, Aug 23, 2023 at 9:46 PM Jing Ge wrote: > yes please > > On Wed, Aug 23, 2023 at 3:35 PM Rui Fan <1996fan...@gmail.com> wrote: > > > Hi Jing, > > > > Thanks for the effort and update! > > > > It means the PR of flink-1.19 can be merged to master b

Re: [ANNOUNCE] release-1.18 branch cut

2023-08-23 Thread Jing Ge
yes please On Wed, Aug 23, 2023 at 3:35 PM Rui Fan <1996fan...@gmail.com> wrote: > Hi Jing, > > Thanks for the effort and update! > > It means the PR of flink-1.19 can be merged to master branch, right? > > Best, > Rui > > On Wed, Aug 23, 2023 at 9:29 PM Jing Ge > wrote: > > > Hi devs, The relea

Re: [ANNOUNCE] release-1.18 branch cut

2023-08-23 Thread Rui Fan
Hi Jing, Thanks for the effort and update! It means the PR of flink-1.19 can be merged to master branch, right? Best, Rui On Wed, Aug 23, 2023 at 9:29 PM Jing Ge wrote: > Hi devs, The release-1.18 branch has been forked out from the master > branch, with commit ID cfa4a9c35563fd8a5973ec2f3525

[ANNOUNCE] release-1.18 branch cut

2023-08-23 Thread Jing Ge
Hi devs, The release-1.18 branch has been forked out from the master branch, with commit ID cfa4a9c35563fd8a5973ec2f35251190e365be14. The version on the master branch has been upgraded to 1.19-SNAPSHOT. From now on, for PRs that should be presented in 1.18.0, please make sure: Merge the PR into bot

Re: [DISCUSS] FLIP-319: Integrating with Kafka’s proper support for 2PC participation (KIP-939).

2023-08-23 Thread Gyula Fóra
Hi Gordon! Thank you for preparing the detailed FLIP, I think this is a huge improvement that enables the exactly-once Kafka sink in many environments / use-cases where this was previously unfeasible due to the limitations highlighted in the FLIP. Big +1 Cheers, Gyula On Fri, Aug 18, 2023 at 7:

[jira] [Created] (FLINK-32946) Start End of Life discussion thread for now outdated Flink minor version

2023-08-23 Thread Matthias Pohl (Jira)
Matthias Pohl created FLINK-32946: - Summary: Start End of Life discussion thread for now outdated Flink minor version Key: FLINK-32946 URL: https://issues.apache.org/jira/browse/FLINK-32946 Project: F

[jira] [Created] (FLINK-32945) NullPointerException when executing TopSpeedWindowing example with checkpointing enabled

2023-08-23 Thread Matthias Pohl (Jira)
Matthias Pohl created FLINK-32945: - Summary: NullPointerException when executing TopSpeedWindowing example with checkpointing enabled Key: FLINK-32945 URL: https://issues.apache.org/jira/browse/FLINK-32945

Re: [DISCUSS] FLIP-323: Support Attached Execution on Flink Application Completion for Batch Jobs

2023-08-23 Thread Becket Qin
Hi Weihua, Just want to clarify. "client.attached.after.submission" is going to be a pure client side configuration. On the cluster side, it is only "execution.shutdown-on-attached-exit" controlling whether the cluster will shutdown or not when an attached client is disconnected. In order to hono

[jira] [Created] (FLINK-32944) Caused by: org.apache.flink.api.common.InvalidProgramException: Table program cannot be compiled. This is a bug. Please file an issue.

2023-08-23 Thread Tan Kim (Jira)
Tan Kim created FLINK-32944: --- Summary: Caused by: org.apache.flink.api.common.InvalidProgramException: Table program cannot be compiled. This is a bug. Please file an issue. Key: FLINK-32944 URL: https://issues.apache.

Re: [DISCUSS] Update Flink Roadmap

2023-08-23 Thread Jing Ge
Thanks Jark, +1 for the OLAP :-) Best regards, Jing On Sun, Aug 20, 2023 at 5:04 PM Jark Wu wrote: > Hi all, > > I have addressed another bunch of comments on the Google doc (mainly about > the OLAP roadmap). > And I have opened a pull request for the website: > https://github.com/apache/flink-

[jira] [Created] (FLINK-32943) sometime run batch tasks concurrently, the tasks still in the initialization status

2023-08-23 Thread zhu (Jira)
zhu created FLINK-32943: --- Summary: sometime run batch tasks concurrently, the tasks still in the initialization status Key: FLINK-32943 URL: https://issues.apache.org/jira/browse/FLINK-32943 Project: Flink

Re: Heartbeat of TaskManager with id container

2023-08-23 Thread xiangyu feng
Hi Nagireddy, I'm not sure how you monitoring kafka lag. AFAIK, you can check the metadata of the topic in your Kafka cluster to see the actual lag by following command. ./kafka-consumer-groups.sh --bootstrap-server 192.168.0.107:39092 --group --describe This tool is provided with Kafka distri

Fwd: [Discussion] Slack Channel

2023-08-23 Thread Jing Ge
Hi devs, Thanks Giannis for your suggestion. It seems that the last email wasn't sent to the dev ML. It is also an interesting topic for devs and user-zh. Best regards, Jing -- Forwarded message - From: Giannis Polyzos Date: Tue, Aug 22, 2023 at 11:11 AM Subject: [Discussion] Sl