[jira] [Created] (FLINK-32730) BatchExecMultipleInput may contain unsupported SourceTransformation

2023-08-01 Thread Weijie Guo (Jira)
Weijie Guo created FLINK-32730: -- Summary: BatchExecMultipleInput may contain unsupported SourceTransformation Key: FLINK-32730 URL: https://issues.apache.org/jira/browse/FLINK-32730 Project: Flink

Re: [ANNOUNCE] Flink 1.18 feature freeze

2023-08-01 Thread Jing Ge
Hi, Thanks for keeping us in the loop. +1 for merging these PRs. @Alex Please feel free to contact us for CR and PR merges. @Matthias the Pekko migration is an important task, since it is related to CVE issues. Thanks for bringing it to our attention. Best regards, Jing On Tue, Aug 1, 2023

Re: FLINK-20767 - Support for nested fields filter push down

2023-08-01 Thread Venkatakrishnan Sowrirajan
Thanks Becket for the suggestion. That makes sense. Let me try it out and get back to you. Regards Venkata krishnan On Tue, Aug 1, 2023 at 9:04 AM Becket Qin wrote: > This is a very useful feature in practice. > > It looks to me that the key issue here is that Flink ResolvedExpression > does

[jira] [Created] (FLINK-32729) allow create an initial deployment with suspend state

2023-08-01 Thread Zhenqiu Huang (Jira)
Zhenqiu Huang created FLINK-32729: - Summary: allow create an initial deployment with suspend state Key: FLINK-32729 URL: https://issues.apache.org/jira/browse/FLINK-32729 Project: Flink

[DISCUSS] addition of Flink Script Operators

2023-08-01 Thread Kevin Tseng
Hello Dev Mailing List, I have a feature proposition that I am interested to hear more and to gather feedback from everyone on the mailing list. (please view this email in monospace font when possible) _*Proposal*_* * Currently, when Flink DataStream API is to be utilized, every data

[jira] [Created] (FLINK-32728) Metrics are not reported in Python UDF (used inside FlinkSQL) when exception is raised

2023-08-01 Thread Elkhan Dadashov (Jira)
Elkhan Dadashov created FLINK-32728: --- Summary: Metrics are not reported in Python UDF (used inside FlinkSQL) when exception is raised Key: FLINK-32728 URL: https://issues.apache.org/jira/browse/FLINK-32728

Re: FLINK-20767 - Support for nested fields filter push down

2023-08-01 Thread Becket Qin
This is a very useful feature in practice. It looks to me that the key issue here is that Flink ResolvedExpression does not have necessary abstraction for nested field access. So the Calcite RexFieldAccess does not have a counterpart in the ResolvedExpression. The FieldReferenceExpression only

Re: [DISCUSS] FLIP-334 : Decoupling autoscaler and kubernetes

2023-08-01 Thread Rui Fan
Hi Max, Thanks for your quick response! > 1. Handle state in the AutoScalerEventHandler which will receive > all related scaling and metric collection events, and can keep > track of any state. If I understand correctly, you mean that updating state is just part of handling events, right? If

Re: FLINK-20767 - Support for nested fields filter push down

2023-08-01 Thread Venkatakrishnan Sowrirajan
Thanks for the response. Looking forward to your pointers. In the meanwhile, let me figure out how we can implement it. Will keep you posted. On Mon, Jul 31, 2023, 11:43 PM liu ron wrote: > Hi, Venkata > > Thanks for reporting this issue. Currently, Flink doesn't support nested > filter

[jira] [Created] (FLINK-32727) Adding capability of setting Attributes in Google Pub/Sub sink

2023-08-01 Thread Yinghai Zhao (Jira)
Yinghai Zhao created FLINK-32727: Summary: Adding capability of setting Attributes in Google Pub/Sub sink Key: FLINK-32727 URL: https://issues.apache.org/jira/browse/FLINK-32727 Project: Flink

Re: [DISCUSS] FLIP-334 : Decoupling autoscaler and kubernetes

2023-08-01 Thread Maximilian Michels
Hi Rui, Thanks for the proposal. I think it makes a lot of sense to decouple the autoscaler from Kubernetes-related dependencies. A couple of notes when I read the proposal: 1. You propose AutoScalerEventHandler, AutoScalerStateStore, AutoScalerStateStoreFactory, and AutoScalerEventHandler.

[SUMMARY] Flink 1.18 Release Sync 08/01/2023

2023-08-01 Thread Qingsheng Ren
Hi devs, I'd like to share some highlights about the 1.18 release sync on Aug 1, 2023. - Release syncs are switched to weekly after feature freeze. The link to the updated iCal calendar can be found at the top of the 1.18 release wiki page [1]. - There are 2 requests to merge into master after

Re: [ANNOUNCE] Flink 1.18 feature freeze

2023-08-01 Thread Qingsheng Ren
Thanks for letting us know, Matthias! As discussed in the release sync, +1 for merging these PRs. Best, Qingsheng On Tue, Aug 1, 2023 at 5:17 PM Matthias Pohl wrote: > I'm requesting to merge in FLINK-32098 [1]. It's a minor change that > reduces the amount of exists calls to S3 while

Re: [ANNOUNCE] Flink 1.18 feature freeze

2023-08-01 Thread Matthias Pohl
I'm requesting to merge in FLINK-32098 [1]. It's a minor change that reduces the amount of exists calls to S3 while submitting a job (which can be an expensive operation if the object actually doesn't exist but the corresponding bucket itself contains a lot of objects). The PR is reviewed and

[jira] [Created] (FLINK-32726) [Umbrella] Test Flink Release 1.18

2023-08-01 Thread Qingsheng Ren (Jira)
Qingsheng Ren created FLINK-32726: - Summary: [Umbrella] Test Flink Release 1.18 Key: FLINK-32726 URL: https://issues.apache.org/jira/browse/FLINK-32726 Project: Flink Issue Type: Bug

Re: [ANNOUNCE] Flink 1.18 feature freeze

2023-08-01 Thread Qingsheng Ren
Thanks for letting us know, Alexander and Leonard! I checked these PRs and the changes are trivial. +1 for merging them. Best, Qingsheng On Tue, Aug 1, 2023 at 12:14 AM Leonard Xu wrote: > Thank all Release Managers for driving the 1.18 release work! > > > - Deprecation works for 2.0 > > > >

[jira] [Created] (FLINK-32725) Add option to control writing of timestamp to Kafka topic in KafkaRecordSerializationSchema.builder

2023-08-01 Thread xiechenling (Jira)
xiechenling created FLINK-32725: --- Summary: Add option to control writing of timestamp to Kafka topic in KafkaRecordSerializationSchema.builder Key: FLINK-32725 URL: https://issues.apache.org/jira/browse/FLINK-32725

[jira] [Created] (FLINK-32724) Mark CEP API classes as Public / PublicEvolving

2023-08-01 Thread Dian Fu (Jira)
Dian Fu created FLINK-32724: --- Summary: Mark CEP API classes as Public / PublicEvolving Key: FLINK-32724 URL: https://issues.apache.org/jira/browse/FLINK-32724 Project: Flink Issue Type:

[DISCUSS] FLIP-334 : Decoupling autoscaler and kubernetes

2023-08-01 Thread Rui Fan
Hi all, I and Samrat(cc'ed) created the FLIP-334[1] to decoupling the autoscaler and kubernetes. Currently, the flink-autoscaler is tightly integrated with Kubernetes. There are compelling reasons to extend the use of flink-autoscaler to more types of Flink jobs: 1. With the recent merge of the

[jira] [Created] (FLINK-32723) FLIP-334 : Decoupling autoscaler and kubernetes

2023-08-01 Thread Rui Fan (Jira)
Rui Fan created FLINK-32723: --- Summary: FLIP-334 : Decoupling autoscaler and kubernetes Key: FLINK-32723 URL: https://issues.apache.org/jira/browse/FLINK-32723 Project: Flink Issue Type: New

[jira] [Created] (FLINK-32722) 239 exit code in flink-runtime

2023-08-01 Thread Matthias Pohl (Jira)
Matthias Pohl created FLINK-32722: - Summary: 239 exit code in flink-runtime Key: FLINK-32722 URL: https://issues.apache.org/jira/browse/FLINK-32722 Project: Flink Issue Type: Bug

Re: FLINK-20767 - Support for nested fields filter push down

2023-08-01 Thread liu ron
Hi, Venkata Thanks for reporting this issue. Currently, Flink doesn't support nested filter pushdown. I also think that this optimization would be useful, especially for jobs, which may need to read a lot of data from the parquet or orc file. We didn't move forward with this for some priority