Re: FLINK-20767 - Support for nested fields filter push down

2023-08-06 Thread Venkatakrishnan Sowrirajan
(Sorry, I pressed send too early) Thanks for the help @zhengyunhon...@gmail.com. Agree on not changing the API as much as possible as well as wrt simplifying Projection pushdown with nested fields eventually as well. In terms of the code itself, currently I am trying to leverage the

Re: FLINK-20767 - Support for nested fields filter push down

2023-08-06 Thread Venkatakrishnan Sowrirajan
Thanks @zhengyunhon...@gmail.com Regards Venkata krishnan On Sun, Aug 6, 2023 at 6:16 PM yh z wrote: > Hi, Venkatakrishnan, > I think this is a very useful feature. I have been focusing on the > development of the flink-table-planner module recently, so if you need some > help, I can assist

[jira] [Created] (FLINK-32766) Support kafka catalog

2023-08-06 Thread melin (Jira)
melin created FLINK-32766: - Summary: Support kafka catalog Key: FLINK-32766 URL: https://issues.apache.org/jira/browse/FLINK-32766 Project: Flink Issue Type: New Feature Reporter: melin

[jira] [Created] (FLINK-32765) create view should reuse calcite tree

2023-08-06 Thread xiaogang zhou (Jira)
xiaogang zhou created FLINK-32765: - Summary: create view should reuse calcite tree Key: FLINK-32765 URL: https://issues.apache.org/jira/browse/FLINK-32765 Project: Flink Issue Type:

Re: [DISCUSS] FLIP-334 : Decoupling autoscaler and kubernetes

2023-08-06 Thread Rui Fan
Hi Ron: Thanks for the feedback! The goal is indeed to turn the autoscaler into a general tool that can support other resource management. Hi Max, Gyula: My proposed `AutoScalerStateStore` is similar to Map, it can really be improved. > public interface AutoScalerStateStore { > Map

[jira] [Created] (FLINK-32764) SlotManager supports pulling up all TaskManagers at initialization

2023-08-06 Thread xiangyu feng (Jira)
xiangyu feng created FLINK-32764: Summary: SlotManager supports pulling up all TaskManagers at initialization Key: FLINK-32764 URL: https://issues.apache.org/jira/browse/FLINK-32764 Project: Flink

Re: [DISCUSS] FLIP-334 : Decoupling autoscaler and kubernetes

2023-08-06 Thread liu ron
Hi, Rui Thanks for driving the FLIP. The tuning of streaming jobs by autoscaler is very important. Although the mainstream trend now is cloud-native, many companies still run their Flink jobs on Yarn for historical reasons. If we can decouple autoscaler from K8S and turn it into a common tool

Re: [ANNOUNCE] New Apache Flink Committer - Weihua Hu

2023-08-06 Thread Yuepeng Pan
Congratulations, Weihua! Best, Yuepeng Pan 在 2023-08-07 09:17:41,"yh z" 写道: >Congratulations, Weihua! > >Best, >Yunhong Zheng (Swuferhong) > >Runkang He 于2023年8月5日周六 21:34写道: > >> Congratulations, Weihua! >> >> Best, >> Runkang He >> >> Kelu Tao 于2023年8月4日周五 18:21写道: >> >> >

Re: Projection pushdown for Avro files seems to be buggy

2023-08-06 Thread Xingcan Cui
After rechecking it, I realized that some of my changes broke the expected schema passed to GenericDatumReader#getResolver. The logic in Flink codebase is okay and we should only read a portion of the Avro record. Thanks, Xingcan On Sun, Aug 6, 2023 at 2:31 PM liu ron wrote: > Hi, Xingcan > >

Re: [ANNOUNCE] New Apache Flink PMC Member - Matthias Pohl

2023-08-06 Thread Andriy Redko
Congrats Matthias, well deserved!! DC> Congrats Matthias! DC> Very well deserved, thankyou for your continuous, consistent contributions. DC> Welcome. DC> Thanks, DC> Danny DC> On Fri, Aug 4, 2023 at 9:30 AM Feng Jin wrote: >> Congratulations, Matthias! >> >> Best regards >> >> Feng >> >> On

Re: [ANNOUNCE] New Apache Flink Committer - Weihua Hu

2023-08-06 Thread yh z
Congratulations, Weihua! Best, Yunhong Zheng (Swuferhong) Runkang He 于2023年8月5日周六 21:34写道: > Congratulations, Weihua! > > Best, > Runkang He > > Kelu Tao 于2023年8月4日周五 18:21写道: > > > Congratulations! > > > > On 2023/08/04 08:35:49 Danny Cranmer wrote: > > > Congrats and welcome to the team,

Re: FLINK-20767 - Support for nested fields filter push down

2023-08-06 Thread yh z
Hi, Venkatakrishnan, I think this is a very useful feature. I have been focusing on the development of the flink-table-planner module recently, so if you need some help, I can assist you in completing the development of some sub-tasks or code review. Returning to the design itself, I think it's

Re: [VOTE] Release flink-connector-mongodb v1.0.2, release candidate 1

2023-08-06 Thread Ahmed Hamdy
Thanks Danny + 1 (non-binding) * Signature and checksums are matching. * Source Code builds locally. * Web PR looks good. Best Regards Ahmed Hamdy On Fri, 4 Aug 2023 at 17:44, Jiabao Sun wrote: > Thanks Danny, > > +1 (non-binding) > > - Build and compile the source code locally > - Tag

Re: Projection pushdown for Avro files seems to be buggy

2023-08-06 Thread liu ron
Hi, Xingcan After deep dive into the source code, I also think it is a bug. Best, Ron Xingcan Cui 于2023年8月5日周六 23:27写道: > Hi all, > > We tried to read some Avro files with the Flink SQL (1.16.1) and noticed > that the projection pushdown seems to be buggy. > > The Avro schema we used has 4