Re: [DISCUSS] FLIP-356: Support Nested Fields Filter Pushdown

2023-08-29 Thread Venkatakrishnan Sowrirajan
Sure, will reference this discussion to resume where we started as part of the flip to refactor SupportsProjectionPushDown. On Tue, Aug 29, 2023, 7:22 PM Jark Wu wrote: > I'm fine with this. `ReferenceExpression` and `SupportsProjectionPushDown` > can be another FLIP. However, could you

[jira] [Created] (FLINK-32993) Datagen connector produce unexpected Char type data

2023-08-29 Thread Yubin Li (Jira)
Yubin Li created FLINK-32993: Summary: Datagen connector produce unexpected Char type data Key: FLINK-32993 URL: https://issues.apache.org/jira/browse/FLINK-32993 Project: Flink Issue Type:

Re: [DISCUSS] FLIP-356: Support Nested Fields Filter Pushdown

2023-08-29 Thread Jark Wu
I'm fine with this. `ReferenceExpression` and `SupportsProjectionPushDown` can be another FLIP. However, could you summarize the design of this part in the future part of the FLIP? This can be easier to get started with in the future. Best, Jark On Wed, 30 Aug 2023 at 02:45, Venkatakrishnan

Re: [DISCUSS] FLIP-328: Allow source operators to determine isProcessingBacklog based on watermark lag

2023-08-29 Thread Xuannan Su
Hi Jing, Thank you for the suggestion. The definition of watermark lag is the same as the watermarkLag metric in FLIP-33[1]. More specifically, the watermark lag calculation is computed at the time when a watermark is emitted downstream in the following way: watermarkLag = CurrentTime -

Re: [DISCUSS] FLIP-356: Support Nested Fields Filter Pushdown

2023-08-29 Thread Venkatakrishnan Sowrirajan
Thanks Jark. Sounds good. One more thing, earlier in my summary I mentioned, Introduce a new *ReferenceExpression* (or *BaseReferenceExpression*) > abstract class which will be extended by both *FieldReferenceExpression* > and *NestedFieldReferenceExpression* (to be introduced as part of this >

[VOTE] FLIP-356: Support Nested Fields Filter Pushdown

2023-08-29 Thread Venkatakrishnan Sowrirajan
Hi everyone, Thank you all for your feedback on FLIP-356. I'd like to start a vote. Discussion thread: https://lists.apache.org/thread/686bhgwrrb4xmbfzlk60szwxos4z64t7 FLIP: https://cwiki.apache.org/confluence/display/FLINK/FLIP-356%3A+Support+Nested+Fields+Filter+Pushdown Regards Venkata

[jira] [Created] (FLINK-32991) Some metrics from autoscaler never get registered

2023-08-29 Thread Maximilian Michels (Jira)
Maximilian Michels created FLINK-32991: -- Summary: Some metrics from autoscaler never get registered Key: FLINK-32991 URL: https://issues.apache.org/jira/browse/FLINK-32991 Project: Flink

[jira] [Created] (FLINK-32992) Recommended parallelism metric is a duplicate of Parallelism metric

2023-08-29 Thread Maximilian Michels (Jira)
Maximilian Michels created FLINK-32992: -- Summary: Recommended parallelism metric is a duplicate of Parallelism metric Key: FLINK-32992 URL: https://issues.apache.org/jira/browse/FLINK-32992

Re: [DISCUSS] FLIP-356: Support Nested Fields Filter Pushdown

2023-08-29 Thread Jark Wu
Hi Venkata, Your summary looks good to me. +1 to start a vote. I think we don't need "inputIndex" in NestedFieldReferenceExpression. Actually, I think it is also not needed in FieldReferenceExpression, and we should try to remove it (another topic). The RexInputRef in Calcite also doesn't

[jira] [Created] (FLINK-32990) CREATE TABLE AS statement can't as Plan.translate function parameter

2023-08-29 Thread Licho Sun (Jira)
Licho Sun created FLINK-32990: - Summary: CREATE TABLE AS statement can't as Plan.translate function parameter Key: FLINK-32990 URL: https://issues.apache.org/jira/browse/FLINK-32990 Project: Flink

Re: [DISCUSS] FLIP-348: Support System Columns in SQL and Table API

2023-08-29 Thread Timo Walther
Thanks everyone for the positive feedback. I updated the FLIP with the proposed minimal solution: https://cwiki.apache.org/confluence/display/FLINK/FLIP-348%3A+Make+expanding+behavior+of+virtual+metadata+columns+configurable I also changed the title to not cause any confusion with the concept

[jira] [Created] (FLINK-32989) PyFlink wheel package build failed

2023-08-29 Thread Dian Fu (Jira)
Dian Fu created FLINK-32989: --- Summary: PyFlink wheel package build failed Key: FLINK-32989 URL: https://issues.apache.org/jira/browse/FLINK-32989 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-32988) HiveITCase failed due to TestContainer not coming up

2023-08-29 Thread Matthias Pohl (Jira)
Matthias Pohl created FLINK-32988: - Summary: HiveITCase failed due to TestContainer not coming up Key: FLINK-32988 URL: https://issues.apache.org/jira/browse/FLINK-32988 Project: Flink Issue

[jira] [Created] (FLINK-32987) BlobClientSslTest>BlobClientTest.testSocketTimeout expected SocketTimeoutException but identified SSLException

2023-08-29 Thread Matthias Pohl (Jira)
Matthias Pohl created FLINK-32987: - Summary: BlobClientSslTest>BlobClientTest.testSocketTimeout expected SocketTimeoutException but identified SSLException Key: FLINK-32987 URL:

Re: [DISCUSS] FLIP-307: Flink connector Redshift

2023-08-29 Thread Samrat Deb
Hi Martijn, I fully comprehend your apprehensions regarding AWS's reliance on JDBC and the resultant impact on the release cycle. However, an effective approach is to utilize the latest version of flink-connector-jdbc as a Maven dependency. In this arrangement, flink-connector-aws can be