[jira] [Created] (FLINK-31376) CSVReader for streaming does not support splittable

2023-03-08 Thread ramkrishna.s.vasudevan (Jira)
ramkrishna.s.vasudevan created FLINK-31376: -- Summary: CSVReader for streaming does not support splittable Key: FLINK-31376 URL: https://issues.apache.org/jira/browse/FLINK-31376 Project: Flink

[jira] [Created] (FLINK-31375) the values of map are truncated by the CASE WHEN function.

2023-03-08 Thread jeff-zou (Jira)
jeff-zou created FLINK-31375: Summary: the values of map are truncated by the CASE WHEN function. Key: FLINK-31375 URL: https://issues.apache.org/jira/browse/FLINK-31375 Project: Flink Issue Typ

[ANNOUNCE] Release 1.17.0, release candidate #1

2023-03-08 Thread Qingsheng Ren
Hi everyone, The RC1 for Apache Flink 1.17.0 has been created. This RC currently is for preview only to facilitate the integrated testing since the release announcement is still under review. The voting process will be triggered once the announcement is ready. It has all the artifacts that we woul

[Vote] FLIP-298: Unifying the Implementation of SlotManager

2023-03-08 Thread Weihua Hu
Hi Everyone, I would like to start the vote on FLIP-298: Unifying the Implementation of SlotManager [1]. The FLIP was discussed in this thread [2]. This FLIP aims to unify the implementation of SlotManager in order to reduce maintenance costs. The vote will last for at least 72 hours (03/14, 15:

Re: StreamingFileSink's DefaultRollingPolicy

2023-03-08 Thread ramkrishna vasudevan
Hi Keith, With default rolling policy, when you try to restore from the older save points, there is a high chance that the inProgressPart file would have been renamed to a newer file. So when we try to restore from older save points this might not work. The onCheckpointRolling policy might be the

Re: [DISCUSS] FLIP-301: Hybrid Shuffle supports Remote Storage

2023-03-08 Thread Weihua Hu
Hi, Yuxin Thanks for driving this FLIP. The remote storage shuffle could improve the stability of Batch jobs. In our internal scenario, we use a hybrid cluster to run both Streaming(high priority) and Batch jobs(low priority). When there is not enough resources(such as cpu usage reaches a thresh

Re: [DISCUSS] FLIP-301: Hybrid Shuffle supports Remote Storage

2023-03-08 Thread Yuxin Tan
Hi, Junrui, Thanks for the suggestions and ideas. > If they are fixed, I suggest that FLIP could provide clearer explanations. I have updated the FLIP and described the segment size more clearly. > can we provide configuration options for users to manually adjust the sizes? The segment size can b

[jira] [Created] (FLINK-31374) ProxyStreamPartitioner should implement ConfigurableStreamPartitioner

2023-03-08 Thread Zhipeng Zhang (Jira)
Zhipeng Zhang created FLINK-31374: - Summary: ProxyStreamPartitioner should implement ConfigurableStreamPartitioner Key: FLINK-31374 URL: https://issues.apache.org/jira/browse/FLINK-31374 Project: Flin

[jira] [Created] (FLINK-31373) PerRoundWrapperOperator should carry epoch information in watermark

2023-03-08 Thread Zhipeng Zhang (Jira)
Zhipeng Zhang created FLINK-31373: - Summary: PerRoundWrapperOperator should carry epoch information in watermark Key: FLINK-31373 URL: https://issues.apache.org/jira/browse/FLINK-31373 Project: Flink

[jira] [Created] (FLINK-31372) Memory Leak in HTTPMetricHandler when reporting fails

2023-03-08 Thread Krzysztof Dziolak (Jira)
Krzysztof Dziolak created FLINK-31372: - Summary: Memory Leak in HTTPMetricHandler when reporting fails Key: FLINK-31372 URL: https://issues.apache.org/jira/browse/FLINK-31372 Project: Flink

[jira] [Created] (FLINK-31371) Stream failure if the topic doesn't exist

2023-03-08 Thread Enzo Dechaene (Jira)
Enzo Dechaene created FLINK-31371: - Summary: Stream failure if the topic doesn't exist Key: FLINK-31371 URL: https://issues.apache.org/jira/browse/FLINK-31371 Project: Flink Issue Type: Bug

Re: [VOTE] FLIP-297: Improve Auxiliary Sql Statements

2023-03-08 Thread Sergey Nuyanzin
+1 (binding) i guess based on [1], please correct me if i am wrong [1] https://cwiki.apache.org/confluence/display/FLINK/Flink+Bylaws#FlinkBylaws-Actions On Wed, Mar 8, 2023 at 8:04 AM yuxia wrote: > +1 (non-binding) > Thanks Ran Tao for driving it. > > Best regards, > Yuxia > > - 原始邮件

[jira] [Created] (FLINK-31370) Cancellation of the StreamTask should prevent more timers from being fired

2023-03-08 Thread Jira
David Morávek created FLINK-31370: - Summary: Cancellation of the StreamTask should prevent more timers from being fired Key: FLINK-31370 URL: https://issues.apache.org/jira/browse/FLINK-31370 Project:

Re: [DISCUSS] FLIP-299 Pub/Sub Lite Connector

2023-03-08 Thread Etienne Chauchot
Hi, I agree with Ryan, even if clients might be totally different the backend technologies are the same so hosting them in the same repo makes sense. Similar thinking made us put all the Cassandra related connectors in the same cassandra repo. Etienne Le 02/03/2023 à 14:43, Daniel Collins a

Re: [DISCUSS] FLIP-300: Add targetColumns to DynamicTableSink#Context to solve the null overwrite problem of partial-insert

2023-03-08 Thread Lincoln Lee
Hi Jing, Agree with you that using formal terms can be easier to users, I've updated the FLIP[1], since this is only one of the application scenarios for partial insert, our java doc for the corresponding interface will describe the partial insert message itself from a generic point of view, WDTY?

Re: [DISCUSS] FLIP-300: Add targetColumns to DynamicTableSink#Context to solve the null overwrite problem of partial-insert

2023-03-08 Thread Jacky Lau
Thanks for bringing this up. this is a good feature. but i have two questions: 1. if the two insert into with same columns, the result is not nondeterminism. will it check in planner and throw exception 2. some sink connectors can not supports it like queue such as kafka compacted topic. will also

[jira] [Created] (FLINK-31369) Harden modifiers for sql-gateway module

2023-03-08 Thread Sergey Nuyanzin (Jira)
Sergey Nuyanzin created FLINK-31369: --- Summary: Harden modifiers for sql-gateway module Key: FLINK-31369 URL: https://issues.apache.org/jira/browse/FLINK-31369 Project: Flink Issue Type: Sub

Re: Regarding new command to download jars in flink cluster

2023-03-08 Thread Surendra Singh Lilhore
This command will help you download the user JARs without depending on the Filesystem CLI and init-container, as shown in the attached image. It handles the host path in Kubernetes, ensuring that only one process downloads the JARs while the other waits for it. This can also be used in a session cl

RE: Re: StreamingFileSink's DefaultRollingPolicy

2023-03-08 Thread Lee, Keith
Hi, > Can't `OnCheckpointRollingPolicy` meet your requirements? OnCheckpointRollingPolicy does not allow users to define rollover thresholds such as inactivity time, window size. However, that is beside the point of default behaviour, which I think should be thoughtfully considered for Flink us

Re: StreamingFileSink's DefaultRollingPolicy

2023-03-08 Thread Lee, Keith
Hi, > Can't `OnCheckpointRollingPolicy` meet your requirements? OnCheckpointRollingPolicy does not allow users to define rollover thresholds such as inactivity time, window size. However, that is beside the point of default behaviour, which I think should be thoughtfully considered for Flink us

[jira] [Created] (FLINK-31368) Move operation execution logic out from TableEnvironmentImpl

2023-03-08 Thread Jark Wu (Jira)
Jark Wu created FLINK-31368: --- Summary: Move operation execution logic out from TableEnvironmentImpl Key: FLINK-31368 URL: https://issues.apache.org/jira/browse/FLINK-31368 Project: Flink Issue Typ

[jira] [Created] (FLINK-31367) Support filter's function push down

2023-03-08 Thread tanjialiang (Jira)
tanjialiang created FLINK-31367: --- Summary: Support filter's function push down Key: FLINK-31367 URL: https://issues.apache.org/jira/browse/FLINK-31367 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-31366) Exception is thrown when s3a and s3p are used together

2023-03-08 Thread Hangxiang Yu (Jira)
Hangxiang Yu created FLINK-31366: Summary: Exception is thrown when s3a and s3p are used together Key: FLINK-31366 URL: https://issues.apache.org/jira/browse/FLINK-31366 Project: Flink Issue