[jira] [Created] (FLINK-34224) ChangelogStorageMetricsTest.testAttemptsPerUpload(ChangelogStorageMetricsTest timed out

2024-01-23 Thread Matthias Pohl (Jira)
Matthias Pohl created FLINK-34224: - Summary: ChangelogStorageMetricsTest.testAttemptsPerUpload(ChangelogStorageMetricsTest timed out Key: FLINK-34224 URL: https://issues.apache.org/jira/browse/FLINK-34224

[jira] [Created] (FLINK-34223) Introduce a migration tool to transfer legacy config file to new config file

2024-01-23 Thread Junrui Li (Jira)
Junrui Li created FLINK-34223: - Summary: Introduce a migration tool to transfer legacy config file to new config file Key: FLINK-34223 URL: https://issues.apache.org/jira/browse/FLINK-34223 Project:

[jira] [Created] (FLINK-34222) Get minibatch join operator involved

2024-01-23 Thread Shuai Xu (Jira)
Shuai Xu created FLINK-34222: Summary: Get minibatch join operator involved Key: FLINK-34222 URL: https://issues.apache.org/jira/browse/FLINK-34222 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-34221) Introduce operator for minibatch join

2024-01-23 Thread Shuai Xu (Jira)
Shuai Xu created FLINK-34221: Summary: Introduce operator for minibatch join Key: FLINK-34221 URL: https://issues.apache.org/jira/browse/FLINK-34221 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-34220) introduce buffer bundle for minibatch join

2024-01-23 Thread Shuai Xu (Jira)
Shuai Xu created FLINK-34220: Summary: introduce buffer bundle for minibatch join Key: FLINK-34220 URL: https://issues.apache.org/jira/browse/FLINK-34220 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-34219) Introduce a new join operator to support minibatch

2024-01-23 Thread Shuai Xu (Jira)
Shuai Xu created FLINK-34219: Summary: Introduce a new join operator to support minibatch Key: FLINK-34219 URL: https://issues.apache.org/jira/browse/FLINK-34219 Project: Flink Issue Type: New

[RESULT][VOTE] FLIP-415: Introduce a new join operator to support minibatch

2024-01-23 Thread shuai xu
Hi devs, I'm glad to announce that the FLIP-415[1] has been accepted. The voting thread is here[2]. The proposal received 5 approving votes, 4 of which are binding: - Lincoln Lee (binding) - Ron liu (binding) - Jane Chan (binding) - Benchao Li (binding) - Xuyang (non-binding) And there is no

[jira] [Created] (FLINK-34218) AutoRescalingITCase#testCheckpointRescalingInKeyedState fails

2024-01-23 Thread Rui Fan (Jira)
Rui Fan created FLINK-34218: --- Summary: AutoRescalingITCase#testCheckpointRescalingInKeyedState fails Key: FLINK-34218 URL: https://issues.apache.org/jira/browse/FLINK-34218 Project: Flink Issue

Re: [DISCUSS] FLIP-417: Expose JobManagerOperatorMetrics via REST API

2024-01-23 Thread Mason Chen
Hi all, I hope that everyone has had sufficient time to review the discussion and the updates on the FLIP doc. If there are no objections, I'll start a voting thread in 2 days. Best, Mason On Thu, Jan 18, 2024 at 2:39 PM Mason Chen wrote: > Hi Lijie, > > That's also a possibility but I would

[jira] [Created] (FLINK-34217) Update Serialization-related doc with the new way of configuration

2024-01-23 Thread Zhanghao Chen (Jira)
Zhanghao Chen created FLINK-34217: - Summary: Update Serialization-related doc with the new way of configuration Key: FLINK-34217 URL: https://issues.apache.org/jira/browse/FLINK-34217 Project: Flink

[jira] [Created] (FLINK-34216) Support fine-grained configuration to control filter push down for MongoDB Connector

2024-01-23 Thread Jiabao Sun (Jira)
Jiabao Sun created FLINK-34216: -- Summary: Support fine-grained configuration to control filter push down for MongoDB Connector Key: FLINK-34216 URL: https://issues.apache.org/jira/browse/FLINK-34216

[jira] [Created] (FLINK-34215) Support fine-grained configuration to control filter push down for JDBC Connector

2024-01-23 Thread Jiabao Sun (Jira)
Jiabao Sun created FLINK-34215: -- Summary: Support fine-grained configuration to control filter push down for JDBC Connector Key: FLINK-34215 URL: https://issues.apache.org/jira/browse/FLINK-34215

[jira] [Created] (FLINK-34214) FLIP-377: Support fine-grained configuration to control filter push down for Table/SQL Sources

2024-01-23 Thread Jiabao Sun (Jira)
Jiabao Sun created FLINK-34214: -- Summary: FLIP-377: Support fine-grained configuration to control filter push down for Table/SQL Sources Key: FLINK-34214 URL: https://issues.apache.org/jira/browse/FLINK-34214

[RESULT][VOTE] FLIP-377: Support fine-grained configuration to control filter push down for Table/SQL Sources

2024-01-23 Thread Jiabao Sun
Hi devs, I'm glad to announce that the FLIP-377[1] has been accepted. The voting thread is here[2]. The proposal received 7 approving votes, three of which are binding: - Hang Ruan (non-binding) - Leonard Xu (binding) - Xuyang (non-binding) - Jark Wu (binding) - Qingsheng Ren (binding) - Feng

[VOTE] Release flink-connector-parent, release candidate #1

2024-01-23 Thread Etienne Chauchot
Hi everyone, Please review and vote on the release candidate #1 for the version 1.1.0, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) The complete staging area is available for your review, which includes: * JIRA release notes

FileNotFoundException for ".inprogess" files in StreamingFileSink

2024-01-23 Thread Y SREEKARA BHARGAVA REDDY
Hi Team, Flink Version is *1.10* *java.io.FileNotFoundException: File does not exist: /data/staging/inbound/_user/cdcdcd/input/aaa_/30e73444-3405-471f-a05a-b32fb15fddfd/.SEP705052768755-12-3643.done.inprogress.4f2bba7c-68fa-4044-86f9-444057aa45ac* at

Re: [DISCUSS] FLIP-406: Reorganize State & Checkpointing & Recovery Configuration

2024-01-23 Thread Zakelly Lan
Hi everyone, It seems too late to make this in 1.19, so I suggest changing it to 1.20. Another thing I'd like to highlight is that there are some existing option classes lacking annotations, which are: - org.apache.flink.configuration.CheckpointingOptions -

Re: [DISCUSS] FLIP-402: Extend ZooKeeper Curator configurations

2024-01-23 Thread Matthias Pohl
> > - Regarding names: sure it totally makes sense to follow the kebab case > and Flip has reflected the change. > Regarding the convention, Flink has this widely used configuration > storageDir, which doesn't follow the kebab rule and creates some confusion. > IMHO it would be valuable to add a

[jira] [Created] (FLINK-34213) Consider using accumulated busy time instead of busyMsPerSecond

2024-01-23 Thread Maximilian Michels (Jira)
Maximilian Michels created FLINK-34213: -- Summary: Consider using accumulated busy time instead of busyMsPerSecond Key: FLINK-34213 URL: https://issues.apache.org/jira/browse/FLINK-34213 Project:

Re: [VOTE] FLIP-377: Support fine-grained configuration to control filter push down for Table/SQL Sources

2024-01-23 Thread Zhongqiang Gong
+1 (non-binding) Best, Zhongqiang Gong On 2024/01/09 11:38:40 Jiabao Sun wrote: > Hi Devs, > > I'd like to start a vote on FLIP-377: Support fine-grained configuration to > control filter push down for Table/SQL Sources[1] > which has been discussed in this thread[2]. > > The vote will be

Re: Re: [VOTE] FLIP-377: Support fine-grained configuration to control filter push down for Table/SQL Sources

2024-01-23 Thread Feng Jin
+1 (non-binding) Best, Feng Jin On Tue, Jan 23, 2024 at 5:46 PM Qingsheng Ren wrote: > +1 (binding) > > Thanks for driving this, Jiabao! > > Best, > Qingsheng > > On Fri, Jan 19, 2024 at 2:11 PM Jark Wu wrote: > > > +1 (binding) > > > > Best, > > Jark > > > > On Tue, 16 Jan 2024 at 18:01,

Re: [DISCUSS] FLIP-418: Show data skew score on Flink Dashboard

2024-01-23 Thread Kartoglu, Emre
Hi Krzysztof, Thank you for the feedback! Please find my comments below. 1. Configurability Adding a feature flag / configuration to enable this is still on the table as far as I am concerned. However I believe adding a new metric shouldn't warrant a flag/configuration. One might argue that

Re: [DISCUSS] FLIP-418: Show data skew score on Flink Dashboard

2024-01-23 Thread Krzysztof Dziołak
Hi Emre, Thank you for driving this proposal. I've got two questions about the extensions to the proposal that are not captured in the FLIP. 1. Configurability - what kind of configuration would you propose to maintain for this feature? Would On/off switch and/or aggregated period length

[jira] [Created] (FLINK-34212) Autoscaler Standalone should clean up stopped jobs to prevent memory leaks

2024-01-23 Thread Rui Fan (Jira)
Rui Fan created FLINK-34212: --- Summary: Autoscaler Standalone should clean up stopped jobs to prevent memory leaks Key: FLINK-34212 URL: https://issues.apache.org/jira/browse/FLINK-34212 Project: Flink

Re: DataOutputSerializer serializing long UTF Strings

2024-01-23 Thread Gyula Fóra
Hi Peter! I think this is a good additional serialization utility to Flink that may benefit different data formats / connectors in the future. +1 Cheers, Gyula On Mon, Jan 22, 2024 at 8:04 PM Steven Wu wrote: > I think this is a reasonable extension to `DataOutputSerializer`. Although > 64

Re: Re: [DISCUSS] FLIP-331: Support EndOfStreamWindows and isOutputOnEOF operator attribute to optimize task deployment

2024-01-23 Thread Xintong Song
Sorry, I just sent an incomplete email draft by mistake. Thanks for reviving this, Xuannan. The updated FLIP LGTM. +1 for it. Best, Xintong On Tue, Jan 23, 2024 at 5:51 PM Xintong Song wrote: > Thanks > > Best, > > Xintong > > > > On Wed, Jan 10, 2024 at 5:56 PM Xuannan Su wrote: > >> Hi

Re: Re: [DISCUSS] FLIP-331: Support EndOfStreamWindows and isOutputOnEOF operator attribute to optimize task deployment

2024-01-23 Thread Xintong Song
Thanks Best, Xintong On Wed, Jan 10, 2024 at 5:56 PM Xuannan Su wrote: > Hi all, > > After several rounds of offline discussions with Xingtong and Jinhao, > we have decided to narrow the scope of the FLIP. It will now focus on > introducing OperatorAttributes that indicate whether an

[jira] [Created] (FLINK-34211) Filtering on Column names with ?s fails for JDBC lookup join.

2024-01-23 Thread david radley (Jira)
david radley created FLINK-34211: Summary: Filtering on Column names with ?s fails for JDBC lookup join. Key: FLINK-34211 URL: https://issues.apache.org/jira/browse/FLINK-34211 Project: Flink

[jira] [Created] (FLINK-34210) DefaultExecutionGraphBuilder#isCheckpointingEnabled may return Wrong Value when checkpoint disabled

2024-01-23 Thread Yue Ma (Jira)
Yue Ma created FLINK-34210: -- Summary: DefaultExecutionGraphBuilder#isCheckpointingEnabled may return Wrong Value when checkpoint disabled Key: FLINK-34210 URL: https://issues.apache.org/jira/browse/FLINK-34210

Re: Re: [VOTE] FLIP-377: Support fine-grained configuration to control filter push down for Table/SQL Sources

2024-01-23 Thread Qingsheng Ren
+1 (binding) Thanks for driving this, Jiabao! Best, Qingsheng On Fri, Jan 19, 2024 at 2:11 PM Jark Wu wrote: > +1 (binding) > > Best, > Jark > > On Tue, 16 Jan 2024 at 18:01, Xuyang wrote: > > > +1 (non-binding) > > > > > > -- > > > > Best! > > Xuyang > > > > > > > > > > > > 在

Re: [DISCUSS] Support detecting table schema from external files.

2024-01-23 Thread Jane Chan
Hi Yisha, Thanks for driving this discussion. I think it's a valuable feature, and feel free to go ahead. Best, Jane On Tue, Jan 16, 2024 at 6:58 PM Benchao Li wrote: > Thanks Yisha for bringing up this discussion. Schema inferring is a > very interesting and useful feature, especially when

[jira] [Created] (FLINK-34209) Migrate FileSink to the new SinkV2 API

2024-01-23 Thread Peter Vary (Jira)
Peter Vary created FLINK-34209: -- Summary: Migrate FileSink to the new SinkV2 API Key: FLINK-34209 URL: https://issues.apache.org/jira/browse/FLINK-34209 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-34208) Migrate SinkV1Adapter to the new SinkV2 API

2024-01-23 Thread Peter Vary (Jira)
Peter Vary created FLINK-34208: -- Summary: Migrate SinkV1Adapter to the new SinkV2 API Key: FLINK-34208 URL: https://issues.apache.org/jira/browse/FLINK-34208 Project: Flink Issue Type: Sub-task