[jira] [Created] (FLINK-34259) flink-connector-jdbc fails to compile with NPE on hasGenericTypesDisabled

2024-01-29 Thread Martijn Visser (Jira)
Martijn Visser created FLINK-34259: -- Summary: flink-connector-jdbc fails to compile with NPE on hasGenericTypesDisabled Key: FLINK-34259 URL: https://issues.apache.org/jira/browse/FLINK-34259 Project

[jira] [Created] (FLINK-34260) Make flink-connector-aws compatible with SinkV2 changes

2024-01-29 Thread Martijn Visser (Jira)
Martijn Visser created FLINK-34260: -- Summary: Make flink-connector-aws compatible with SinkV2 changes Key: FLINK-34260 URL: https://issues.apache.org/jira/browse/FLINK-34260 Project: Flink I

[DISCUSS] Drop support for HBase v1

2024-01-29 Thread Martijn Visser
Hi all, While working on adding support for Flink 1.19 for HBase, we've run into a dependency convergence issue because HBase v1 relies on a really old version of Guava. HBase v2 has been made available since May 2018, and there have been no new releases of HBase v1 since August 2022. I would li

Re: [DISCUSS] FLIP-408: [Umbrella] Introduce DataStream API V2

2024-01-29 Thread Xuannan Su
Hi Weijie, Thanks for driving the work! There are indeed many pain points in the current DataStream API, which are challenging to resolve with its existing design. It is a great opportunity to propose a new DataStream API that tackles these issues. I like the way we've divided the FLIP into multip

[jira] [Created] (FLINK-34261) When slotmanager.redundant-taskmanager-num is set to a value greater than 1, redundant task managers may be repeatedly released and requested

2024-01-29 Thread junzhong qin (Jira)
junzhong qin created FLINK-34261: Summary: When slotmanager.redundant-taskmanager-num is set to a value greater than 1, redundant task managers may be repeatedly released and requested Key: FLINK-34261 URL: https

[jira] [Created] (FLINK-34262) Support to set user specified labels for Rest Service Object

2024-01-29 Thread Prabhu Joseph (Jira)
Prabhu Joseph created FLINK-34262: - Summary: Support to set user specified labels for Rest Service Object Key: FLINK-34262 URL: https://issues.apache.org/jira/browse/FLINK-34262 Project: Flink

[VOTE] FLIP-418: Show data skew score on Flink Dashboard

2024-01-29 Thread Kartoglu, Emre
Hello, I'd like to call votes on FLIP-418: Show data skew score on Flink Dashboard. FLIP: https://cwiki.apache.org/confluence/display/FLINK/FLIP-418%3A+Show+data+skew+score+on+Flink+Dashboard Discussion: https://lists.apache.org/thread/m5ockoork0h2zr78h77dcrn71rbt35ql Kind regards, Emre

Re:[VOTE] FLIP-418: Show data skew score on Flink Dashboard

2024-01-29 Thread 嘉伟
At 2024-01-29 18:09:10, "Kartoglu, Emre" wrote: >Hello, > >I'd like to call votes on FLIP-418: Show data skew score on Flink Dashboard. > >FLIP: >https://cwiki.apache.org/confluence/display/FLINK/FLIP-418%3A+Show+data+skew+score+on+Flink+Dashboard >Discussion: https://lists.apa

Re: [DISCUSS] FLIP-409: DataStream V2 Building Blocks: DataStream, Partitioning and ProcessFunction

2024-01-29 Thread Xuannan Su
Hi Weijie, Thank you for driving the design of the new DataStream API. I have a few questions regarding the FLIP: 1. In the partitioning section, it says that "broadcast can only be used as a side-input of other Inputs." Could you clarify what is meant by "side-input"? If I understand correctly,

Re: Re: [VOTE] FLIP-417: Expose JobManagerOperatorMetrics via REST API

2024-01-29 Thread Maximilian Michels
+1 (binding) On Fri, Jan 26, 2024 at 6:03 AM Rui Fan <1996fan...@gmail.com> wrote: > > +1(binding) > > Best, > Rui > > On Fri, Jan 26, 2024 at 11:55 AM Xuyang wrote: > > > +1 (non-binding) > > > > > > -- > > > > Best! > > Xuyang > > > > > > > > > > > > 在 2024-01-26 10:12:34,"Hang Ruan" 写

Re: [DISCUSS] FLIP-410: Config, Context and Processing Timer Service of DataStream API V2

2024-01-29 Thread Xuannan Su
Hi Weijie, Thanks for the FLIP! I have a few questions regarding the FLIP. 1. +1 to only use XXXParititionStream if users only need to use the configurable PartitionStream. If there are use cases for both, perhaps we could use `ProcessConfigurableNonKeyedPartitionStream` or `ConfigurableNonKeyed

[VOTE] Release flink-connector-mongodb 1.1.0, release candidate #1

2024-01-29 Thread Leonard Xu
Hey all, Please help review and vote on the release candidate #1 for the version 1.1.0 of the Apache Flink MongoDB Connector as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) The complete staging area is available for your review, whic

[jira] [Created] (FLINK-34263) Converting double to decimal may fail

2024-01-29 Thread Caican Cai (Jira)
Caican Cai created FLINK-34263: -- Summary: Converting double to decimal may fail Key: FLINK-34263 URL: https://issues.apache.org/jira/browse/FLINK-34263 Project: Flink Issue Type: Bug C

Re: Re: [VOTE] FLIP-417: Expose JobManagerOperatorMetrics via REST API

2024-01-29 Thread Thomas Weise
+1 (binding) On Mon, Jan 29, 2024 at 5:45 AM Maximilian Michels wrote: > +1 (binding) > > On Fri, Jan 26, 2024 at 6:03 AM Rui Fan <1996fan...@gmail.com> wrote: > > > > +1(binding) > > > > Best, > > Rui > > > > On Fri, Jan 26, 2024 at 11:55 AM Xuyang wrote: > > > > > +1 (non-binding) > > > > >

[jira] [Created] (FLINK-34264) Prometheuspushgateway's hosturl can use basicAuth certification login

2024-01-29 Thread Yangzhou Huang (Jira)
Yangzhou Huang created FLINK-34264: -- Summary: Prometheuspushgateway's hosturl can use basicAuth certification login Key: FLINK-34264 URL: https://issues.apache.org/jira/browse/FLINK-34264 Project: Fl

RE: [VOTE] Release flink-connector-mongodb 1.1.0, release candidate #1

2024-01-29 Thread Jiabao Sun
Thanks Leonard for driving this. +1(non-binding) - Release notes look good - Tag is present in Github - Validated checksum hash - Verified signature - Build the source with Maven by jdk8,11,17,21 - Verified web PR and left minor comments - Run a filter push down test by sql-client on Flink 1.18.1

[jira] [Created] (FLINK-34265) Add the doc of named parameters

2024-01-29 Thread Feng Jin (Jira)
Feng Jin created FLINK-34265: Summary: Add the doc of named parameters Key: FLINK-34265 URL: https://issues.apache.org/jira/browse/FLINK-34265 Project: Flink Issue Type: Sub-task Compon

[jira] [Created] (FLINK-34266) Output ratios should be computed over the whole metric window instead of averaged

2024-01-29 Thread Gyula Fora (Jira)
Gyula Fora created FLINK-34266: -- Summary: Output ratios should be computed over the whole metric window instead of averaged Key: FLINK-34266 URL: https://issues.apache.org/jira/browse/FLINK-34266 Project

Re: [VOTE] Release flink-connector-parent, release candidate #1

2024-01-29 Thread Maximilian Michels
- Inspected the source for licenses and corresponding headers - Checksums and signature OK +1 (binding) On Tue, Jan 23, 2024 at 4:08 PM Etienne Chauchot wrote: > > Hi everyone, > > Please review and vote on the release candidate #1 for the version > 1.1.0, as follows: > > [ ] +1, Approve the rel

[jira] [Created] (FLINK-34267) Python connector test fails when running on MacBook with m1 processor

2024-01-29 Thread Aleksandr Pilipenko (Jira)
Aleksandr Pilipenko created FLINK-34267: --- Summary: Python connector test fails when running on MacBook with m1 processor Key: FLINK-34267 URL: https://issues.apache.org/jira/browse/FLINK-34267 P

[jira] [Created] (FLINK-34268) Add a test to verify if restore test exists for ExecNode

2024-01-29 Thread Bonnie Varghese (Jira)
Bonnie Varghese created FLINK-34268: --- Summary: Add a test to verify if restore test exists for ExecNode Key: FLINK-34268 URL: https://issues.apache.org/jira/browse/FLINK-34268 Project: Flink

[VOTE] Release flink-connector-jdbc, release candidate #2

2024-01-29 Thread Sergey Nuyanzin
Hi everyone, Please review and vote on the release candidate #2 for the version 3.1.2, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) This version is compatible with Flink 1.16.x, 1.17.x and 1.18.x. The complete staging area is availa

[jira] [Created] (FLINK-34269) Sync maven-shade-plugin across modules in flink-shaded

2024-01-29 Thread Sergey Nuyanzin (Jira)
Sergey Nuyanzin created FLINK-34269: --- Summary: Sync maven-shade-plugin across modules in flink-shaded Key: FLINK-34269 URL: https://issues.apache.org/jira/browse/FLINK-34269 Project: Flink

RE: [VOTE] Release flink-connector-jdbc, release candidate #2

2024-01-29 Thread Jiabao Sun
+1(non-binding) - Release notes look good - Tag is present in Github - Validated checksum hash - Verified signature - Verified web PR and left minor comments Best, Jiabao On 2024/01/30 00:17:54 Sergey Nuyanzin wrote: > Hi everyone, > Please review and vote on the release candidate #2 for the ve

Re: [DISCUSS] FLIP-408: [Umbrella] Introduce DataStream API V2

2024-01-29 Thread weijie guo
Hi Wencong: > The Processing TimerService is currently defined as one of the basic primitives, partly because it's understood that you have to choose between processing time and event time. The other part of the reason is that it needs to work based on the task's mailbox thread model to avoid conc

Re: [DISCUSS] FLIP-408: [Umbrella] Introduce DataStream API V2

2024-01-29 Thread Xintong Song
Thanks for working on this, Weijie. The design flaws of the current DataStream API (i.e., V1) have been a pain for a long time. It's great to see efforts going on trying to resolve them. Significant changes to such an important and comprehensive set of public APIs deserves caution. From that pers

Re: Re: [DISCUSS] FLIP-331: Support EndOfStreamWindows and isOutputOnEOF operator attribute to optimize task deployment

2024-01-29 Thread Xuannan Su
Hi all, Thanks for the comments and suggestions. If there are no further comments, I will open the voting thread tomorrow. Best regards, Xuannan On Fri, Jan 26, 2024 at 3:16 PM Dong Lin wrote: > > Thanks Xuannan for the update! > > +1 (binding) > > On Wed, Jan 10, 2024 at 5:54 PM Xuannan Su wro

Re: [DISCUSS] FLIP-409: DataStream V2 Building Blocks: DataStream, Partitioning and ProcessFunction

2024-01-29 Thread weijie guo
Hi Wencong: Thank you for your attention > Q1. Other DataStream types are converted into Non-Keyed DataStreams by using a "shuffle" operation to convert Input into output. Does this "shuffle" include the various repartition operations (rebalance/rescale/shuffle) from DataStream V1? Yes, The name

Re: [DISCUSS] FLIP-409: DataStream V2 Building Blocks: DataStream, Partitioning and ProcessFunction

2024-01-29 Thread weijie guo
Hi Yunfeng, Thank you for your attention > 1. Will we provide any API to support choosing which input to consume between the two inputs of TwoInputStreamProcessFunction? It would be helpful in online machine learning cases, where a process function needs to receive the first machine learning mode

Re: [DISCUSS] FLIP-409: DataStream V2 Building Blocks: DataStream, Partitioning and ProcessFunction

2024-01-29 Thread weijie guo
Hi Xuannan, Thank you for your attention. > In the partitioning section, it says that "broadcast can only be used as a side-input of other Inputs." Could you clarify what is meant by "side-input"? If I understand correctly, it refer to one of the inputs of the `TwoInputStreamProcessFunction`. If

Re: [DISCUSS] FLIP-410: Config, Context and Processing Timer Service of DataStream API V2

2024-01-29 Thread weijie guo
Thanks Yunfeng, Let me try to answer your question :) > 1. Would it be better to have all XXXPartitionStream classes implement ProcessConfigurable, instead of defining both XXXPartitionStream and ProcessConfigurableAndXXXPartitionStream? I wonder whether users would need to operate on a non-confi

[jira] [Created] (FLINK-34270) Update connector developer-facing doc

2024-01-29 Thread Zhanghao Chen (Jira)
Zhanghao Chen created FLINK-34270: - Summary: Update connector developer-facing doc Key: FLINK-34270 URL: https://issues.apache.org/jira/browse/FLINK-34270 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-34271) Fix the unstable test about GroupAggregateRestoreTest#AGG_WITH_STATE_TTL_HINT

2024-01-29 Thread xuyang (Jira)
xuyang created FLINK-34271: -- Summary: Fix the unstable test about GroupAggregateRestoreTest#AGG_WITH_STATE_TTL_HINT Key: FLINK-34271 URL: https://issues.apache.org/jira/browse/FLINK-34271 Project: Flink

Re: [DISCUSS] FLIP-409: DataStream V2 Building Blocks: DataStream, Partitioning and ProcessFunction

2024-01-29 Thread Xintong Song
Just trying to understand. > Is there a particular reason we do not support a > `TwoInputProcessFunction` to combine a KeyedStream with a > BroadcastStream to result in a KeyedStream? There seems to be a valid > use case where a KeyedStream is enriched with a BroadcastStream and > returns a Stream

Re: [VOTE] Release flink-connector-jdbc, release candidate #2

2024-01-29 Thread Hang Ruan
+1 (non-binding) - Validated checksum hash - Verified signature - Verified that no binaries exist in the source archive - Build the source with Maven and jdk11 - Verified web PR - Check that the jar is built by jdk8 Best, Hang Jiabao Sun 于2024年1月30日周二 10:52写道: > +1(non-binding) > > - Release n

Re: [DISCUSS] FLIP-410: Config, Context and Processing Timer Service of DataStream API V2

2024-01-29 Thread weijie guo
Hi Xuannan, > 1. +1 to only use XXXParititionStream if users only need to use the configurable PartitionStream. If there are use cases for both, perhaps we could use `ProcessConfigurableNonKeyedPartitionStream` or `ConfigurableNonKeyedPartitionStream` for simplicity. As for why we need both, you

Re: [VOTE] Release flink-connector-mongodb 1.1.0, release candidate #1

2024-01-29 Thread Hang Ruan
Hi, Leonard. I find that META-INF/MANIFEST.MF in flink-sql-connector-mongodb-1.1.0-1.18.jar shows as follow. Manifest-Version: 1.0 Archiver-Version: Plexus Archiver Created-By: Apache Maven 3.8.1 Built-By: bangjiangxu Build-Jdk: 11.0.11 Specification-Title: Flink : Connectors : SQL : MongoDB Spec

Re: [VOTE] Release flink-connector-mongodb 1.1.0, release candidate #1

2024-01-29 Thread Leonard Xu
Thanks Hang for the review! We should use jdk8 to compile the jar, I will cancel the rc1 and prepare rc2 soon. Best, Leonard > 2024年1月30日 下午2:39,Hang Ruan 写道: > > Hi, Leonard. > > I find that META-INF/MANIFEST.MF in > flink-sql-connector-mongodb-1.1.0-1.18.jar shows as follow. > > Manifest

Re: [DISCUSS] FLIP-409: DataStream V2 Building Blocks: DataStream, Partitioning and ProcessFunction

2024-01-29 Thread weijie guo
Hi Xintong, Thanks for your reply. > Does this mean if we want to support (KeyedStream, BroadcastStream) -> (KeyedStream), we must make sure that no data can be output upon processing records from the input BroadcastStream? That's probably a reasonable limitation. I think so, this is the restric

Re: [DISCUSS] FLIP-410: Config, Context and Processing Timer Service of DataStream API V2

2024-01-29 Thread weijie guo
Hi Wencong, > Q1. In the "Configuration" section, it is mentioned that configurations can be set continuously using the withXXX methods. Are these configuration options the same as those provided by DataStream V1, or might there be different options compared to V1? I haven't considered options th

Re: Re: [DISCUSS] FLIP-331: Support EndOfStreamWindows and isOutputOnEOF operator attribute to optimize task deployment

2024-01-29 Thread weijie guo
Thanks Dong and Xuannan, The updated FLIP LGTM. Best regards, Weijie Xuannan Su 于2024年1月30日周二 11:10写道: > Hi all, > > Thanks for the comments and suggestions. If there are no further > comments, I will open the voting thread tomorrow. > > Best regards, > Xuannan > On Fri, Jan 26, 2024 at 3:1

Re: [VOTE] Release flink-connector-kafka v3.1.0, release candidate #1

2024-01-29 Thread Mason Chen
+1 (non-binding) * Verified LICENSE and NOTICE files (this RC has a NOTICE file that points to 2023 that has since been updated on the main branch by Hang) * Verified hashes and signatures * Verified no binaries * Verified poms point to 3.1.0 * Reviewed web PR * Built from source * Verified git ta

[jira] [Created] (FLINK-34272) AdaptiveSchedulerClusterITCase failure due to MiniCluster not running

2024-01-29 Thread Matthias Pohl (Jira)
Matthias Pohl created FLINK-34272: - Summary: AdaptiveSchedulerClusterITCase failure due to MiniCluster not running Key: FLINK-34272 URL: https://issues.apache.org/jira/browse/FLINK-34272 Project: Flin