[jira] [Created] (FLINK-27410) Create ArchUnit rules for Public API dependencies

2022-04-25 Thread Alexander Preuss (Jira)
Alexander Preuss created FLINK-27410: Summary: Create ArchUnit rules for Public API dependencies Key: FLINK-27410 URL: https://issues.apache.org/jira/browse/FLINK-27410 Project: Flink Iss

[jira] [Created] (FLINK-27409) Cleanup stale slot allocation record when the resource requirement of a job is empty

2022-04-25 Thread Yangze Guo (Jira)
Yangze Guo created FLINK-27409: -- Summary: Cleanup stale slot allocation record when the resource requirement of a job is empty Key: FLINK-27409 URL: https://issues.apache.org/jira/browse/FLINK-27409 Proj

[DISCUSS] More public interfaces for internal data structures

2022-04-25 Thread Jingsong Li
Hi everyone, The source sink for the Table/SQL has now been completely switched to a new API (DynamicTableFactory), which requires connector developers to prioritize the use of internal data structures. For example, SourceProvider and SinkProvider. There are complex connectors that have many requ

[jira] [Created] (FLINK-27408) Package Flink runtime dependencies into associated table store modules instead of flink-table-store-dist

2022-04-25 Thread Caizhi Weng (Jira)
Caizhi Weng created FLINK-27408: --- Summary: Package Flink runtime dependencies into associated table store modules instead of flink-table-store-dist Key: FLINK-27408 URL: https://issues.apache.org/jira/browse/FLINK-2

[jira] [Created] (FLINK-27407) Exception of "unable to find common type of Boolean and Char(4)" will be thrown when RexSarg contains "Boolean" and "Char(4)"

2022-04-25 Thread luoyuxia (Jira)
luoyuxia created FLINK-27407: Summary: Exception of "unable to find common type of Boolean and Char(4)" will be thrown when RexSarg contains "Boolean" and "Char(4)" Key: FLINK-27407 URL: https://issues.apache.org/jira

[RESULT] [VOTE] Release 1.15.0, release candidate #4

2022-04-25 Thread Yun Gao
I'm happy to announce that we have unanimously approved this release. There are 6 explicit approving votes, 3 of which are binding: * Dawid Wysakowicz (binding) * Xingbo Huang * Matthias Pohl * Yang Wang * Zhu Zhu (binding) * Guowei Ma (binding) There are no disapproving votes. Thank

[jira] [Created] (FLINK-27406) After canceling the job, it is falsely reported as a job failure.

2022-04-25 Thread Guowei Ma (Jira)
Guowei Ma created FLINK-27406: - Summary: After canceling the job, it is falsely reported as a job failure. Key: FLINK-27406 URL: https://issues.apache.org/jira/browse/FLINK-27406 Project: Flink

Re: Re: Re: [VOTE] Release 1.15.0, release candidate #4

2022-04-25 Thread Yun Gao
Very thanks everyone for the verifying! Now we should reach the condition to finalize this candidate, I'll annouce that in a separate thread. Also for the remaining non-blocking issues, perhaps we may start scheduling for the next minor version. Best, Yun Gao --Original Mai

[jira] [Created] (FLINK-27405) Refactor SourceCoordinator to abstract BaseCoordinator implementation

2022-04-25 Thread gang ye (Jira)
gang ye created FLINK-27405: --- Summary: Refactor SourceCoordinator to abstract BaseCoordinator implementation Key: FLINK-27405 URL: https://issues.apache.org/jira/browse/FLINK-27405 Project: Flink

Re: Re: [VOTE] Release 1.15.0, release candidate #4

2022-04-25 Thread Guowei Ma
+1 (binding) 1. Build from the source 2. Verify the checksums and signatures 3. Start a local standalone cluster 4. Submit a streaming example job and stop it on the ui. 5. Submit a job running with batch mode. Best, Guowei On Tue, Apr 26, 2022 at 10:56 AM Zhu Zhu wrote: > +1 (binding) > - Ch

[jira] [Created] (FLINK-27404) Add algorithms completeness test in ML Python API

2022-04-25 Thread Huang Xingbo (Jira)
Huang Xingbo created FLINK-27404: Summary: Add algorithms completeness test in ML Python API Key: FLINK-27404 URL: https://issues.apache.org/jira/browse/FLINK-27404 Project: Flink Issue Type:

Re: Re: [VOTE] Release 1.15.0, release candidate #4

2022-04-25 Thread Zhu Zhu
+1 (binding) - Checked checksums and signatures - Built from source - Ran a batch job using AdaptiveBatchScheduler on yarn cluster - Ran an example job(parallelism=1000) on yarn cluster - Checked the website PR Thanks, Zhu Yun Gao 于2022年4月26日周二 10:37写道: > > Very thanks Matthias for tracking and

Re: [VOTE] Release 1.15.0, release candidate #4

2022-04-25 Thread Yang Wang
+1(non-binding) - Verified signature and checksum - Build image with flink-docker repo - Play above flink rc4 image with flink-kubernetes-operator which could verify the following aspects - Native K8s application integration - Multiple Component Kubernetes HA services - Session cluster

[jira] [Created] (FLINK-27403) Align existing feature engineering algorithms in ML Python API

2022-04-25 Thread Huang Xingbo (Jira)
Huang Xingbo created FLINK-27403: Summary: Align existing feature engineering algorithms in ML Python API Key: FLINK-27403 URL: https://issues.apache.org/jira/browse/FLINK-27403 Project: Flink

Re: Re: [VOTE] Release 1.15.0, release candidate #4

2022-04-25 Thread Yun Gao
Very thanks Matthias for tracking and double confirming this issue! Best, Yun Gao --Original Mail -- Sender:Matthias Pohl Send Date:Mon Apr 25 23:52:30 2022 Recipients:Yun Gao CC:dev , Yun Gao Subject:Re: [VOTE] Release 1.15.0, release candidate #4 +1 (non-b

[jira] [Created] (FLINK-27402) Unexpected boolean expression simplification for AND expression

2022-04-25 Thread luoyuxia (Jira)
luoyuxia created FLINK-27402: Summary: Unexpected boolean expression simplification for AND expression Key: FLINK-27402 URL: https://issues.apache.org/jira/browse/FLINK-27402 Project: Flink Is

[jira] [Created] (FLINK-27401) CEP supports AggregateFunction in IterativeCondition

2022-04-25 Thread Nicholas Jiang (Jira)
Nicholas Jiang created FLINK-27401: -- Summary: CEP supports AggregateFunction in IterativeCondition Key: FLINK-27401 URL: https://issues.apache.org/jira/browse/FLINK-27401 Project: Flink Issu

[jira] [Created] (FLINK-27400) Pulsar connector subscribed the system topic when using the regex

2022-04-25 Thread Yufan Sheng (Jira)
Yufan Sheng created FLINK-27400: --- Summary: Pulsar connector subscribed the system topic when using the regex Key: FLINK-27400 URL: https://issues.apache.org/jira/browse/FLINK-27400 Project: Flink

[jira] [Created] (FLINK-27399) Pulsar connector didn't set start consuming position correctly

2022-04-25 Thread Yufan Sheng (Jira)
Yufan Sheng created FLINK-27399: --- Summary: Pulsar connector didn't set start consuming position correctly Key: FLINK-27399 URL: https://issues.apache.org/jira/browse/FLINK-27399 Project: Flink

[DISCUSS] Next Flink Kubernetes Operator release timeline

2022-04-25 Thread Gyula Fóra
Hi Devs! The community has been working hard on cleaning up the operator logic and adding some core features that have been missing from the preview release (session jobs for example). We have also added some significant improvements around deployment/operations. With the current pace of the deve

Re: [VOTE] Release 1.15.0, release candidate #4

2022-04-25 Thread Matthias Pohl
+1 (non-binding) The issue around the user mailing list thread [1] is identified. We couldn't come up with a reason for why some change in 1.15 would have caused the behavior. The bug itself is/was already present in 1.14. All the findings are collected in FLINK-27354 [2]. We're not considering th

[jira] [Created] (FLINK-27398) Move completeness test to flink-scala

2022-04-25 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-27398: Summary: Move completeness test to flink-scala Key: FLINK-27398 URL: https://issues.apache.org/jira/browse/FLINK-27398 Project: Flink Issue Type: Tec

[jira] [Created] (FLINK-27397) Improve the CrdReferenceDoclet generator to handle the abstract class

2022-04-25 Thread Aitozi (Jira)
Aitozi created FLINK-27397: -- Summary: Improve the CrdReferenceDoclet generator to handle the abstract class Key: FLINK-27397 URL: https://issues.apache.org/jira/browse/FLINK-27397 Project: Flink Is

[jira] [Created] (FLINK-27396) Reduce the Heartbeat timeout after zookeeper suspended

2022-04-25 Thread fanrui (Jira)
fanrui created FLINK-27396: -- Summary: Reduce the Heartbeat timeout after zookeeper suspended Key: FLINK-27396 URL: https://issues.apache.org/jira/browse/FLINK-27396 Project: Flink Issue Type: Improv

Re: Re: [DISCUSS] FLIP-216 Decouple Hive connector with Flink planner

2022-04-25 Thread Martijn Visser
Hi Jark and Yuxia, I think this sounds like a good idea +1. Thanks a lot for taking the time to invest into this. Best regards, Martijn Visser https://twitter.com/MartijnVisser82 https://github.com/MartijnVisser On Tue, 19 Apr 2022 at 17:30, Jark Wu wrote: > Hi Martijn, > > I have discussed

Re: [DISCUSS] Maintain a Calcite repository for Flink to accelerate the development for Flink SQL features

2022-04-25 Thread Martijn Visser
Hi all, Just a couple of remarks on some of things from this thread: > I think we will upgrade Calcite to 1.31 only when Flink depends on some significant features of Calcite. > Such as: new syntax PTF (CALCITE-4865). Like Timo also mentions, I think this is a bad practice. Calcite is a key depe

Re: [DISCUSS] Maintain a Calcite repository for Flink to accelerate the development for Flink SQL features

2022-04-25 Thread Timo Walther
Hi Godfrey, I'm also strictly against maintaining a Calcite fork. We had similar discussions during the merge of the Blink code base in the past and I'm happy that we could prevent a fork until today. Let me elaborate a bit on my strict opinion here: 1) Calcite does not offer bugfix releases

Re: [DISCUSS] Maintain a Calcite repository for Flink to accelerate the development for Flink SQL features

2022-04-25 Thread godfrey he
Hi Jark, Agree with you, thanks for the feedback. Best, Godfrey Jark Wu 于2022年4月25日周一 13:02写道: > > Thanks, Godfrey, for starting this discussion, > > I understand the motivation behind it. > No bugfix releases, slow feature reviewing, and no compatibility guaranteed > are genuinely blocking the

Re: [DISCUSS] FLIP-221 Abstraction for lookup source cache and metric

2022-04-25 Thread Arvid Heise
Hi Qingsheng, Thanks for driving this; the inconsistency was not satisfying for me. I second Alexander's idea though but could also live with an easier solution as the first step: Instead of making caching an implementation detail of TableFunction X, rather devise a caching layer around X. So the

Re: [DISCUSS] FLIP-217 Support watermark alignment of source splits

2022-04-25 Thread Arvid Heise
Thanks for pushing this effort. I'd actually be in favor of 1b). I fully agree that decorator interfaces should be avoided but I'm also not a big fan of overloading the base interfaces (they are hard to implement as is). The usual feedback to Source-related interfaces are always that they are over

[jira] [Created] (FLINK-27395) IllegalStateException: Could not find policy 'pick_first'. on Flink Application

2022-04-25 Thread Natan HP (Jira)
Natan HP created FLINK-27395: Summary: IllegalStateException: Could not find policy 'pick_first'. on Flink Application Key: FLINK-27395 URL: https://issues.apache.org/jira/browse/FLINK-27395 Project: Flin

[jira] [Created] (FLINK-27394) Integrate the Flink Elasticsearch documentation in the Flink documentation

2022-04-25 Thread Martijn Visser (Jira)
Martijn Visser created FLINK-27394: -- Summary: Integrate the Flink Elasticsearch documentation in the Flink documentation Key: FLINK-27394 URL: https://issues.apache.org/jira/browse/FLINK-27394 Projec

[jira] [Created] (FLINK-27393) Move Elasticsearch documentation from Flink repository to the Flink Elasticsearch connector repository

2022-04-25 Thread Martijn Visser (Jira)
Martijn Visser created FLINK-27393: -- Summary: Move Elasticsearch documentation from Flink repository to the Flink Elasticsearch connector repository Key: FLINK-27393 URL: https://issues.apache.org/jira/browse/FLI

Re: [DISCUSS] FLIP-217 Support watermark alignment of source splits

2022-04-25 Thread Dawid Wysakowicz
a) "MySourceReader implements SourceReader, WithSplitsAlignment", along with "MySplitReader implements SplitReader, WithSplitsAlignment", or b) "MySourceReader implements AlignedSourceReader" and "MySplitReader implements AlignedSplitReader", or c) "MySourceReader implements SourceR

[jira] [Created] (FLINK-27392) CEP Pattern supports definition of the maximum time gap between events

2022-04-25 Thread Nicholas Jiang (Jira)
Nicholas Jiang created FLINK-27392: -- Summary: CEP Pattern supports definition of the maximum time gap between events Key: FLINK-27392 URL: https://issues.apache.org/jira/browse/FLINK-27392 Project: F

Re: [DISCUSS] FLIP-217 Support watermark alignment of source splits

2022-04-25 Thread Becket Qin
Thanks for the comment, Jark. 3. Interface/Method Name. > Can the interface be used to align other things in the future? For example, > align read speed, I have > seen users requesting global rate limits. This feature may also need an > interface like this. > If we don't plan to extend this interf

[DISCUSS] FLIP-223: Implement standalone mode support in the kubernetes operator

2022-04-25 Thread Jassat, Usamah
Hi everyone, We would like to start the discussion of the adding standalone mode support to the Flink Kubernetes operator. Standalone mode was initially considered as part of FLIP-212 but decided to be out of scope to focus on Flink native k8s integration for that FLIP [1]. Standalone support w

[jira] [Created] (FLINK-27391) Support Hive bucket table

2022-04-25 Thread tartarus (Jira)
tartarus created FLINK-27391: Summary: Support Hive bucket table Key: FLINK-27391 URL: https://issues.apache.org/jira/browse/FLINK-27391 Project: Flink Issue Type: Sub-task Affects Versions:

Re: [DISCUSS] FLIP-223: Support HiveServer2 Endpoint

2022-04-25 Thread Jark Wu
Thank Shengkai for driving this effort, I think this is an essential addition to Flink Batch. I have some small suggestions: 1) Kyuubi provides three ways to configure Hive metastore [1]. Could we provide similar abilities? Especially with the JDBC Connection URL, users can visit different Hive me

[jira] [Created] (FLINK-27390) Remove unused flink-tests dependencies

2022-04-25 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-27390: Summary: Remove unused flink-tests dependencies Key: FLINK-27390 URL: https://issues.apache.org/jira/browse/FLINK-27390 Project: Flink Issue Type: Te

[jira] [Created] (FLINK-27389) Move test utils from flink-test to flink-test-utils

2022-04-25 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-27389: Summary: Move test utils from flink-test to flink-test-utils Key: FLINK-27389 URL: https://issues.apache.org/jira/browse/FLINK-27389 Project: Flink I

Re: [DISCUSS] FLIP-217 Support watermark alignment of source splits

2022-04-25 Thread Jark Wu
Thank Dawid for the reminder on FLIP-182. Sorry I did miss it. I don't have other concerns then. Best, Jark On Mon, 25 Apr 2022 at 15:40, Dawid Wysakowicz wrote: > @Jark: > > 1. Will the framework always align with watermarks when the source > implements the interface? > I'm afraid not every ca

[jira] [Created] (FLINK-27388) PulsarSourceEnumeratorTest#discoverPartitionsTriggersAssignments seems flaky

2022-04-25 Thread Sebastian Mattheis (Jira)
Sebastian Mattheis created FLINK-27388: -- Summary: PulsarSourceEnumeratorTest#discoverPartitionsTriggersAssignments seems flaky Key: FLINK-27388 URL: https://issues.apache.org/jira/browse/FLINK-27388

[jira] [Created] (FLINK-27387) Support Insert Multi-Table

2022-04-25 Thread tartarus (Jira)
tartarus created FLINK-27387: Summary: Support Insert Multi-Table Key: FLINK-27387 URL: https://issues.apache.org/jira/browse/FLINK-27387 Project: Flink Issue Type: Sub-task Components:

[jira] [Created] (FLINK-27386) throw NPE if multi MAPJOIN hint union all

2022-04-25 Thread tartarus (Jira)
tartarus created FLINK-27386: Summary: throw NPE if multi MAPJOIN hint union all Key: FLINK-27386 URL: https://issues.apache.org/jira/browse/FLINK-27386 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-27385) Field #1: values VARCHAR(2147483647) ARRAY does not exist for expression index($0, 0)

2022-04-25 Thread tartarus (Jira)
tartarus created FLINK-27385: Summary: Field #1: values VARCHAR(2147483647) ARRAY does not exist for expression index($0, 0) Key: FLINK-27385 URL: https://issues.apache.org/jira/browse/FLINK-27385 Project

[jira] [Created] (FLINK-27384) In the Hive dimension table, when the data is changed on the original partition, the create_time configuration does not take effect

2022-04-25 Thread Jira
陈磊 created FLINK-27384: -- Summary: In the Hive dimension table, when the data is changed on the original partition, the create_time configuration does not take effect Key: FLINK-27384 URL: https://issues.apache.org/jira/brows

[jira] [Created] (FLINK-27382) Make Job mode wait with cluster shutdown until the cleanup is done

2022-04-25 Thread Matthias Pohl (Jira)
Matthias Pohl created FLINK-27382: - Summary: Make Job mode wait with cluster shutdown until the cleanup is done Key: FLINK-27382 URL: https://issues.apache.org/jira/browse/FLINK-27382 Project: Flink

[jira] [Created] (FLINK-27383) Make Application mode wait with cluster shutdown until cleanup is done

2022-04-25 Thread Matthias Pohl (Jira)
Matthias Pohl created FLINK-27383: - Summary: Make Application mode wait with cluster shutdown until cleanup is done Key: FLINK-27383 URL: https://issues.apache.org/jira/browse/FLINK-27383 Project: Fli

Re: [DISCUSS] FLIP-217 Support watermark alignment of source splits

2022-04-25 Thread Dawid Wysakowicz
@Jark: 1. Will the framework always align with watermarks when the source implements the interface? I'm afraid not every case needs watermark alignment even if Kafka implements the interface, and this will affect the throughput somehow. I agree with Becket we may need a `supp

[jira] [Created] (FLINK-27381) HybridSource split should use Arrays.hashcode

2022-04-25 Thread Ran Tao (Jira)
Ran Tao created FLINK-27381: --- Summary: HybridSource split should use Arrays.hashcode Key: FLINK-27381 URL: https://issues.apache.org/jira/browse/FLINK-27381 Project: Flink Issue Type: Improvement