Re: [DISCUSS] FLIP-200: Support Multiple Rule and Dynamic Rule Changing (Flink CEP)

2021-12-14 Thread Nicholas Jiang
Hi Martijin, Thanks for your feedback about my email reply. I left comments for all the points you mentioned: -- Since this FLIP addresses the most watched Jira ticket for Flink (https://issues.apache.org/jira/browse/FLINK-7129), I'm assuming this will be used a lot. Does this also mean that y

[jira] [Created] (FLINK-25318) Improvement of scheduler and execution for Flink OLAP

2021-12-14 Thread Shammon (Jira)
Shammon created FLINK-25318: --- Summary: Improvement of scheduler and execution for Flink OLAP Key: FLINK-25318 URL: https://issues.apache.org/jira/browse/FLINK-25318 Project: Flink Issue Type: Impro

[jira] [Created] (FLINK-25317) ApplicationDispatcherBootstrapITCase blocked on azure

2021-12-14 Thread Yun Gao (Jira)
Yun Gao created FLINK-25317: --- Summary: ApplicationDispatcherBootstrapITCase blocked on azure Key: FLINK-25317 URL: https://issues.apache.org/jira/browse/FLINK-25317 Project: Flink Issue Type: Bug

Re: Some doubts about the FLINK-22848

2021-12-14 Thread Ingo Bürk
Hi, Calcite does indeed support the (unquoted) SET statement, but the arguments are parsed as identifiers, and Flink configuration options (and values) are not valid identifiers. If we could utilize Calcite's SET implementation to correctly parse all valid configurations, it'd be of course b

Re: [VOTE] FLIP-190: Support Version Upgrades for Table API & SQL Programs

2021-12-14 Thread Jing Zhang
+1 Thanks for driving this, Timo. Best, Jing Zhang wenlong.lwl 于2021年12月15日周三 13:31写道: > +1 (non-binding). Thanks for driving this Timo > > > Best, > Wenlong > > On Wed, 15 Dec 2021 at 03:21, Martijn Visser > wrote: > > > +1 (non-binding). Thanks for driving this Timo > > > > Op di 14 dec. 20

Some doubts about the FLINK-22848

2021-12-14 Thread 董剑辉
I have been in contact with Flink and Calcite for a while, and I have some questions about this issue: https://issues.apache.org/jira/browse/FLINK-22848. First of all, the discussion about this issue mentioned that since calcite did not support the syntax analysis of set a=b (without quotes), regul

[jira] [Created] (FLINK-25316) BlobServer can get stuck during shutdown

2021-12-14 Thread Robert Metzger (Jira)
Robert Metzger created FLINK-25316: -- Summary: BlobServer can get stuck during shutdown Key: FLINK-25316 URL: https://issues.apache.org/jira/browse/FLINK-25316 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-25315) Add some extensions and utils to help the Junit5 migration

2021-12-14 Thread Hang Ruan (Jira)
Hang Ruan created FLINK-25315: - Summary: Add some extensions and utils to help the Junit5 migration Key: FLINK-25315 URL: https://issues.apache.org/jira/browse/FLINK-25315 Project: Flink Issue Ty

Re: [VOTE] FLIP-190: Support Version Upgrades for Table API & SQL Programs

2021-12-14 Thread wenlong.lwl
+1 (non-binding). Thanks for driving this Timo Best, Wenlong On Wed, 15 Dec 2021 at 03:21, Martijn Visser wrote: > +1 (non-binding). Thanks for driving this Timo > > Op di 14 dec. 2021 om 17:45 schreef Timo Walther > > > Hi everyone, > > > > I'd like to start a vote on FLIP-190: Support Versi

Re: [DISCUSS] Change some default config values of blocking shuffle

2021-12-14 Thread Yingjie Cao
Hi Till, Thanks for the suggestion. I think it makes a lot of sense to also extend the documentation for the sort shuffle to include a tuning guide. Best, Yingjie Till Rohrmann 于2021年12月14日周二 18:57写道: > As part of this FLIP, does it make sense to also extend the documentation > for the sort sh

[jira] [Created] (FLINK-25314) Update log4j2 version to 2.16.0

2021-12-14 Thread Jinzhong Li (Jira)
Jinzhong Li created FLINK-25314: --- Summary: Update log4j2 version to 2.16.0 Key: FLINK-25314 URL: https://issues.apache.org/jira/browse/FLINK-25314 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-25313) Enable flink-web-ui

2021-12-14 Thread Junfan Zhang (Jira)
Junfan Zhang created FLINK-25313: Summary: Enable flink-web-ui Key: FLINK-25313 URL: https://issues.apache.org/jira/browse/FLINK-25313 Project: Flink Issue Type: Improvement Compone

[jira] [Created] (FLINK-25312) Hive supports managed table

2021-12-14 Thread Jingsong Lee (Jira)
Jingsong Lee created FLINK-25312: Summary: Hive supports managed table Key: FLINK-25312 URL: https://issues.apache.org/jira/browse/FLINK-25312 Project: Flink Issue Type: Sub-task Co

[VOTE] Release 1.11.6/1.12.7/1.13.5/1.14.2, release candidate #1

2021-12-14 Thread Chesnay Schepler
Hi everyone, This vote is for the emergency patch releases for 1.11, 1.12, 1.13 and 1.14 to address CVE-2021-44228/CVE-2021-45046. It covers all 4 releases as they contain the same changes (upgrading Log4j to 2.16.0) and were prepared simultaneously by the same person. (Hence, if something is

[jira] [Created] (FLINK-25311) DelimitedInputFormat cannot read compressed files correctly

2021-12-14 Thread Caizhi Weng (Jira)
Caizhi Weng created FLINK-25311: --- Summary: DelimitedInputFormat cannot read compressed files correctly Key: FLINK-25311 URL: https://issues.apache.org/jira/browse/FLINK-25311 Project: Flink Is

[jira] [Created] (FLINK-25310) Fix the incorrect description for output network buffers in buffer debloat documenation

2021-12-14 Thread Liu (Jira)
Liu created FLINK-25310: --- Summary: Fix the incorrect description for output network buffers in buffer debloat documenation Key: FLINK-25310 URL: https://issues.apache.org/jira/browse/FLINK-25310 Project: Flink

Re: [Proposal] It is hoped that Apache APISIX and Apache Flink will carry out diversified community cooperation

2021-12-14 Thread yeliang wang
x, there are two > > application scenarios: > > > >1. client -> gateway -> kafka -> flink,This is the mainstream > >application scenario (data analysis scenario) in the industry. > >2. client -> gateway -> flink(REST API),Flink has some rest A

Re: [CANCELLED] Release 1.11.5/1.12.6/1.13.4/1.14.1, release candidate #1

2021-12-14 Thread Seth Wiesman
Thank you for managing these updates Chesnay! On Tue, Dec 14, 2021 at 3:51 PM Chesnay Schepler wrote: > Since the maven artifacts have already been published we will use the > next patch version for each release, i.e.: > 1.11.6 > 1.12.7 > 1.13.5 > 1.14.2 > > (We could technically just update t

Re: [CANCELLED] Release 1.11.5/1.12.6/1.13.4/1.14.1, release candidate #1

2021-12-14 Thread Chesnay Schepler
Since the maven artifacts have already been published we will use the next patch version for each release, i.e.: 1.11.6 1.12.7 1.13.5 1.14.2 (We could technically just update the source/binaries, but that seems fishy). On 14/12/2021 22:38, Chesnay Schepler wrote: I'm canceling the release bec

[CANCELLED] Release 1.11.5/1.12.6/1.13.4/1.14.1, release candidate #1

2021-12-14 Thread Chesnay Schepler
I'm canceling the release because the issue was not fully fixed in Log4j 2.15.0; see CVE-2021-45046. I will start preparing new release candidates that use Log4j 2.16.0 . On 14/12/2021 21:28, Chesnay Schepler wrote: The vote duration has passed and we have approved the releases. Binding votes

Re: [VOTE] Release 1.11.5/1.12.6/1.13.4/1.14.1, release candidate #1

2021-12-14 Thread Chesnay Schepler
The vote duration has passed and we have approved the releases. Binding votes: * Stephan * Till * Xintong * Zhu * Gordon I will not finalize the release. On 13/12/2021 20:28, Chesnay Schepler wrote: Hi everyone, This vote is for the emergency patch releases for 1.11, 1.12, 1.13 and 1.14 to a

Re: [VOTE] FLIP-190: Support Version Upgrades for Table API & SQL Programs

2021-12-14 Thread Martijn Visser
+1 (non-binding). Thanks for driving this Timo Op di 14 dec. 2021 om 17:45 schreef Timo Walther > Hi everyone, > > I'd like to start a vote on FLIP-190: Support Version Upgrades for Table > API & SQL Programs [1] which has been discussed in this thread [2]. > > The vote will be open for at least

[jira] [Created] (FLINK-25309) Upgrade to the latest Flink version

2021-12-14 Thread Igal Shilman (Jira)
Igal Shilman created FLINK-25309: Summary: Upgrade to the latest Flink version Key: FLINK-25309 URL: https://issues.apache.org/jira/browse/FLINK-25309 Project: Flink Issue Type: Improvement

Re: [VOTE] Release 1.11.5/1.12.6/1.13.4/1.14.1, release candidate #1

2021-12-14 Thread Tzu-Li (Gordon) Tai
+1 (binding) - verified hashes and signatures - checked that diff of all RCs contain only the log4j version upgrade On Tue, Dec 14, 2021 at 4:06 AM Yun Gao wrote: > +1 (non-binding) > > * Reviewed the blog post. > * Verified each version could run normally with example jobs. > * Checked each ve

Re: [DISCUSS] Change some default config values of blocking shuffle

2021-12-14 Thread Yun Gao
Hi, > I think setting taskmanager.network.sort-shuffle.min-parallelism to 1 and > using sort-shuffle for all cases by default is a good suggestion. I am not > choosing this value mainly because two reasons: > 1. The first one is that it increases the usage of network memory which may > cause

[VOTE] FLIP-190: Support Version Upgrades for Table API & SQL Programs

2021-12-14 Thread Timo Walther
Hi everyone, I'd like to start a vote on FLIP-190: Support Version Upgrades for Table API & SQL Programs [1] which has been discussed in this thread [2]. The vote will be open for at least 72 hours unless there is an objection or not enough votes. [1] https://cwiki.apache.org/confluence/x/K

Re: Unable to restore from checkpoint (Flink 1.13.2 and 1.13.0): serial version has changed in InflightDataRescalingDescriptor

2021-12-14 Thread Dawid Wysakowicz
Hey Adam, We support version upgrades only through a savepoint. We do not support rolling upgrades or upgrades via a checkpoint store. Please also take a look at this issue, which has been created before[1]. Best, Dawid [1] https://issues.apache.org/jira/browse/FLINK-24621 On 14/12/2021 17:16,

[jira] [Created] (FLINK-25308) Support different remote functions module names

2021-12-14 Thread Igal Shilman (Jira)
Igal Shilman created FLINK-25308: Summary: Support different remote functions module names Key: FLINK-25308 URL: https://issues.apache.org/jira/browse/FLINK-25308 Project: Flink Issue Type: I

Re: [DISCUSS] Changing the minimal supported version of Hadoop

2021-12-14 Thread Martijn Visser
Hi David, Thanks for bringing this up for discussion! Given that Hadoop 2.8 is considered EOL, shouldn't we bump the version to Hadoop 2.10? [1] Best regards, Martijn [1] https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+Active+Release+Lines On Tue, 14 Dec 2021 at 10:28, Till Rohrmann

Unable to restore from checkpoint (Flink 1.13.2 and 1.13.0): serial version has changed in InflightDataRescalingDescriptor

2021-12-14 Thread Adam Roberts
Hi everyone, I'm aware of the compatibility matrix for Flink which is available here https://nightlies.apache.org/flink/flink-docs-master/docs/ops/upgrading/ - which suggests 1.13.x and 1.13.x should work just fine. However, if we try to restore a 1.13.0 checkpoint with 1.13.2, we'll get a proble

[jira] [Created] (FLINK-25307) Resuming Savepoint (hashmap, async, no parallelism change) end-to-end test timeout on azure

2021-12-14 Thread Yun Gao (Jira)
Yun Gao created FLINK-25307: --- Summary: Resuming Savepoint (hashmap, async, no parallelism change) end-to-end test timeout on azure Key: FLINK-25307 URL: https://issues.apache.org/jira/browse/FLINK-25307 Pro

[jira] [Created] (FLINK-25306) Flink CLI end-to-end test timeout on azure

2021-12-14 Thread Yun Gao (Jira)
Yun Gao created FLINK-25306: --- Summary: Flink CLI end-to-end test timeout on azure Key: FLINK-25306 URL: https://issues.apache.org/jira/browse/FLINK-25306 Project: Flink Issue Type: Bug Co

[jira] [Created] (FLINK-25305) Always wait for input channel state and result partition state get completed in AsyncRunnable

2021-12-14 Thread Yun Gao (Jira)
Yun Gao created FLINK-25305: --- Summary: Always wait for input channel state and result partition state get completed in AsyncRunnable Key: FLINK-25305 URL: https://issues.apache.org/jira/browse/FLINK-25305 P

Re: [DISCUSS] FLIP-200: Support Multiple Rule and Dynamic Rule Changing (Flink CEP)

2021-12-14 Thread Martijn Visser
Hi all, > IMO, in this FLIP, we only need to introduce the general design of the Table API/SQL level. As for the design details, you can create a new FLIP. Do you think that the current section on Table/SQL API support is sufficient as a general design? > And do we need to take into account the

Re: [DISCUSS] FLIP-200: Support Multiple Rule and Dynamic Rule Changing (Flink CEP)

2021-12-14 Thread Yunfeng Zhou
Hi Martijn, Thanks for your feedback to this FLIP. So far as I know, MATCH_RECOGNIZE does not support multiple & dynamic pattern matching, and there is no other standard SQL grammar that supports this function. In this case I'm not sure whether we would like Table/SQL API to support this newly int

Flink 1.15. Bi-weekly 2021-12-14

2021-12-14 Thread Johannes Moser
Dear Flink Community, Today we had the third release sync for the 1.15 release. Overview There are 8 weeks left until the feature freeze. Since the last sync the progress percentage went from 20% to 36%. Two more items have been completed making it a total of 5 out of 43 items on the l

[jira] [Created] (FLINK-25304) TIMESTAMP/TIMESTAMP_LTZ Pad with trailing zeros for fractional seconds depending on the precision specified

2021-12-14 Thread Marios Trivyzas (Jira)
Marios Trivyzas created FLINK-25304: --- Summary: TIMESTAMP/TIMESTAMP_LTZ Pad with trailing zeros for fractional seconds depending on the precision specified Key: FLINK-25304 URL: https://issues.apache.org/jira/bro

Re: [VOTE] Release 1.11.5/1.12.6/1.13.4/1.14.1, release candidate #1

2021-12-14 Thread Yun Gao
+1 (non-binding) * Reviewed the blog post. * Verified each version could run normally with example jobs. * Checked each version only contains the log4j2 fix. Thanks Chesnay for driving the emergency fix releases! Best, Yun -- From

[jira] [Created] (FLINK-25303) LookupTableSource/TableFunction

2021-12-14 Thread waittting (Jira)
waittting created FLINK-25303: - Summary: LookupTableSource/TableFunction Key: FLINK-25303 URL: https://issues.apache.org/jira/browse/FLINK-25303 Project: Flink Issue Type: Bug Reporte

Re: [DISCUSS] Change some default config values of blocking shuffle

2021-12-14 Thread Till Rohrmann
As part of this FLIP, does it make sense to also extend the documentation for the sort shuffle [1] to include a tuning guide? I am thinking of a more in depth description of what things you might observe and how to influence them with the configuration options. [1] https://nightlies.apache.org/fli

Re: [VOTE] Release 1.11.5/1.12.6/1.13.4/1.14.1, release candidate #1

2021-12-14 Thread Yun Tang
+ 1 (non-binding) for releasing flink-1.13.4 and flink-1.14.1 currently * reviewed blog post * checked that the hot fix verion only contains the log4j2 version bump Best Yun Tang From: Chesnay Schepler Sent: Tuesday, December 14, 2021 17:12 To: dev@flink

[jira] [Created] (FLINK-25302) Find a way to keep release guides for previous releases

2021-12-14 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-25302: Summary: Find a way to keep release guides for previous releases Key: FLINK-25302 URL: https://issues.apache.org/jira/browse/FLINK-25302 Project: Flink

[jira] [Created] (FLINK-25301) Add a separate script for signing binaries

2021-12-14 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-25301: Summary: Add a separate script for signing binaries Key: FLINK-25301 URL: https://issues.apache.org/jira/browse/FLINK-25301 Project: Flink Issue Type

[jira] [Created] (FLINK-25300) Remove CEIL and FLOOR call with one argument for integral values

2021-12-14 Thread Francesco Guardiani (Jira)
Francesco Guardiani created FLINK-25300: --- Summary: Remove CEIL and FLOOR call with one argument for integral values Key: FLINK-25300 URL: https://issues.apache.org/jira/browse/FLINK-25300 Projec

[jira] [Created] (FLINK-25299) Improve LIKE operator efficiency

2021-12-14 Thread Francesco Guardiani (Jira)
Francesco Guardiani created FLINK-25299: --- Summary: Improve LIKE operator efficiency Key: FLINK-25299 URL: https://issues.apache.org/jira/browse/FLINK-25299 Project: Flink Issue Type: Te

[jira] [Created] (FLINK-25298) Create docker image for release environment

2021-12-14 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-25298: Summary: Create docker image for release environment Key: FLINK-25298 URL: https://issues.apache.org/jira/browse/FLINK-25298 Project: Flink Issue Typ

Re: [DISCUSS] Changing the minimal supported version of Hadoop

2021-12-14 Thread Till Rohrmann
Hi David, I think we haven't updated our Hadoop dependencies in a long time. Hence, it is probably time to do so. So +1 for upgrading to the latest patch release. If newer 2.x Hadoop versions are compatible with 2.y with x >= y, then I don't see a problem with dropping support for pre-bundled Had

Re: [VOTE] Release 1.11.5/1.12.6/1.13.4/1.14.1, release candidate #1

2021-12-14 Thread Chesnay Schepler
I think that should be possible. On 14/12/2021 10:06, Till Rohrmann wrote: +1 (binding) - reviewed blog post - verified shasum and signatures - checked that diff only contains the log4j version bump Can we simply add the missing Python binaries for MacOS after the release of the other artifact

Re: [VOTE] Release 1.11.5/1.12.6/1.13.4/1.14.1, release candidate #1

2021-12-14 Thread Till Rohrmann
+1 (binding) - reviewed blog post - verified shasum and signatures - checked that diff only contains the log4j version bump Can we simply add the missing Python binaries for MacOS after the release of the other artifacts? Cheers, Till On Tue, Dec 14, 2021 at 4:56 AM Yun Tang wrote: > Hi Chesn

Re: [DISCUSS] Conventions on assertions to use in tests

2021-12-14 Thread Jing Ge
Hi all, I took a close look at assertj and found there are two concepts for writing tests with two entry points interfaces: WithAssertions for normal style and BDDAssertions for BDD style. I would not suggest using them in one project simultaneously. Since all related work done previously were usi

[VOTE] Release flink-shaded 15.0, release candidate #1

2021-12-14 Thread Chesnay Schepler
Hi everyone, Please review and vote on the release candidate #1 for the version 15.0, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) The complete staging area is available for your review, which includes: * JIRA release notes [1],

[jira] [Created] (FLINK-25297) Publich flink-shaded-swagger

2021-12-14 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-25297: Summary: Publich flink-shaded-swagger Key: FLINK-25297 URL: https://issues.apache.org/jira/browse/FLINK-25297 Project: Flink Issue Type: Improvement

Re: [DISCUSS] FLIP-200: Support Multiple Rule and Dynamic Rule Changing (Flink CEP)

2021-12-14 Thread Nicholas Jiang
Hi Konstantin, Thanks for your detailed explanation for DynamicPattern[Holder] renaming. I have another idea for this renaming, what about renaming the Rule to PatternProcessor? The CEP means that complex event processing, thus the name PatternProcessor corresponds to the concept of CEP. A

Re: [DISCUSS] Strong read-after-write consistency of Flink FileSystems

2021-12-14 Thread David Morávek
Any other thoughts on the topic? If there are no concerns, I'd continue with creating a FLIP for changing the "written" contract of the Flink FileSystems to reflect this. Best, D. On Wed, Dec 8, 2021 at 5:53 PM David Morávek wrote: > Hi Martijn, > > I simply wasn't aware of that one :) It seem

[DISCUSS] Changing the minimal supported version of Hadoop

2021-12-14 Thread David Morávek
Hi, I'd like to start a discussion about upgrading a minimal Hadoop version that Flink supports. Even though the default value for `hadoop.version` property is set to 2.8.3, we're still ensuring both runtime and compile compatibility with Hadoop 2.4.x with the scheduled pipeline[1]. Here is list

[jira] [Created] (FLINK-25296) CheckpointingOptions[state.checkpoints.num-retained ]

2021-12-14 Thread chenqizhu (Jira)
chenqizhu created FLINK-25296: - Summary: CheckpointingOptions[state.checkpoints.num-retained ] Key: FLINK-25296 URL: https://issues.apache.org/jira/browse/FLINK-25296 Project: Flink Issue Type: B

[jira] [Created] (FLINK-25295) Update Log4j to 2.16.0

2021-12-14 Thread Martijn Visser (Jira)
Martijn Visser created FLINK-25295: -- Summary: Update Log4j to 2.16.0 Key: FLINK-25295 URL: https://issues.apache.org/jira/browse/FLINK-25295 Project: Flink Issue Type: Technical Debt