[jira] [Created] (FLINK-25339) Moving to hadoop-free flink runtime.

2021-12-15 Thread Jira
David Morávek created FLINK-25339: - Summary: Moving to hadoop-free flink runtime. Key: FLINK-25339 URL: https://issues.apache.org/jira/browse/FLINK-25339 Project: Flink Issue Type: Technical

[jira] [Created] (FLINK-25338) Improvement of connection from TM to JM in session cluster

2021-12-15 Thread Shammon (Jira)
Shammon created FLINK-25338: --- Summary: Improvement of connection from TM to JM in session cluster Key: FLINK-25338 URL: https://issues.apache.org/jira/browse/FLINK-25338 Project: Flink Issue Type:

Re: Re: [DISCUSS] FLIP-191: Extend unified Sink interface to support small file compaction

2021-12-15 Thread Yun Gao
Hi Fabian, Very thanks for the update! I think the latest version in general looks good from my side and I think using separate feature interface would be much more easy to understand and extend in the future. I have some pending issues on the details though: 1. The first one is if we could

Re: [VOTE] Release 1.11.6/1.12.7/1.13.5/1.14.2, release candidate #1

2021-12-15 Thread Israel Ekpo
Thanks for running and release Chesnay I hope so too. On Wed, Dec 15, 2021 at 9:56 PM Chesnay Schepler wrote: > The vote duration has passed and we have approved the releases. > > Binding votes: > * Till > * Yu > * Stephan > > I will now finalize the release. Let's hope this time no new CVE

[jira] [Created] (FLINK-25337) Check whether the target table is valid when SqlToOperationConverter.convertSqlInsert

2021-12-15 Thread vim-wang (Jira)
vim-wang created FLINK-25337: Summary: Check whether the target table is valid when SqlToOperationConverter.convertSqlInsert Key: FLINK-25337 URL: https://issues.apache.org/jira/browse/FLINK-25337

[jira] [Created] (FLINK-25336) Kafka connector compatible problem in Flink sql

2021-12-15 Thread Yuan Zhu (Jira)
Yuan Zhu created FLINK-25336: Summary: Kafka connector compatible problem in Flink sql Key: FLINK-25336 URL: https://issues.apache.org/jira/browse/FLINK-25336 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-25335) Improvoment of task deployment by enable source split async enumerate

2021-12-15 Thread KevinyhZou (Jira)
KevinyhZou created FLINK-25335: -- Summary: Improvoment of task deployment by enable source split async enumerate Key: FLINK-25335 URL: https://issues.apache.org/jira/browse/FLINK-25335 Project: Flink

Re: [VOTE] Release 1.11.6/1.12.7/1.13.5/1.14.2, release candidate #1

2021-12-15 Thread Chesnay Schepler
The vote duration has passed and we have approved the releases. Binding votes: * Till * Yu * Stephan I will now finalize the release. Let's hope this time no new CVE pops up... On 15/12/2021 03:55, Chesnay Schepler wrote: Hi everyone, This vote is for the emergency patch releases for 1.11,

Re: [VOTE] Release 1.11.6/1.12.7/1.13.5/1.14.2, release candidate #1

2021-12-15 Thread Israel Ekpo
The pull request for Flink-web looks good to me +1 (non binding) from me On Wed, Dec 15, 2021 at 8:19 PM Yun Gao wrote: > > Ah actually we may have to merge another commit into the release > > branches for that to be the case for 1.11/1.12. > > > Anyway, I still wouldn't block the release on

Re: [DISCUSS] FLIP-198: Working directory for Flink processes

2021-12-15 Thread Yang Wang
I am afraid creating a dedicated StatefulSet for each TaskManager is too expensive and using a shared StatefulSet for all the TaskManagers is not flexible enough. Maybe setting a proper restart policy for TaskManager pods could benefit from this FLIP. But we might need to tackle some other issues,

[jira] [Created] (FLINK-25334) Manage and share gateways of taskmanagers between jobs in session cluster

2021-12-15 Thread Shammon (Jira)
Shammon created FLINK-25334: --- Summary: Manage and share gateways of taskmanagers between jobs in session cluster Key: FLINK-25334 URL: https://issues.apache.org/jira/browse/FLINK-25334 Project: Flink

Re: [VOTE] Release 1.11.6/1.12.7/1.13.5/1.14.2, release candidate #1

2021-12-15 Thread Yun Gao
> Ah actually we may have to merge another commit into the release > branches for that to be the case for 1.11/1.12. > Anyway, I still wouldn't block the release on that. Yes, it should only be cases for 1.11 / 1.12~ With a double check the other two versions contains the doc modification. Very

Re: [VOTE] Release 1.11.6/1.12.7/1.13.5/1.14.2, release candidate #1

2021-12-15 Thread Yun Gao
Hi Chesnay, Very thanks for the explanation! Then +1 (non-binding) * Verifies the binary distributions could run successfully with the example jobs. * Verifies the new tags only contains the log4j version bump PRs. * The latest blog post and release notes looks good to me. Very thanks for

Re: [VOTE] Release 1.11.6/1.12.7/1.13.5/1.14.2, release candidate #1

2021-12-15 Thread Chesnay Schepler
Ah actually we may have to merge another commit into the release branches for that to be the case for 1.11/1.12. Anyway, I still wouldn't block the release on that. On 16/12/2021 02:03, Chesnay Schepler wrote: That's not a problem as far as I'm concerned, because the documentation of these

Re: [VOTE] Release 1.11.6/1.12.7/1.13.5/1.14.2, release candidate #1

2021-12-15 Thread Chesnay Schepler
That's not a problem as far as I'm concerned, because the documentation of these releases is never actually published anywhere. The documentation at nightlies.apache.org will reference 2.16.0. On 16/12/2021 02:01, Yun Gao wrote: Hi, May I have a double confirmation that it seems we still have

Re: Re: [VOTE] Release 1.11.6/1.12.7/1.13.5/1.14.2, release candidate #1

2021-12-15 Thread Yun Gao
Hi, May I have a double confirmation that it seems we still have log4j version written as 2.15.0 in dev/project-configuration.md in the commit corresponding to release tags, which seems to be not consistent with the PR in the github, is it expected and would it have influence? Best, Yun

Re: [VOTE] Release 1.11.6/1.12.7/1.13.5/1.14.2, release candidate #1

2021-12-15 Thread Stephan Ewen
+1 (binding) - Verified commit history, looks good => stumbled over the changes in the "create_release_branch.sh ", which are present in each release commit. [1] => agree that these are not an issue, because this is an out-of-band release - Release notes for 1.14.2 are off,

Re: [VOTE] Release 1.11.6/1.12.7/1.13.5/1.14.2, release candidate #1

2021-12-15 Thread Seth Wiesman
+1 (non-binding) - Checked diff of all versions and verified dep upgrade - Verified checksum and signatures - Built 1.14 from source - checked blog post Seth On Wed, Dec 15, 2021 at 10:22 AM Yu Li wrote: > +1 > > * Verified checksums and signatures > * Reviewed website PR >- Minor: left a

[jira] [Created] (FLINK-25333) Harden UID management when constructing statefun universe

2021-12-15 Thread Seth Wiesman (Jira)
Seth Wiesman created FLINK-25333: Summary: Harden UID management when constructing statefun universe Key: FLINK-25333 URL: https://issues.apache.org/jira/browse/FLINK-25333 Project: Flink

[jira] [Created] (FLINK-25332) When using Pyflink Table API, 'where' clause seems to work incorrectly

2021-12-15 Thread TongMeng (Jira)
TongMeng created FLINK-25332: Summary: When using Pyflink Table API, 'where' clause seems to work incorrectly Key: FLINK-25332 URL: https://issues.apache.org/jira/browse/FLINK-25332 Project: Flink

Re: [VOTE] Release 1.11.6/1.12.7/1.13.5/1.14.2, release candidate #1

2021-12-15 Thread Yu Li
+1 * Verified checksums and signatures * Reviewed website PR - Minor: left a comment to mention CVE-2021-45046 * Checked and confirmed new tags only contain log4j version bump * Checked release notes and found no issues - I've moved FLINK-25317 to 1.14.3 Thanks for driving these releases

[jira] [Created] (FLINK-25331) Allow testcontainer tests to run on Java 11

2021-12-15 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-25331: Summary: Allow testcontainer tests to run on Java 11 Key: FLINK-25331 URL: https://issues.apache.org/jira/browse/FLINK-25331 Project: Flink Issue

Re: [VOTE] Release 1.11.6/1.12.7/1.13.5/1.14.2, release candidate #1

2021-12-15 Thread Chesnay Schepler
FYI; the publication of the python release for 1.11/1.12 will be delayed because we hit the project size limit on pypi again, and increasing that limit may take a while. On the positive side, this gives us more time to fix the mac builds. On 15/12/2021 03:55, Chesnay Schepler wrote: Hi

[jira] [Created] (FLINK-25330) Flink SQL doesn't retract all versions of Hbase data

2021-12-15 Thread Bruce Wong (Jira)
Bruce Wong created FLINK-25330: -- Summary: Flink SQL doesn't retract all versions of Hbase data Key: FLINK-25330 URL: https://issues.apache.org/jira/browse/FLINK-25330 Project: Flink Issue Type:

Re: [CANCELLED] Release 1.11.5/1.12.6/1.13.4/1.14.1, release candidate #1

2021-12-15 Thread Stephan Ewen
That's right, they are referenced in POMs published with the jars, though. But that's minor. On Wed, Dec 15, 2021 at 12:28 PM Chesnay Schepler wrote: > AFAIK none of the jars we publish actually contains log4j. > It's only bundled by the distribution/python binaries/docker images. > > Hence I

[jira] [Created] (FLINK-25329) Improvement of disk operations in flink session cluster for jobs

2021-12-15 Thread Shammon (Jira)
Shammon created FLINK-25329: --- Summary: Improvement of disk operations in flink session cluster for jobs Key: FLINK-25329 URL: https://issues.apache.org/jira/browse/FLINK-25329 Project: Flink

Re: [CANCELLED] Release 1.11.5/1.12.6/1.13.4/1.14.1, release candidate #1

2021-12-15 Thread Chesnay Schepler
AFAIK none of the jars we publish actually contains log4j. It's only bundled by the distribution/python binaries/docker images. Hence I don't think the jars help in this case. On 15/12/2021 10:42, Stephan Ewen wrote: Given that these artifacts are published already, users can use them if they

[jira] [Created] (FLINK-25328) Improvement of share memory manager between jobs if they use the same slot in TaskManager for flink olap queries

2021-12-15 Thread Shammon (Jira)
Shammon created FLINK-25328: --- Summary: Improvement of share memory manager between jobs if they use the same slot in TaskManager for flink olap queries Key: FLINK-25328 URL:

[jira] [Created] (FLINK-25327) ApplicationMode "DELETE /cluster" REST call leads to exit code 2, instead of 0

2021-12-15 Thread Robert Metzger (Jira)
Robert Metzger created FLINK-25327: -- Summary: ApplicationMode "DELETE /cluster" REST call leads to exit code 2, instead of 0 Key: FLINK-25327 URL: https://issues.apache.org/jira/browse/FLINK-25327

Re: [DISCUSS] FLIP-198: Working directory for Flink processes

2021-12-15 Thread Till Rohrmann
This is true. But this is not a new problem and I think that Flink should be susceptible to this problem already. One solution for this concrete case could be that the BlobServer stores some checksums and validates the file before serving it to the TM. Cheers, Till On Wed, Dec 15, 2021 at 11:59

Re: [DISCUSS] FLIP-198: Working directory for Flink processes

2021-12-15 Thread Chesnay Schepler
The issue with corrupted files is that some of them aren't read by the component that stores them. For example, a file can be corrupted in the blob server of the JM, but that it is corrupted will only be noticed by the TaskExecutor. On 15/12/2021 11:36, Till Rohrmann wrote: Thanks everyone

Re: [DISCUSS] FLIP-198: Working directory for Flink processes

2021-12-15 Thread Till Rohrmann
Thanks everyone for your feedback. Let me try to address it by grouping some of the individual comments: ### Will this feature work for native Yarn and K8s deployments? The working directory is an optional feature that can be used to recover additional information. You can think of it like a

Re: [VOTE] FLIP-190: Support Version Upgrades for Table API & SQL Programs

2021-12-15 Thread godfrey he
+1 (binding) Best, Godfrey Ingo Bürk 于2021年12月15日周三 16:19写道: > > +1 (binding) > > Thanks for driving this much needed feature! > > On 14.12.21 17:45, Timo Walther wrote: > > Hi everyone, > > > > I'd like to start a vote on FLIP-190: Support Version Upgrades for Table > > API & SQL Programs [1]

Re: [VOTE] Release 1.11.6/1.12.7/1.13.5/1.14.2, release candidate #1

2021-12-15 Thread Chesnay Schepler
I had to monkey patch the release scripts a bit; it's unfortunate that the changes ended up in the releases, but I don't think it's a blocker. On 15/12/2021 11:01, Till Rohrmann wrote: Thanks for driving these releases Chesnay! +1 * Verified checksums and signatures * Reviewed website PR *

[jira] [Created] (FLINK-25326) KafkaUtil.createKafkaContainer log levels are not set correctly

2021-12-15 Thread Alexander Preuss (Jira)
Alexander Preuss created FLINK-25326: Summary: KafkaUtil.createKafkaContainer log levels are not set correctly Key: FLINK-25326 URL: https://issues.apache.org/jira/browse/FLINK-25326 Project:

[jira] [Created] (FLINK-25325) Migration Flink from Junit4 to Junit5

2021-12-15 Thread Jing Ge (Jira)
Jing Ge created FLINK-25325: --- Summary: Migration Flink from Junit4 to Junit5 Key: FLINK-25325 URL: https://issues.apache.org/jira/browse/FLINK-25325 Project: Flink Issue Type: New Feature

[jira] [Created] (FLINK-25324) Migration Flink from Junit4 to Junit5

2021-12-15 Thread Jing Ge (Jira)
Jing Ge created FLINK-25324: --- Summary: Migration Flink from Junit4 to Junit5 Key: FLINK-25324 URL: https://issues.apache.org/jira/browse/FLINK-25324 Project: Flink Issue Type: New Feature

Re: [VOTE] Release 1.11.6/1.12.7/1.13.5/1.14.2, release candidate #1

2021-12-15 Thread Till Rohrmann
Thanks for driving these releases Chesnay! +1 * Verified checksums and signatures * Reviewed website PR * Checked that build tags only contain log4j version bump In the release-1.14.2-rc1 there are some changes to the create_release_branch.sh script that don't seem necessary. But I think this

[jira] [Created] (FLINK-25323) Support side output late data for interval join

2021-12-15 Thread chenyuzhi (Jira)
chenyuzhi created FLINK-25323: - Summary: Support side output late data for interval join Key: FLINK-25323 URL: https://issues.apache.org/jira/browse/FLINK-25323 Project: Flink Issue Type:

Re: [ANNOUNCE] New Apache Flink Committer - Matthias Pohl

2021-12-15 Thread Stephan Ewen
Congratulations, Matthias, and welcome to the Flink committers! On Mon, Dec 13, 2021 at 4:39 AM 刘建刚 wrote: > Congratulations! > > Best > Liu Jiangang > > Nicholas Jiang 于2021年12月13日周一 11:23写道: > > > Congratulations, Matthias! > > > > Best, > > Nicholas Jiang > > >

Re: [ANNOUNCE] New Apache Flink Committer - Ingo Bürk

2021-12-15 Thread Stephan Ewen
Congrats and welcome! On Mon, Dec 13, 2021 at 4:39 AM 刘建刚 wrote: > Congratulations! > > Best > Liu Jiangang > > Nicholas Jiang 于2021年12月13日周一 11:28写道: > > > Congratulations, Ingo! > > > > Best, > > Nicholas Jiang > > >

Re: [CANCELLED] Release 1.11.5/1.12.6/1.13.4/1.14.1, release candidate #1

2021-12-15 Thread Stephan Ewen
Given that these artifacts are published already, users can use them if they want to update now: For example: https://search.maven.org/artifact/org.apache.flink/flink-core/1.14.1/jar Just for the users that really want to update now (rather than rely on the mitigation via config) and are not as

Re: [VOTE] Release 1.11.6/1.12.7/1.13.5/1.14.2, release candidate #1

2021-12-15 Thread Yun Tang
+ 1 (non-binding) * Select several files of each release to verify the signature and sha512sum. * Reviewed the flink-web PR * checked that those 4 release-branchs only contains fix to bump log4j2 version to 2.15.0 and then to 2.16.0 * Checked the JIRA release notes and found

[jira] [Created] (FLINK-25322) Support no-claim mode in changelog state backend

2021-12-15 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-25322: Summary: Support no-claim mode in changelog state backend Key: FLINK-25322 URL: https://issues.apache.org/jira/browse/FLINK-25322 Project: Flink

[jira] [Created] (FLINK-25321) standalone deploy on k8s,pod always OOM killed,actual heap memory usage is normal, gc is normal

2021-12-15 Thread Gao Fei (Jira)
Gao Fei created FLINK-25321: --- Summary: standalone deploy on k8s,pod always OOM killed,actual heap memory usage is normal, gc is normal Key: FLINK-25321 URL: https://issues.apache.org/jira/browse/FLINK-25321

[jira] [Created] (FLINK-25320) The flink history web page shows the number of running jobs instead of just showing the number of completed jobs

2021-12-15 Thread Zhuang Liu (Jira)
Zhuang Liu created FLINK-25320: --- Summary: The flink history web page shows the number of running jobs instead of just showing the number of completed jobs Key: FLINK-25320 URL:

Re: [VOTE] FLIP-190: Support Version Upgrades for Table API & SQL Programs

2021-12-15 Thread Ingo Bürk
+1 (binding) Thanks for driving this much needed feature! On 14.12.21 17:45, Timo Walther wrote: Hi everyone, I'd like to start a vote on FLIP-190: Support Version Upgrades for Table API & SQL Programs [1] which has been discussed in this thread [2]. The vote will be open for at least 72

[jira] [Created] (FLINK-25319) Quickstarts Scala nightly end-to-end test failed due to akka rpc server failed to start

2021-12-15 Thread Yun Gao (Jira)
Yun Gao created FLINK-25319: --- Summary: Quickstarts Scala nightly end-to-end test failed due to akka rpc server failed to start Key: FLINK-25319 URL: https://issues.apache.org/jira/browse/FLINK-25319