Re: [VOTE] Apache Flink Stateful Functions 3.0.0, release candidate #1

2021-04-07 Thread Tzu-Li (Gordon) Tai
Thanks for voting and testing everyone! We have a total of 6 +1 votes, 3 of which are binding: - Igal Shilman - Gordon Tai (binding) - Konstantin Knauf - Seth Wiesman - Yu Li (binding) - Robert Metzger (binding) I'll proceed now with finalizing the release of StateFun 3.0.0. The official announce

[jira] [Created] (FLINK-22148) Planner rules should use RexCall#equsls to check whether two rexCalls are equivalent

2021-04-07 Thread Shuo Cheng (Jira)
Shuo Cheng created FLINK-22148: -- Summary: Planner rules should use RexCall#equsls to check whether two rexCalls are equivalent Key: FLINK-22148 URL: https://issues.apache.org/jira/browse/FLINK-22148 Proj

Re: [DISCUSS] Backport FLIP-27 Kafka source connector fixes with API change to release-1.12.

2021-04-07 Thread Arvid Heise
Hi Becket, did you need to change anything to the source interface itself? Wouldn't it be possible for users to simply use the 1.13 connector with their Flink 1.12 deployment? I think the late-upgrade argument can be made for any feature, but I also see that the Kafka connector is of high interes

Re: [VOTE] Apache Flink Stateful Functions 3.0.0, release candidate #1

2021-04-07 Thread Robert Metzger
I see. Thanks a lot for clarifying. I then vote +1 (binding) on this release. Thanks a lot for driving this! On Thu, Apr 8, 2021 at 5:19 AM Tzu-Li (Gordon) Tai wrote: > @Robert Metzger > > Sorry, this is the correct link to the class file you are referring to > (previous link I mentioned is

[jira] [Created] (FLINK-22147) Refactor Partition Discovery Logic in KafkaSourceEnumerator

2021-04-07 Thread Qingsheng Ren (Jira)
Qingsheng Ren created FLINK-22147: - Summary: Refactor Partition Discovery Logic in KafkaSourceEnumerator Key: FLINK-22147 URL: https://issues.apache.org/jira/browse/FLINK-22147 Project: Flink

Re: [VOTE] Apache Flink Stateful Functions 3.0.0, release candidate #1

2021-04-07 Thread Tzu-Li (Gordon) Tai
@Robert Metzger Sorry, this is the correct link to the class file you are referring to (previous link I mentioned is incorrect): https://github.com/apache/flink-statefun/blob/master/statefun-flink/statefun-flink-common/src/main/java/com/google/protobuf/MoreByteStrings.java On Thu, Apr 8, 2021 at

Re: [VOTE] Apache Flink Stateful Functions 3.0.0, release candidate #1

2021-04-07 Thread Tzu-Li (Gordon) Tai
@Robert Metzger I assume the com/google/protobuf classfile you found is this one: https://github.com/apache/flink-statefun/blob/master/statefun-shaded/statefun-protobuf-shaded/src/main/java/org/apache/flink/statefun/sdk/shaded/com/google/protobuf/MoreByteStrings.java This actually isn't a class

[jira] [Created] (FLINK-22146) Migrate StateBackend related Chinese docs to latest state backend

2021-04-07 Thread Yun Tang (Jira)
Yun Tang created FLINK-22146: Summary: Migrate StateBackend related Chinese docs to latest state backend Key: FLINK-22146 URL: https://issues.apache.org/jira/browse/FLINK-22146 Project: Flink Is

[jira] [Created] (FLINK-22145) CheckStyle for scala not work

2021-04-07 Thread MaChengLong (Jira)
MaChengLong created FLINK-22145: --- Summary: CheckStyle for scala not work Key: FLINK-22145 URL: https://issues.apache.org/jira/browse/FLINK-22145 Project: Flink Issue Type: Improvement

Re: [DISCUSS] Backport FLIP-27 Kafka source connector fixes with API change to release-1.12.

2021-04-07 Thread Becket Qin
Thanks for the comment, Till and Thomas. As far as I know there are some users who have just upgraded their Flink version from 1.8 / 1.9 to Flink 1.12 and might not upgrade the version in 6 months or more. There are also some organizations that have the strategy of not running the latest version o

Re: Zigzag shape in TM JVM used memory

2021-04-07 Thread Lu Niu
Hi, Piotr Thanks for replying. I asked this because such a pattern might imply memory oversubscription. For example, I tuned down the memory of one app from heap 2.63GB to 367MB and the job still runs fine: before: https://drive.google.com/file/d/1o8k9Vv3yb5gXITi4GvmlXMteQcRfmOhr/view?usp=sharing

Re: [VOTE] Apache Flink Stateful Functions 3.0.0, release candidate #1

2021-04-07 Thread Robert Metzger
This jar contains a com/google/protobuf classfile, which is not declared in any NOTICE file (and doesn't ship the license file of protobuf): https://repository.apache.org/content/repositories/orgapacheflink-1415/org/apache/flink/statefun-flink-common/3.0.0/statefun-flink-common-3.0.0.jar I fear th

Re: [DISCUSS] Backport FLIP-27 Kafka source connector fixes with API change to release-1.12.

2021-04-07 Thread Thomas Weise
Hi, Thanks for fixing the new KafkaSource issues. I'm interested in using these fixes with 1.12 for experimental purposes. +1 for backporting. 1.12 is the current stable release and users who would like to try the FLIP-27 sources are likely to use that release. Thomas On Wed, Apr 7, 2021 at 2:

[jira] [Created] (FLINK-22144) Test display last n exceptions/causes for job restarts in Web UI

2021-04-07 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-22144: - Summary: Test display last n exceptions/causes for job restarts in Web UI Key: FLINK-22144 URL: https://issues.apache.org/jira/browse/FLINK-22144 Project: Flink

[ANNOUNCE] Release 1.13 branch

2021-04-07 Thread Dawid Wysakowicz
Hi committers! I'd like to remind that we agreed we won't cut off the 1.13 branch until we are a bit more confident with the stability of the master branch. Therefore I'd kindly ask not to push the release-1.13 branch yet. I did push that branch accidentally when creating the rc0, but I reported

[jira] [Created] (FLINK-22143) Flink returns less rows than expected when using limit in SQL

2021-04-07 Thread Peng Yu (Jira)
Peng Yu created FLINK-22143: --- Summary: Flink returns less rows than expected when using limit in SQL Key: FLINK-22143 URL: https://issues.apache.org/jira/browse/FLINK-22143 Project: Flink Issue Ty

[jira] [Created] (FLINK-22142) Remove console logging for Kafka connector for AZP runs

2021-04-07 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-22142: - Summary: Remove console logging for Kafka connector for AZP runs Key: FLINK-22142 URL: https://issues.apache.org/jira/browse/FLINK-22142 Project: Flink Iss

[jira] [Created] (FLINK-22141) Manually test exactly-once JDBC sink

2021-04-07 Thread Roman Khachatryan (Jira)
Roman Khachatryan created FLINK-22141: - Summary: Manually test exactly-once JDBC sink Key: FLINK-22141 URL: https://issues.apache.org/jira/browse/FLINK-22141 Project: Flink Issue Type: Te

[jira] [Created] (FLINK-22140) Test the unified binary savepoint

2021-04-07 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-22140: Summary: Test the unified binary savepoint Key: FLINK-22140 URL: https://issues.apache.org/jira/browse/FLINK-22140 Project: Flink Issue Type: Task

[ANNOUNCE] Apache Flink-shaded 13.0 released

2021-04-07 Thread Chesnay Schepler
The Apache Flink community is very happy to announce the release of Apache Flink-shaded 13.0. The flink-shaded project contains a number of shaded dependencies for Apache Flink. Apache FlinkĀ® is an open-source stream processing framework for distributed, high-performing, always-available, an

[jira] [Created] (FLINK-22139) Flink Jobmanager & Task Manger logs are not writing to the logs files

2021-04-07 Thread Bhagi (Jira)
Bhagi created FLINK-22139: - Summary: Flink Jobmanager & Task Manger logs are not writing to the logs files Key: FLINK-22139 URL: https://issues.apache.org/jira/browse/FLINK-22139 Project: Flink Issu

Re: [DISCUSS] Backport FLIP-27 Kafka source connector fixes with API change to release-1.12.

2021-04-07 Thread Till Rohrmann
Hi Becket, If I remember correctly, then we deliberately not documented the Kafka connector in the 1.12 release. Hence, from this point there should be no need to backport any fixes because users are not aware of this feature. On the other hand this also means that we should be able to break anyt

[jira] [Created] (FLINK-22138) Better support structured types as toDataStream output

2021-04-07 Thread Timo Walther (Jira)
Timo Walther created FLINK-22138: Summary: Better support structured types as toDataStream output Key: FLINK-22138 URL: https://issues.apache.org/jira/browse/FLINK-22138 Project: Flink Issue

[jira] [Created] (FLINK-22137) Execute unaligned checkpoint test on a cluster

2021-04-07 Thread Arvid Heise (Jira)
Arvid Heise created FLINK-22137: --- Summary: Execute unaligned checkpoint test on a cluster Key: FLINK-22137 URL: https://issues.apache.org/jira/browse/FLINK-22137 Project: Flink Issue Type: Sub-

[jira] [Created] (FLINK-22136) Device application for unaligned checkpoint test on cluster

2021-04-07 Thread Arvid Heise (Jira)
Arvid Heise created FLINK-22136: --- Summary: Device application for unaligned checkpoint test on cluster Key: FLINK-22136 URL: https://issues.apache.org/jira/browse/FLINK-22136 Project: Flink Is

[jira] [Created] (FLINK-22135) Test the adaptive scheduler

2021-04-07 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-22135: - Summary: Test the adaptive scheduler Key: FLINK-22135 URL: https://issues.apache.org/jira/browse/FLINK-22135 Project: Flink Issue Type: Task Comp

[jira] [Created] (FLINK-22134) Test the reactive mode

2021-04-07 Thread Till Rohrmann (Jira)
Till Rohrmann created FLINK-22134: - Summary: Test the reactive mode Key: FLINK-22134 URL: https://issues.apache.org/jira/browse/FLINK-22134 Project: Flink Issue Type: Task Component

[DISCUSS] Backport FLIP-27 Kafka source connector fixes with API change to release-1.12.

2021-04-07 Thread Becket Qin
Hi folks, I'd like to start a discussion thread about backporting some FLIP-27 Kafka source connector fixes to release-1.12. These fixes include some API changes and thus needs a public discussion. The tickets in question are following: https://issues.apache.org/jira/browse/FLINK-20379 https://is

[jira] [Created] (FLINK-22133) SplitEmumerator does not provide checkpoint id in snapshot

2021-04-07 Thread Brian Zhou (Jira)
Brian Zhou created FLINK-22133: -- Summary: SplitEmumerator does not provide checkpoint id in snapshot Key: FLINK-22133 URL: https://issues.apache.org/jira/browse/FLINK-22133 Project: Flink Issue

[jira] [Created] (FLINK-22132) Test unaligned checkpoints rescaling manually on a real cluster

2021-04-07 Thread Piotr Nowojski (Jira)
Piotr Nowojski created FLINK-22132: -- Summary: Test unaligned checkpoints rescaling manually on a real cluster Key: FLINK-22132 URL: https://issues.apache.org/jira/browse/FLINK-22132 Project: Flink

[jira] [Created] (FLINK-22131) Fix the bug of general udf and pandas udf chained together in map operation

2021-04-07 Thread Huang Xingbo (Jira)
Huang Xingbo created FLINK-22131: Summary: Fix the bug of general udf and pandas udf chained together in map operation Key: FLINK-22131 URL: https://issues.apache.org/jira/browse/FLINK-22131 Project:

[jira] [Created] (FLINK-22130) Test row based operations in Python Table API

2021-04-07 Thread Huang Xingbo (Jira)
Huang Xingbo created FLINK-22130: Summary: Test row based operations in Python Table API Key: FLINK-22130 URL: https://issues.apache.org/jira/browse/FLINK-22130 Project: Flink Issue Type: Tes