[jira] [Created] (FLINK-20671) Partition doesn't commit until the end of partition

2020-12-17 Thread zhuxiaoshang (Jira)
zhuxiaoshang created FLINK-20671: Summary: Partition doesn't commit until the end of partition Key: FLINK-20671 URL: https://issues.apache.org/jira/browse/FLINK-20671 Project: Flink Issue Typ

Re: [DISCUSS] Releasing Apache Flink 1.12.1

2020-12-17 Thread Tzu-Li (Gordon) Tai
Thanks Xintong for driving this. I'd like to make two more issues related to the Kinesis connector changes in 1.12.0 a blocker for 1.12.1: https://issues.apache.org/jira/browse/FLINK-20630 https://issues.apache.org/jira/browse/FLINK-20629 There are already PRs for these issues from @Cranmer, Dann

[jira] [Created] (FLINK-20670) Support query hints for Flink SQL

2020-12-17 Thread Shuo Cheng (Jira)
Shuo Cheng created FLINK-20670: -- Summary: Support query hints for Flink SQL Key: FLINK-20670 URL: https://issues.apache.org/jira/browse/FLINK-20670 Project: Flink Issue Type: New Feature

Re: [DISCUSS] FLIP-153: Support state access in Python DataStream API

2020-12-17 Thread Shuiqiang Chen
Hi wei, Big thanks for pointing out the mistakes! I have updated the FLIP according to your suggestions. Best, Shuiqiang > 在 2020年12月18日,下午2:37,Wei Zhong 写道: > > Hi Shuiqiang, > > Thanks for driving this. +1 for this feature, just a minor comment to the > design doc. > > The interface of t

Re: [DISCUSS] FLIP-153: Support state access in Python DataStream API

2020-12-17 Thread Wei Zhong
Hi Shuiqiang, Thanks for driving this. +1 for this feature, just a minor comment to the design doc. The interface of the `AppendingState` should be: class AppendingState(State, Generic[IN, OUT]): @abstractmethod def get(self) -> OUT: pass @abstractmethod def add(self, value

Re: [DISCUSS] Releasing Apache Flink 1.12.1

2020-12-17 Thread Guowei Ma
Thanks for driving this release Xintong. I think https://issues.apache.org/jira/browse/FLINK-20652 should be addressed. Best, Guowei On Fri, Dec 18, 2020 at 11:53 AM Jingsong Li wrote: > Thanks for volunteering as our release manager Xintong. +1 for releasing > Flink 1.12.1 soon. > > I think h

Re: [DISCUSS] Releasing Apache Flink 1.12.1

2020-12-17 Thread Jingsong Li
Thanks for volunteering as our release manager Xintong. +1 for releasing Flink 1.12.1 soon. I think https://issues.apache.org/jira/browse/FLINK-20665 should be addressed, I marked it as a Blocker. Best, Jingsong On Fri, Dec 18, 2020 at 11:16 AM Yang Wang wrote: > Hi David, > > I will take a lo

[jira] [Created] (FLINK-20669) Add the jzlib LICENSE file in flink-python module

2020-12-17 Thread Huang Xingbo (Jira)
Huang Xingbo created FLINK-20669: Summary: Add the jzlib LICENSE file in flink-python module Key: FLINK-20669 URL: https://issues.apache.org/jira/browse/FLINK-20669 Project: Flink Issue Type:

[jira] [Created] (FLINK-20668) Introduce translateToExecNode method for FlinkPhysicalRel

2020-12-17 Thread godfrey he (Jira)
godfrey he created FLINK-20668: -- Summary: Introduce translateToExecNode method for FlinkPhysicalRel Key: FLINK-20668 URL: https://issues.apache.org/jira/browse/FLINK-20668 Project: Flink Issue T

Re: [DISCUSS] Releasing Apache Flink 1.12.1

2020-12-17 Thread Yang Wang
Hi David, I will take a look this ticket FLINK-20648 and try to get it resolved in this release cycle. @Xintong Song One more Kubernetes HA related issue. We need to support setting service account for TaskManager pod[1]. Even though we have a work around for this issue, but it is not acceptable

[jira] [Created] (FLINK-20667) Configurable batch size for vectorized reader

2020-12-17 Thread Rui Li (Jira)
Rui Li created FLINK-20667: -- Summary: Configurable batch size for vectorized reader Key: FLINK-20667 URL: https://issues.apache.org/jira/browse/FLINK-20667 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-20666) Fix the deserialized Row losing the field_name information in PyFlink

2020-12-17 Thread Huang Xingbo (Jira)
Huang Xingbo created FLINK-20666: Summary: Fix the deserialized Row losing the field_name information in PyFlink Key: FLINK-20666 URL: https://issues.apache.org/jira/browse/FLINK-20666 Project: Flink

[jira] [Created] (FLINK-20665) FileNotFoundException when restore from latest Checkpoint

2020-12-17 Thread zhuxiaoshang (Jira)
zhuxiaoshang created FLINK-20665: Summary: FileNotFoundException when restore from latest Checkpoint Key: FLINK-20665 URL: https://issues.apache.org/jira/browse/FLINK-20665 Project: Flink Iss

[jira] [Created] (FLINK-20664) Support setting service account for TaskManager pod

2020-12-17 Thread Yang Wang (Jira)
Yang Wang created FLINK-20664: - Summary: Support setting service account for TaskManager pod Key: FLINK-20664 URL: https://issues.apache.org/jira/browse/FLINK-20664 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-20663) Managed memory may not be released in time when operators use managed memory frequently

2020-12-17 Thread Caizhi Weng (Jira)
Caizhi Weng created FLINK-20663: --- Summary: Managed memory may not be released in time when operators use managed memory frequently Key: FLINK-20663 URL: https://issues.apache.org/jira/browse/FLINK-20663

[jira] [Created] (FLINK-20662) UnalignedCheckpointITCase.execute failed with IndexOutOfBoundsException

2020-12-17 Thread Huang Xingbo (Jira)
Huang Xingbo created FLINK-20662: Summary: UnalignedCheckpointITCase.execute failed with IndexOutOfBoundsException Key: FLINK-20662 URL: https://issues.apache.org/jira/browse/FLINK-20662 Project: Flin

[jira] [Created] (FLINK-20661) Elasticsearch6DynamicSinkITCase.testWritingDocuments test failed with ConnectException

2020-12-17 Thread Huang Xingbo (Jira)
Huang Xingbo created FLINK-20661: Summary: Elasticsearch6DynamicSinkITCase.testWritingDocuments test failed with ConnectException Key: FLINK-20661 URL: https://issues.apache.org/jira/browse/FLINK-20661

[jira] [Created] (FLINK-20660) Time window operator with computed column triggers an exception in batch mode

2020-12-17 Thread sujun (Jira)
sujun created FLINK-20660: - Summary: Time window operator with computed column triggers an exception in batch mode Key: FLINK-20660 URL: https://issues.apache.org/jira/browse/FLINK-20660 Project: Flink

[jira] [Created] (FLINK-20659) YARNSessionCapacitySchedulerITCase.perJobYarnClusterOffHeap test failed with NPE

2020-12-17 Thread Huang Xingbo (Jira)
Huang Xingbo created FLINK-20659: Summary: YARNSessionCapacitySchedulerITCase.perJobYarnClusterOffHeap test failed with NPE Key: FLINK-20659 URL: https://issues.apache.org/jira/browse/FLINK-20659 Proj

Re: [DISCUSS] Releasing Apache Flink 1.12.1

2020-12-17 Thread David Morávek
Hi, I think https://issues.apache.org/jira/browse/FLINK-20648 should be addressed, as Kubernetes HA was one of the main selling points of this release. WDYT? D. Sent from my iPhone > On 17. 12. 2020, at 13:54, Yun Tang wrote: > > Thanks for driving this quick-fix release. > +1 for fixing th

[jira] [Created] (FLINK-20658) Make sure jdbc connection is established by target driver

2020-12-17 Thread Kezhu Wang (Jira)
Kezhu Wang created FLINK-20658: -- Summary: Make sure jdbc connection is established by target driver Key: FLINK-20658 URL: https://issues.apache.org/jira/browse/FLINK-20658 Project: Flink Issue T

[jira] [Created] (FLINK-20657) Migrate jdbc connection code of JdbcInputFormat, JdbcLookupFunction, JdbcRowDataLookupFunction to SimpleJdbcConnectionProvider

2020-12-17 Thread Kezhu Wang (Jira)
Kezhu Wang created FLINK-20657: -- Summary: Migrate jdbc connection code of JdbcInputFormat, JdbcLookupFunction, JdbcRowDataLookupFunction to SimpleJdbcConnectionProvider Key: FLINK-20657 URL: https://issues.apache.org

[jira] [Created] (FLINK-20656) Update docs for new KafkaSource connector.

2020-12-17 Thread Jiangjie Qin (Jira)
Jiangjie Qin created FLINK-20656: Summary: Update docs for new KafkaSource connector. Key: FLINK-20656 URL: https://issues.apache.org/jira/browse/FLINK-20656 Project: Flink Issue Type: Task

[jira] [Created] (FLINK-20655) Add E2E tests to the new KafkaSource based on FLIP-27.

2020-12-17 Thread Jiangjie Qin (Jira)
Jiangjie Qin created FLINK-20655: Summary: Add E2E tests to the new KafkaSource based on FLIP-27. Key: FLINK-20655 URL: https://issues.apache.org/jira/browse/FLINK-20655 Project: Flink Issue

[jira] [Created] (FLINK-20654) Unaligned checkpoint recovery may lead to corrupted data stream

2020-12-17 Thread Arvid Heise (Jira)
Arvid Heise created FLINK-20654: --- Summary: Unaligned checkpoint recovery may lead to corrupted data stream Key: FLINK-20654 URL: https://issues.apache.org/jira/browse/FLINK-20654 Project: Flink

Re: [DISCUSS] FLIP-153: Support state access in Python DataStream API

2020-12-17 Thread Shuiqiang Chen
Hi Yun, Highly appreciate for your questions! I have the corresponding answers as bellow: Re 1: You are right that the state access occurs in an async thread. However, all the state access will be synchrouzed in the Java operator and so there will be no concurrent access to the state backend.

Re: [DISCUSS] Releasing Apache Flink 1.12.1

2020-12-17 Thread Yun Tang
Thanks for driving this quick-fix release. +1 for fixing the bug of RocksDB state-backend with reduce operators. Best Yun Tang From: Till Rohrmann Sent: Thursday, December 17, 2020 20:51 To: dev Subject: Re: [DISCUSS] Releasing Apache Flink 1.12.1 Thanks for vol

Re: [DISCUSS] Releasing Apache Flink 1.12.1

2020-12-17 Thread Till Rohrmann
Thanks for volunteering as our release manager Xintong. +1 for a swift bug fix release. Cheers, Till On Thu, Dec 17, 2020 at 1:20 PM Xintong Song wrote: > Hi devs, > > It's been one week since we announced Apache Flink 1.12.0, and there are > already many issues reported, some of which are quit

[jira] [Created] (FLINK-20653) SharedState should be registered after PendingCheckpoint storeCheckpoint

2020-12-17 Thread weiyunqing (Jira)
weiyunqing created FLINK-20653: -- Summary: SharedState should be registered after PendingCheckpoint storeCheckpoint Key: FLINK-20653 URL: https://issues.apache.org/jira/browse/FLINK-20653 Project: Flink

[DISCUSS] Releasing Apache Flink 1.12.1

2020-12-17 Thread Xintong Song
Hi devs, It's been one week since we announced Apache Flink 1.12.0, and there are already many issues reported, some of which are quite critical. Thus, I would like to start a discussion on releasing Flink 1.12.1 soon. I would like to volunteer for managing this release. I've noticed the followi

[jira] [Created] (FLINK-20652) Improve the document for making the user could write a DataStream job that could be execute in the batch execution mode.

2020-12-17 Thread Guowei Ma (Jira)
Guowei Ma created FLINK-20652: - Summary: Improve the document for making the user could write a DataStream job that could be execute in the batch execution mode. Key: FLINK-20652 URL: https://issues.apache.org/jira/br

Re: [DISCUSS] Enforce common opinionated coding style using Spotless

2020-12-17 Thread Aljoscha Krettek
Thanks for the broad support! I created a Jira issue: https://issues.apache.org/jira/browse/FLINK-20651 For the timing, I agree with Piotr. Best, Aljoscha On 17.12.20 11:45, Piotr Nowojski wrote: Hi, +1 for this kind of approach (with reformatting the code base). Maybe ideally it would be b

[jira] [Created] (FLINK-20651) Use Spotless/google-java-format for code formatting/enforcement

2020-12-17 Thread Aljoscha Krettek (Jira)
Aljoscha Krettek created FLINK-20651: Summary: Use Spotless/google-java-format for code formatting/enforcement Key: FLINK-20651 URL: https://issues.apache.org/jira/browse/FLINK-20651 Project: Flin

Re: [DISCUSS] Enforce common opinionated coding style using Spotless

2020-12-17 Thread Piotr Nowojski
Hi, +1 for this kind of approach (with reformatting the code base). Maybe ideally it would be best to do it either next week, or between December 24th and January 1st? Maybe such timing would minimize the number of open PRs that would need to be rewritten/modified to comply? I'm afraid pending PR

[jira] [Created] (FLINK-20650) Rename "native-k8s" command to "generic" in docker-entrypoint.sh

2020-12-17 Thread Yang Wang (Jira)
Yang Wang created FLINK-20650: - Summary: Rename "native-k8s" command to "generic" in docker-entrypoint.sh Key: FLINK-20650 URL: https://issues.apache.org/jira/browse/FLINK-20650 Project: Flink I

[jira] [Created] (FLINK-20649) StreamTask closeAllOperators stuck

2020-12-17 Thread weiyunqing (Jira)
weiyunqing created FLINK-20649: -- Summary: StreamTask closeAllOperators stuck Key: FLINK-20649 URL: https://issues.apache.org/jira/browse/FLINK-20649 Project: Flink Issue Type: Bug Comp

[jira] [Created] (FLINK-20648) Unable to restore job from savepoint when using Kubernetes based HA services

2020-12-17 Thread Jira
David Morávek created FLINK-20648: - Summary: Unable to restore job from savepoint when using Kubernetes based HA services Key: FLINK-20648 URL: https://issues.apache.org/jira/browse/FLINK-20648 Projec

[jira] [Created] (FLINK-20647) Use yield to generate output datas in ProcessFunction for Python DataStream

2020-12-17 Thread Shuiqiang Chen (Jira)
Shuiqiang Chen created FLINK-20647: -- Summary: Use yield to generate output datas in ProcessFunction for Python DataStream Key: FLINK-20647 URL: https://issues.apache.org/jira/browse/FLINK-20647 Proje

[jira] [Created] (FLINK-20646) ReduceTransformation does not work with RocksDBStateBackend

2020-12-17 Thread Xintong Song (Jira)
Xintong Song created FLINK-20646: Summary: ReduceTransformation does not work with RocksDBStateBackend Key: FLINK-20646 URL: https://issues.apache.org/jira/browse/FLINK-20646 Project: Flink

Re: [DISCUSS] Enforce common opinionated coding style using Spotless

2020-12-17 Thread Till Rohrmann
+1 Cheers, Till On Thu, Dec 17, 2020 at 7:59 AM Tzu-Li (Gordon) Tai wrote: > +1 > > On Thu, Dec 17, 2020, 2:56 PM Zhu Zhu wrote: > > > +1. It can be very helpful for future development. Thanks for driving > this! > > > > Thanks, > > Zhu > > > > Yangze Guo 于2020年12月17日周四 下午2:48写道: > > > > > +1

[jira] [Created] (FLINK-20645) Extra space typo when assembling dynamic configs

2020-12-17 Thread Li Wang (Jira)
Li Wang created FLINK-20645: --- Summary: Extra space typo when assembling dynamic configs Key: FLINK-20645 URL: https://issues.apache.org/jira/browse/FLINK-20645 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-20644) table api use UDF(ScalarFunction ) if eval return Unit

2020-12-17 Thread shiyu (Jira)
shiyu created FLINK-20644: - Summary: table api use UDF(ScalarFunction ) if eval return Unit Key: FLINK-20644 URL: https://issues.apache.org/jira/browse/FLINK-20644 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-20643) flink-connector-elasticsearch7 causes Direct buffer memory OOM

2020-12-17 Thread Echo Lee (Jira)
Echo Lee created FLINK-20643: Summary: flink-connector-elasticsearch7 causes Direct buffer memory OOM Key: FLINK-20643 URL: https://issues.apache.org/jira/browse/FLINK-20643 Project: Flink Issue