[jira] [Created] (FLINK-20642) Introduce InternalRow to optimize Row used in Python UDAF

2020-12-16 Thread Huang Xingbo (Jira)
Huang Xingbo created FLINK-20642: Summary: Introduce InternalRow to optimize Row used in Python UDAF Key: FLINK-20642 URL: https://issues.apache.org/jira/browse/FLINK-20642 Project: Flink

[jira] [Created] (FLINK-20641) flink-connector-elasticsearch6 will deadlock

2020-12-16 Thread Echo Lee (Jira)
Echo Lee created FLINK-20641: Summary: flink-connector-elasticsearch6 will deadlock Key: FLINK-20641 URL: https://issues.apache.org/jira/browse/FLINK-20641 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-20640) Use cython to optimize the logic of Python UDAF

2020-12-16 Thread Huang Xingbo (Jira)
Huang Xingbo created FLINK-20640: Summary: Use cython to optimize the logic of Python UDAF Key: FLINK-20640 URL: https://issues.apache.org/jira/browse/FLINK-20640 Project: Flink Issue Type:

[jira] [Created] (FLINK-20639) Use list to optimize the Row used by Python UDAF intermediate results

2020-12-16 Thread Huang Xingbo (Jira)
Huang Xingbo created FLINK-20639: Summary: Use list to optimize the Row used by Python UDAF intermediate results Key: FLINK-20639 URL: https://issues.apache.org/jira/browse/FLINK-20639 Project: Flink

Re: [DISCUSS] Enforce common opinionated coding style using Spotless

2020-12-16 Thread Tzu-Li (Gordon) Tai
+1 On Thu, Dec 17, 2020, 2:56 PM Zhu Zhu wrote: > +1. It can be very helpful for future development. Thanks for driving this! > > Thanks, > Zhu > > Yangze Guo 于2020年12月17日周四 下午2:48写道: > > > +1 > > Thanks for driving this! > > > > Best, > > Yangze Guo > > > > On Thu, Dec 17, 2020 at 2:39 PM

Re: [DISCUSS] Enforce common opinionated coding style using Spotless

2020-12-16 Thread Zhu Zhu
+1. It can be very helpful for future development. Thanks for driving this! Thanks, Zhu Yangze Guo 于2020年12月17日周四 下午2:48写道: > +1 > Thanks for driving this! > > Best, > Yangze Guo > > On Thu, Dec 17, 2020 at 2:39 PM Igal Shilman wrote: > > > > +1 it works really well in StateFun for quite some

Re: [DISCUSS] Enforce common opinionated coding style using Spotless

2020-12-16 Thread Yangze Guo
+1 Thanks for driving this! Best, Yangze Guo On Thu, Dec 17, 2020 at 2:39 PM Igal Shilman wrote: > > +1 it works really well in StateFun for quite some time. > > On Thursday, December 17, 2020, Wei Zhong wrote: > > > +1 for the coding style automation. Thanks for driving this! > > > > Best, >

Re: [DISCUSS] Enforce common opinionated coding style using Spotless

2020-12-16 Thread Igal Shilman
+1 it works really well in StateFun for quite some time. On Thursday, December 17, 2020, Wei Zhong wrote: > +1 for the coding style automation. Thanks for driving this! > > Best, > Wei > > > 在 2020年12月17日,10:10,Xingbo Huang 写道: > > > > +1 asap. Spotless can greatly help us save the time of

[jira] [Created] (FLINK-20638) Flink hive modules Function with the same name bug

2020-12-16 Thread HideOnBush (Jira)
HideOnBush created FLINK-20638: -- Summary: Flink hive modules Function with the same name bug Key: FLINK-20638 URL: https://issues.apache.org/jira/browse/FLINK-20638 Project: Flink Issue Type:

[jira] [Created] (FLINK-20637) Table convert to dataStream twice will result in two data streams

2020-12-16 Thread Wu (Jira)
Wu created FLINK-20637: -- Summary: Table convert to dataStream twice will result in two data streams Key: FLINK-20637 URL: https://issues.apache.org/jira/browse/FLINK-20637 Project: Flink Issue Type:

[jira] [Created] (FLINK-20636) Require unaligned checkpoints to be disabled in StateFun applications

2020-12-16 Thread Tzu-Li (Gordon) Tai (Jira)
Tzu-Li (Gordon) Tai created FLINK-20636: --- Summary: Require unaligned checkpoints to be disabled in StateFun applications Key: FLINK-20636 URL: https://issues.apache.org/jira/browse/FLINK-20636

Re: [DISCUSS] FLIP-153: Support state access in Python DataStream API

2020-12-16 Thread Yun Tang
Hi Shuiqiang, Thanks for driving this. I have several questions below: 1. Thread safety of state write-access. As you might know, state access is not thread-safe [1] in Flink, we depend on task single thread access. Since you change the state access to another async thread, can we still

[RESULT] [VOTE] Release 1.11.3, release candidate #2

2020-12-16 Thread Xintong Song
Hi everyone, I'm happy to announce that we have unanimously approved this release. There are 9 approving votes, 3 of which are binding: * Tzu-Li (Gordon) Tai (binding) * Xingbo Huang (non-binding) * Yang Wang (non-binding) * Xintong Song (non-binding) * Dian Fu (binding) * Wei Zhong

Re: Re: [VOTE] Release 1.11.3, release candidate #2

2020-12-16 Thread Xintong Song
Thanks everyone for the verifications and votes. The vote has been opened for more than 72 hours and we have collected 3 binding votes. I'm closing this vote in a separate thread. Thank you~ Xintong Song On Thu, Dec 17, 2020 at 10:33 AM hailongwang <18868816...@163.com> wrote: > +1

[jira] [Created] (FLINK-20635) Scala REPL with java.lang.NoSuchMethod: Errorjava.lang.NoSuchMethodError

2020-12-16 Thread Lu Guoqing (Jira)
Lu Guoqing created FLINK-20635: -- Summary: Scala REPL with java.lang.NoSuchMethod: Errorjava.lang.NoSuchMethodError Key: FLINK-20635 URL: https://issues.apache.org/jira/browse/FLINK-20635 Project: Flink

Re: [DISCUSS] Enforce common opinionated coding style using Spotless

2020-12-16 Thread Wei Zhong
+1 for the coding style automation. Thanks for driving this! Best, Wei > 在 2020年12月17日,10:10,Xingbo Huang 写道: > > +1 asap. Spotless can greatly help us save the time of fixing checkstyle > errors. > > Best, > Xingbo > > Kostas Kloudas 于2020年12月17日周四 上午4:14写道: > >> +1 asap from my side as

[jira] [Created] (FLINK-20634) Flink with Scala 2.12 that scala shell is not supported still provides start-scala-shell.sh

2020-12-16 Thread Yao Zhang (Jira)
Yao Zhang created FLINK-20634: - Summary: Flink with Scala 2.12 that scala shell is not supported still provides start-scala-shell.sh Key: FLINK-20634 URL: https://issues.apache.org/jira/browse/FLINK-20634

Re:Re: [VOTE] Release 1.11.3, release candidate #2

2020-12-16 Thread hailongwang
+1 (non-binding) - check checksum and signature - build source code with tests and checkstyle included - use sql-client testing some sql jobs, and the results are as expected. Best, Hailong At 2020-12-16 22:50:34, "Timo Walther" wrote: >+1 (binding) > >I manually scanned the commit diff and

[jira] [Created] (FLINK-20633) test_configuration. test_add_all test failed in py35

2020-12-16 Thread Huang Xingbo (Jira)
Huang Xingbo created FLINK-20633: Summary: test_configuration. test_add_all test failed in py35 Key: FLINK-20633 URL: https://issues.apache.org/jira/browse/FLINK-20633 Project: Flink Issue

Re: [DISCUSS] Enforce common opinionated coding style using Spotless

2020-12-16 Thread Xingbo Huang
+1 asap. Spotless can greatly help us save the time of fixing checkstyle errors. Best, Xingbo Kostas Kloudas 于2020年12月17日周四 上午4:14写道: > +1 asap from my side as well. > > On Wed, Dec 16, 2020 at 8:04 PM Arvid Heise wrote: > > > > +1 asap. > > > > On Wed, Dec 16, 2020 at 7:44 PM Robert Metzger

Re: Changing application configuration when restoring from checkpoint/savepoint

2020-12-16 Thread vishalovercome
Will this work - In main method, serialize config into a string and store it using ParameterTool with key as taskName and value as config (serialized as string). Then in the open method, lookup the relevant configuration using getTaskName(). A follow up to this would be configuring custom

Will job manager restarts lead to repeated savepoint restoration?

2020-12-16 Thread vishalovercome
My flink job runs in kubernetes. This is the setup: 1. One job running as a job cluster with one job manager 2. HA powered by zookeeper (works fine) 3. Job/Deployment manifests stored in Github and deployed to kubernetes by Argo 4. State persisted to S3 If I were to stop (drain and take a

Changing application configuration when restoring from checkpoint/savepoint

2020-12-16 Thread vishalovercome
My flink job loads several configuration files that contain job, operator and business configuration. One of the operators is an AsyncOperator with function like so: class AsyncFun(config: T) extends RichAsyncFunction[X, Y] { @transient private lazy val client = f(config, metricGroup, etc.)

Re: [DISCUSS] Enforce common opinionated coding style using Spotless

2020-12-16 Thread Kostas Kloudas
+1 asap from my side as well. On Wed, Dec 16, 2020 at 8:04 PM Arvid Heise wrote: > > +1 asap. > > On Wed, Dec 16, 2020 at 7:44 PM Robert Metzger wrote: > > > +1 > > > > Thanks for driving this. > > > > On Wed, Dec 16, 2020 at 7:33 PM Chesnay Schepler > > wrote: > > > > > +1 to set this up

[jira] [Created] (FLINK-20632) Missing docker images for 1.12 releas

2020-12-16 Thread Piotr Gwiazda (Jira)
Piotr Gwiazda created FLINK-20632: - Summary: Missing docker images for 1.12 releas Key: FLINK-20632 URL: https://issues.apache.org/jira/browse/FLINK-20632 Project: Flink Issue Type: Bug

Re: [DISCUSS] Enforce common opinionated coding style using Spotless

2020-12-16 Thread Arvid Heise
+1 asap. On Wed, Dec 16, 2020 at 7:44 PM Robert Metzger wrote: > +1 > > Thanks for driving this. > > On Wed, Dec 16, 2020 at 7:33 PM Chesnay Schepler > wrote: > > > +1 to set this up ASAP. Now's a good time to (finally) finalize this > > effort with a new release cycle having started and

Re: [DISCUSS] Enforce common opinionated coding style using Spotless

2020-12-16 Thread Thomas Weise
+1 for better coding style automation I see spotless works very well in other projects. On Wed, Dec 16, 2020 at 10:45 AM Robert Metzger wrote: > +1 > > Thanks for driving this. > > On Wed, Dec 16, 2020 at 7:33 PM Chesnay Schepler > wrote: > > > +1 to set this up ASAP. Now's a good time to

Re: [DISCUSS] Enforce common opinionated coding style using Spotless

2020-12-16 Thread Robert Metzger
+1 Thanks for driving this. On Wed, Dec 16, 2020 at 7:33 PM Chesnay Schepler wrote: > +1 to set this up ASAP. Now's a good time to (finally) finalize this > effort with a new release cycle having started and christmas/vacations > being around the corner. > > On 12/16/2020 7:20 PM, Aljoscha

Re: [DISCUSS] Enforce common opinionated coding style using Spotless

2020-12-16 Thread Chesnay Schepler
+1 to set this up ASAP. Now's a good time to (finally) finalize this effort with a new release cycle having started and christmas/vacations being around the corner. On 12/16/2020 7:20 PM, Aljoscha Krettek wrote: Let's try and conclude this discussion! I've prepared a PoC that uses Spotless

Re: [DISCUSS] Enforce common opinionated coding style using Spotless

2020-12-16 Thread Aljoscha Krettek
Let's try and conclude this discussion! I've prepared a PoC that uses Spotless with google-java-format to do the formatting: https://github.com/aljoscha/flink/commits/flink-xxx-spotless To summarize from earlier discussion, the main benefits are: - no more worrying about code style, both as

[jira] [Created] (FLINK-20631) cannot find native-k8s when deploying flink using native kubernetes

2020-12-16 Thread renming.qi (Jira)
renming.qi created FLINK-20631: -- Summary: cannot find native-k8s when deploying flink using native kubernetes Key: FLINK-20631 URL: https://issues.apache.org/jira/browse/FLINK-20631 Project: Flink

Re: [VOTE] Release 1.11.3, release candidate #2

2020-12-16 Thread Timo Walther
+1 (binding) I manually scanned the commit diff and checked for incompatible changes: - checked for changed @Public(Evolving) signatures - checked for SQL plan changes - checked for newly introduced features What I found a bit concerning is the large number of commits around the new source

Re: Kafka Consumer Deserializer Exception on application mode

2020-12-16 Thread Aljoscha Krettek
I believe this is caused by dependency conflicts/mismatch. I also commented this on the Jira issue. Best, Aljoscha On 16.12.20 07:39, han guoguo wrote: Hi, Kafka source may has some issues on application mode when i run it with application mode on flink 1.11.2 it can't startup the detail

[jira] [Created] (FLINK-20630) [Kinesis][DynamoDB] DynamoDB Consumer fails to consume from Latest

2020-12-16 Thread Danny Cranmer (Jira)
Danny Cranmer created FLINK-20630: - Summary: [Kinesis][DynamoDB] DynamoDB Consumer fails to consume from Latest Key: FLINK-20630 URL: https://issues.apache.org/jira/browse/FLINK-20630 Project: Flink

[jira] [Created] (FLINK-20629) [Kinesis][EFO] Migrate from DescribeStream to DescribeStreamSummary

2020-12-16 Thread Danny Cranmer (Jira)
Danny Cranmer created FLINK-20629: - Summary: [Kinesis][EFO] Migrate from DescribeStream to DescribeStreamSummary Key: FLINK-20629 URL: https://issues.apache.org/jira/browse/FLINK-20629 Project: Flink

[jira] [Created] (FLINK-20628) Port RabbitMQ Sources to FLIP-27 API

2020-12-16 Thread Jan Westphal (Jira)
Jan Westphal created FLINK-20628: Summary: Port RabbitMQ Sources to FLIP-27 API Key: FLINK-20628 URL: https://issues.apache.org/jira/browse/FLINK-20628 Project: Flink Issue Type: Improvement

[jira] [Created] (FLINK-20627) Cancel Job button overlaps text in smaller windows

2020-12-16 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-20627: Summary: Cancel Job button overlaps text in smaller windows Key: FLINK-20627 URL: https://issues.apache.org/jira/browse/FLINK-20627 Project: Flink

Re: [VOTE] Release 1.11.3, release candidate #2

2020-12-16 Thread Leonard Xu
+1 (non-binding) - checked/verified signatures and hashes - built from source code with scala 2.11 succeeded - started a cluster, WebUI was accessible, submitted a wordcount job and ran succeeded - test simple sql job in SQL Client, the query result is expected Best, Leonard Xu > 在

Re: [VOTE] Release 1.11.3, release candidate #2

2020-12-16 Thread Wei Zhong
+1 (non-binding) - verified checksums and signatures - pip install pyflink on Windows python 3.7 - run a python job with udfs on Windows Best, Wei > 在 2020年12月16日,15:51,Dian Fu 写道: > > +1 (binding) > > - Verified the checksum and signature > - Checked the pom changes since 1.11.2 for the

[jira] [Created] (FLINK-20626) Canceling a job when it is failing will result in job hanging in CANCELING state

2020-12-16 Thread Zhu Zhu (Jira)
Zhu Zhu created FLINK-20626: --- Summary: Canceling a job when it is failing will result in job hanging in CANCELING state Key: FLINK-20626 URL: https://issues.apache.org/jira/browse/FLINK-20626 Project:

[jira] [Created] (FLINK-20625) Refactor Google Cloud PubSub Source in accordance with FLIP-27

2020-12-16 Thread Jakob Edding (Jira)
Jakob Edding created FLINK-20625: Summary: Refactor Google Cloud PubSub Source in accordance with FLIP-27 Key: FLINK-20625 URL: https://issues.apache.org/jira/browse/FLINK-20625 Project: Flink

[jira] [Created] (FLINK-20624) Refactor StreamExecJoinRule、StreamExecIntervalJoinRule and StreamExecTemporalJoinRule

2020-12-16 Thread wangsan (Jira)
wangsan created FLINK-20624: --- Summary: Refactor StreamExecJoinRule、StreamExecIntervalJoinRule and StreamExecTemporalJoinRule Key: FLINK-20624 URL: https://issues.apache.org/jira/browse/FLINK-20624 Project:

[jira] [Created] (FLINK-20623) Separate the implementation of StreamExecWatermarkAssigner

2020-12-16 Thread godfrey he (Jira)
godfrey he created FLINK-20623: -- Summary: Separate the implementation of StreamExecWatermarkAssigner Key: FLINK-20623 URL: https://issues.apache.org/jira/browse/FLINK-20623 Project: Flink Issue

Re: [DISCUSS] FLIP-153: Support state access in Python DataStream API

2020-12-16 Thread Shuiqiang Chen
Hi Xingbo, Thank you for your valuable suggestions. Indeed, we need to provide clearer abstractions for StateDescriptor and State APIs, I have updated the FLIP accordingly. Looking forward to your feedbacks! Best, Shuiqiang > 在 2020年12月14日,上午11:27,Xingbo Huang 写道: > > Thanks Shuiqiang for

[jira] [Created] (FLINK-20622) Separate the implementation of StreamExecChangelogNormalize

2020-12-16 Thread godfrey he (Jira)
godfrey he created FLINK-20622: -- Summary: Separate the implementation of StreamExecChangelogNormalize Key: FLINK-20622 URL: https://issues.apache.org/jira/browse/FLINK-20622 Project: Flink

[jira] [Created] (FLINK-20621) Refactor the TypeInformation implementation in Python DataStream API

2020-12-16 Thread Shuiqiang Chen (Jira)
Shuiqiang Chen created FLINK-20621: -- Summary: Refactor the TypeInformation implementation in Python DataStream API Key: FLINK-20621 URL: https://issues.apache.org/jira/browse/FLINK-20621 Project:

[jira] [Created] (FLINK-20620) Port StreamExecPythonCalc to java

2020-12-16 Thread godfrey he (Jira)
godfrey he created FLINK-20620: -- Summary: Port StreamExecPythonCalc to java Key: FLINK-20620 URL: https://issues.apache.org/jira/browse/FLINK-20620 Project: Flink Issue Type: Sub-task