[jira] [Created] (FLINK-23077) Running nexmark q5 with 1.13.1 of pipeline.object-reuse=true, the taskmanager will be killed and produce failover.

2021-06-21 Thread xiaojin.wy (Jira)
xiaojin.wy created FLINK-23077: -- Summary: Running nexmark q5 with 1.13.1 of pipeline.object-reuse=true, the taskmanager will be killed and produce failover. Key: FLINK-23077 URL:

[jira] [Created] (FLINK-23076) DispatcherTest.testWaitingForJobMasterLeadership fails on azure

2021-06-21 Thread Xintong Song (Jira)
Xintong Song created FLINK-23076: Summary: DispatcherTest.testWaitingForJobMasterLeadership fails on azure Key: FLINK-23076 URL: https://issues.apache.org/jira/browse/FLINK-23076 Project: Flink

Re: [DISCUSS] Releasing Flink 1.11.4, 1.12.5, 1.13.2

2021-06-21 Thread Xintong Song
Thanks Dawid for starting the discussion, and thanks Yun, Godfrey and Jingsong for volunteering as release managers. +1 for the releases, and +1 for the release managers. Thank you~ Xintong Song On Tue, Jun 22, 2021 at 10:15 AM Jingsong Li wrote: > +1 to the release. > > Thanks Dawid for

[jira] [Created] (FLINK-23075) Python API for enabling ChangelogStateBackend

2021-06-21 Thread Zakelly Lan (Jira)
Zakelly Lan created FLINK-23075: --- Summary: Python API for enabling ChangelogStateBackend Key: FLINK-23075 URL: https://issues.apache.org/jira/browse/FLINK-23075 Project: Flink Issue Type:

[jira] [Created] (FLINK-23074) There is a class conflict between flink-connector-hive and flink-parquet

2021-06-21 Thread Ada Wong (Jira)
Ada Wong created FLINK-23074: Summary: There is a class conflict between flink-connector-hive and flink-parquet Key: FLINK-23074 URL: https://issues.apache.org/jira/browse/FLINK-23074 Project: Flink

Re: trying (and failing) to update pyflink-walkthrough for Flink 1.13

2021-06-21 Thread Dian Fu
Thanks David for taking care of this. I will take a look at this issue. Regards, Dian > 2021年6月22日 上午4:06,David Anderson 写道: > > I've been trying to upgrade the pyflink-walkthrough to Flink 1.13.1, but > without any success. > > Unless I give it a lot of resources the data generator times out

Re: [VOTE] FLIP-129 Register sources/sinks in Table API

2021-06-21 Thread Leonard Xu
+1 Thanks Ingo for picking up this FLIP. Best, Leonard > 在 2021年6月22日,10:16,JING ZHANG 写道: > > +1 (binding) > > Best regards, > JING ZHANG > > Jingsong Li 于2021年6月22日周二 上午10:11写道: > >> +1 (binding) >> >> Thanks for driving. >> >> Best, >> Jingsong >> >> On Tue, Jun 22, 2021 at 12:07

Re: [VOTE] FLIP-129 Register sources/sinks in Table API

2021-06-21 Thread JING ZHANG
+1 (binding) Best regards, JING ZHANG Jingsong Li 于2021年6月22日周二 上午10:11写道: > +1 (binding) > > Thanks for driving. > > Best, > Jingsong > > On Tue, Jun 22, 2021 at 12:07 AM Jark Wu wrote: > > > +1 (binding) > > > > Best, > > Jark > > > > On Mon, 21 Jun 2021 at 22:51, Timo Walther wrote: > > >

Re: [VOTE] FLIP-129 Register sources/sinks in Table API

2021-06-21 Thread Jingsong Li
+1 (binding) Thanks for driving. Best, Jingsong On Tue, Jun 22, 2021 at 12:07 AM Jark Wu wrote: > +1 (binding) > > Best, > Jark > > On Mon, 21 Jun 2021 at 22:51, Timo Walther wrote: > > > +1 (binding) > > > > Thanks for driving this. > > > > Regards, > > Timo > > > > On 21.06.21 13:24, Ingo

Re: [DISCUSS] Releasing Flink 1.11.4, 1.12.5, 1.13.2

2021-06-21 Thread Jingsong Li
+1 to the release. Thanks Dawid for driving this discussion. I am willing to volunteer as the release manager too. Best, Jingsong On Tue, Jun 22, 2021 at 9:58 AM godfrey he wrote: > Thanks for driving this, Dawid. +1 to the release. > I would also like to volunteer as the release manager for

Re: [DISCUSS] Releasing Flink 1.11.4, 1.12.5, 1.13.2

2021-06-21 Thread godfrey he
Thanks for driving this, Dawid. +1 to the release. I would also like to volunteer as the release manager for these versions. Best Godfrey Seth Wiesman 于2021年6月22日周二 上午8:39写道: > +1 to the release. > > It would be great if we could get FLINK-23073 into 1.13.2. There's already > an open PR and

Re: [DISCUSS] Releasing Flink 1.11.4, 1.12.5, 1.13.2

2021-06-21 Thread Seth Wiesman
+1 to the release. It would be great if we could get FLINK-23073 into 1.13.2. There's already an open PR and it unblocks upgrading the table API walkthrough in apache/flink-playgrounds to 1.13. Seth On Mon, Jun 21, 2021 at 6:28 AM Yun Tang wrote: > Hi Dawid, > > Thanks for driving this

Re: [DISCUSS] Dashboard/HistoryServer authentication

2021-06-21 Thread Austin Cawley-Edwards
Hi Gabor + Marton, I don't believe that the issue with this proposal is the specific mechanism proposed (Kerberos), but rather that it is not the level to implement it at (Flink). I'm just one voice, so please take this with a grain of salt. In the other solutions previously noted there is no

Re: PR: "Propagate watermarks to sink API"

2021-06-21 Thread Arvid Heise
Hi Eron, I will check it tomorrow. Sorry for the delay. If someone beats me to that, please go ahead. On Mon, Jun 21, 2021 at 8:18 PM Eron Wright wrote: > Would someone be willing and able to review the PR which adds watermarks to > the sink API? > > https://github.com/apache/flink/pull/15950

trying (and failing) to update pyflink-walkthrough for Flink 1.13

2021-06-21 Thread David Anderson
I've been trying to upgrade the pyflink-walkthrough to Flink 1.13.1, but without any success. Unless I give it a lot of resources the data generator times out trying to connect to Kafka. If I give it 6 cores and 11GB (which is about all I can offer it) it does manage to connect, but then fails

Re: [DISCUSS]FLIP-150: Introduce Hybrid Source

2021-06-21 Thread Thomas Weise
Here is a summary of where we are at with the PR: * Added capability to construct sources at switch time through a factory interface. This can support all previously discussed scenarios. The simple case (sources with fixed start position) is still simple, but for scenarios that require deferred

[jira] [Created] (FLINK-23073) Fix space handling in CSV timestamp parser

2021-06-21 Thread Seth Wiesman (Jira)
Seth Wiesman created FLINK-23073: Summary: Fix space handling in CSV timestamp parser Key: FLINK-23073 URL: https://issues.apache.org/jira/browse/FLINK-23073 Project: Flink Issue Type: Bug

PR: "Propagate watermarks to sink API"

2021-06-21 Thread Eron Wright
Would someone be willing and able to review the PR which adds watermarks to the sink API? https://github.com/apache/flink/pull/15950 Thanks! Eron

Re: [DISCUSS] Dashboard/HistoryServer authentication

2021-06-21 Thread Márton Balassi
Hi team, Thank you for your input. Based on this discussion I agree with G that selecting and standardizing on a specific strong authentication mechanism is more challenging than the whole rest of the scope of this authentication story. :-) I suggest that G and I go back to the drawing board and

Re: [VOTE] FLIP-129 Register sources/sinks in Table API

2021-06-21 Thread Jark Wu
+1 (binding) Best, Jark On Mon, 21 Jun 2021 at 22:51, Timo Walther wrote: > +1 (binding) > > Thanks for driving this. > > Regards, > Timo > > On 21.06.21 13:24, Ingo Bürk wrote: > > Hi everyone, > > > > thanks for all the feedback so far. Based on the discussion[1] we seem to > > have

Re: [VOTE] FLIP-129 Register sources/sinks in Table API

2021-06-21 Thread Timo Walther
+1 (binding) Thanks for driving this. Regards, Timo On 21.06.21 13:24, Ingo Bürk wrote: Hi everyone, thanks for all the feedback so far. Based on the discussion[1] we seem to have consensus, so I would like to start a vote on FLIP-129 for which the FLIP has now also been updated[2]. The

Re: [DISCUSS] Dashboard/HistoryServer authentication

2021-06-21 Thread Gabor Somogyi
Hi All, We see that adding any kind of specific authentication raises more questions than answers. What would be if a generic API would be added without any real authentication logic? That way every provider can add its own protocol implementation as additional jar. BR, G On Thu, Jun 17, 2021

[jira] [Created] (FLINK-23072) Add benchmarks for SQL internal and external serializers

2021-06-21 Thread Timo Walther (Jira)
Timo Walther created FLINK-23072: Summary: Add benchmarks for SQL internal and external serializers Key: FLINK-23072 URL: https://issues.apache.org/jira/browse/FLINK-23072 Project: Flink

Re: Re: [DISCUSS] FLIP-169: DataStream API for Fine-Grained Resource Requirements

2021-06-21 Thread Yangze Guo
Thanks for the feedback, Till! Actually, we cannot give user any resolution for this issue as there is no API for DataStream users to influence the edge types at the moment. The edge types are currently fixed based on the jobs' mode (batch or streaming). a) I think it might not confuse the user a

Re: [DISCUSS] FLIP-171: Async Sink

2021-06-21 Thread Piotr Nowojski
Hi, Thanks Steffen for the explanations. I think it makes sense to me. Re Arvid/Steffen: - Keep in mind that even if we choose to provide a non blocking API using the `isAvailable()`/`getAvailableFuture()` method, we would still need to support blocking inside the sinks. For example at the very

[VOTE] FLIP-129 Register sources/sinks in Table API

2021-06-21 Thread Ingo Bürk
Hi everyone, thanks for all the feedback so far. Based on the discussion[1] we seem to have consensus, so I would like to start a vote on FLIP-129 for which the FLIP has now also been updated[2]. The vote will last for at least 72 hours (Thu, Jun 24th 12:00 GMT) unless there is an objection or

Re: [DISCUSS] Releasing Flink 1.11.4, 1.12.5, 1.13.2

2021-06-21 Thread Yun Tang
Hi Dawid, Thanks for driving this discussion, I am willing to volunteer as the release manager for these versions. Best Yun Tang From: Konstantin Knauf Sent: Friday, June 18, 2021 22:35 To: dev Subject: Re: [DISCUSS] Releasing Flink 1.11.4, 1.12.5, 1.13.2 Hi

[jira] [Created] (FLINK-23071) Implement StatementSet#addInsert(TableDescriptor, Table)

2021-06-21 Thread Jira
Ingo Bürk created FLINK-23071: - Summary: Implement StatementSet#addInsert(TableDescriptor, Table) Key: FLINK-23071 URL: https://issues.apache.org/jira/browse/FLINK-23071 Project: Flink Issue

[jira] [Created] (FLINK-23065) Implement TableEnvironment#createTemporaryTable

2021-06-21 Thread Jira
Ingo Bürk created FLINK-23065: - Summary: Implement TableEnvironment#createTemporaryTable Key: FLINK-23065 URL: https://issues.apache.org/jira/browse/FLINK-23065 Project: Flink Issue Type:

[jira] [Created] (FLINK-23070) Implement TableEnvironment#createTable

2021-06-21 Thread Jira
Ingo Bürk created FLINK-23070: - Summary: Implement TableEnvironment#createTable Key: FLINK-23070 URL: https://issues.apache.org/jira/browse/FLINK-23070 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-23069) Support schema-less #executeInsert(TableDescriptor)

2021-06-21 Thread Jira
Ingo Bürk created FLINK-23069: - Summary: Support schema-less #executeInsert(TableDescriptor) Key: FLINK-23069 URL: https://issues.apache.org/jira/browse/FLINK-23069 Project: Flink Issue Type:

[jira] [Created] (FLINK-23068) Support LIKE in TableDescriptor

2021-06-21 Thread Jira
Ingo Bürk created FLINK-23068: - Summary: Support LIKE in TableDescriptor Key: FLINK-23068 URL: https://issues.apache.org/jira/browse/FLINK-23068 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-23067) Implement Table#executeInsert(TableDescriptor)

2021-06-21 Thread Jira
Ingo Bürk created FLINK-23067: - Summary: Implement Table#executeInsert(TableDescriptor) Key: FLINK-23067 URL: https://issues.apache.org/jira/browse/FLINK-23067 Project: Flink Issue Type:

[jira] [Created] (FLINK-23066) Implement TableEnvironment#from

2021-06-21 Thread Jira
Ingo Bürk created FLINK-23066: - Summary: Implement TableEnvironment#from Key: FLINK-23066 URL: https://issues.apache.org/jira/browse/FLINK-23066 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-23064) Centralize connector options

2021-06-21 Thread Jira
Ingo Bürk created FLINK-23064: - Summary: Centralize connector options Key: FLINK-23064 URL: https://issues.apache.org/jira/browse/FLINK-23064 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-23063) Remove TableEnvironment#connect

2021-06-21 Thread Jira
Ingo Bürk created FLINK-23063: - Summary: Remove TableEnvironment#connect Key: FLINK-23063 URL: https://issues.apache.org/jira/browse/FLINK-23063 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-23062) FLIP-129: Register sources/sinks in Table API

2021-06-21 Thread Jira
Ingo Bürk created FLINK-23062: - Summary: FLIP-129: Register sources/sinks in Table API Key: FLINK-23062 URL: https://issues.apache.org/jira/browse/FLINK-23062 Project: Flink Issue Type: New

[jira] [Created] (FLINK-23061) Split BootstrapTools

2021-06-21 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-23061: Summary: Split BootstrapTools Key: FLINK-23061 URL: https://issues.apache.org/jira/browse/FLINK-23061 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-23060) Move FutureUtils.toJava into separate class

2021-06-21 Thread Chesnay Schepler (Jira)
Chesnay Schepler created FLINK-23060: Summary: Move FutureUtils.toJava into separate class Key: FLINK-23060 URL: https://issues.apache.org/jira/browse/FLINK-23060 Project: Flink Issue

Re: [DISCUSS] FLIP-171: Async Sink

2021-06-21 Thread Arvid Heise
Hi Piotr, to pick up this discussion thread again: - This FLIP is about providing some base implementation for FLIP-143 sinks that make adding new implementations easier, similar to the SourceReaderBase. - The whole availability topic will most likely be a separate FLIP. The basic issue just

Re: Re: [DISCUSS] FLIP-169: DataStream API for Fine-Grained Resource Requirements

2021-06-21 Thread Till Rohrmann
I would be more in favor of what Zhu Zhu proposed to throw an exception with a meaningful and understandable explanation that also includes how to resolve this problem. I do understand the reasoning behind automatically switching the edge types in order to make things easier to use but a) this can

Re: [DISCUSS] Releasing Flink 1.11.4, 1.12.5, 1.13.2

2021-06-21 Thread Till Rohrmann
Thanks for starting this discussion Dawid. We have collected a couple of fixes for the different releases: #fixes: 1.11.4: 72 1.12.5: 35 1.13.2: 49 which in my opinion warrants new bugfix releases. Note that we intend to do another 1.11 release because of the seriousness of the FLINK-22815 [1]

[jira] [Created] (FLINK-23059) Update playgrounds for Flink 1.13

2021-06-21 Thread David Anderson (Jira)
David Anderson created FLINK-23059: -- Summary: Update playgrounds for Flink 1.13 Key: FLINK-23059 URL: https://issues.apache.org/jira/browse/FLINK-23059 Project: Flink Issue Type:

Re: Change in accumutors semantics with jobClient

2021-06-21 Thread Till Rohrmann
Thanks for bringing this to the dev ML Etienne. Could you maybe update the release notes for Flink 1.13 [1] to include this change? That way it might be a bit more prominent. I think the change needs to go into the release-1.13 and master branch. [1]

[jira] [Created] (FLINK-23058) flink-elasticsearch-6 not work

2021-06-21 Thread yandufeng (Jira)
yandufeng created FLINK-23058: - Summary: flink-elasticsearch-6 not work Key: FLINK-23058 URL: https://issues.apache.org/jira/browse/FLINK-23058 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-23057) flink-console.sh doesn't do variable expansion for FLINK_ENV_JAVA_OPTS like flink-daemon.sh

2021-06-21 Thread LIU Xiao (Jira)
LIU Xiao created FLINK-23057: Summary: flink-console.sh doesn't do variable expansion for FLINK_ENV_JAVA_OPTS like flink-daemon.sh Key: FLINK-23057 URL: https://issues.apache.org/jira/browse/FLINK-23057

[jira] [Created] (FLINK-23056) flink docs sql builtin functions out of date

2021-06-21 Thread Junhan Yang (Jira)
Junhan Yang created FLINK-23056: --- Summary: flink docs sql builtin functions out of date Key: FLINK-23056 URL: https://issues.apache.org/jira/browse/FLINK-23056 Project: Flink Issue Type: Bug

[jira] [Created] (FLINK-23055) Add document for Window TVF offset

2021-06-21 Thread JING ZHANG (Jira)
JING ZHANG created FLINK-23055: -- Summary: Add document for Window TVF offset Key: FLINK-23055 URL: https://issues.apache.org/jira/browse/FLINK-23055 Project: Flink Issue Type: Sub-task

Re: [VOTE] FLIP-169: DataStream API for Fine-Grained Resource Requirements

2021-06-21 Thread Yangze Guo
According to the latest comment of Zhu Zhu[1], I append the potential resource deadlock in batch jobs as a known limitation to this FLIP. Thus, I'd extend the voting period for another 72h. [1]

退订

2021-06-21 Thread fengfuli

Re: Re: [DISCUSS] FLIP-169: DataStream API for Fine-Grained Resource Requirements

2021-06-21 Thread Yangze Guo
Thanks, I append it to the known limitations of this FLIP. Best, Yangze Guo On Mon, Jun 21, 2021 at 3:20 PM Zhu Zhu wrote: > > Thanks for the quick response Yangze. > The proposal sounds good to me. > > Thanks, > Zhu > > Yangze Guo 于2021年6月21日周一 下午3:01写道: >> >> Thanks for the comments, Zhu! >>

[jira] [Created] (FLINK-23054) Correct upsert optimization by upsert keys

2021-06-21 Thread Jingsong Lee (Jira)
Jingsong Lee created FLINK-23054: Summary: Correct upsert optimization by upsert keys Key: FLINK-23054 URL: https://issues.apache.org/jira/browse/FLINK-23054 Project: Flink Issue Type: Bug

Re: Re: [DISCUSS] FLIP-169: DataStream API for Fine-Grained Resource Requirements

2021-06-21 Thread Zhu Zhu
Thanks for the quick response Yangze. The proposal sounds good to me. Thanks, Zhu Yangze Guo 于2021年6月21日周一 下午3:01写道: > Thanks for the comments, Zhu! > > Yes, it is a known limitation for fine-grained resource management. We > also have filed this issue in FLINK-20865 when we proposed FLIP-156.

Re: Re: [DISCUSS] FLIP-169: DataStream API for Fine-Grained Resource Requirements

2021-06-21 Thread Yangze Guo
Thanks for the comments, Zhu! Yes, it is a known limitation for fine-grained resource management. We also have filed this issue in FLINK-20865 when we proposed FLIP-156. As a first step, I agree that we can mark batch jobs with PIPELINED edges as an invalid case for this feature. However, just

[jira] [Created] (FLINK-23053) Translate "DataSet Transformations" pages into Chinese

2021-06-21 Thread Zhiliang Zhang (Jira)
Zhiliang Zhang created FLINK-23053: -- Summary: Translate "DataSet Transformations" pages into Chinese Key: FLINK-23053 URL: https://issues.apache.org/jira/browse/FLINK-23053 Project: Flink

Re: Re: [DISCUSS] FLIP-169: DataStream API for Fine-Grained Resource Requirements

2021-06-21 Thread Zhu Zhu
Thanks for proposing this @Yangze Guo and sorry for joining the discussion so late. The proposal generally looks good to me. But I find one problem that batch job with PIPELINED edges might hang if enabling fine-grained resources. see "Resource Deadlocks could still happen in certain Cases"