Re: Fine Grained Recovery / FLIP-1

2019-07-26 Thread Stephan Ewen
Hi Thomas! For Batch, this should be working in release 1.9. For streaming, it is a bit more tricky, mainly because of the fact that you have to deal with downstream correctness. Either a recovery still needs to reset downstream tasks (which means on average half of the tasks) or needs to wait be

Re: [DISCUSS] FLIP-27: Refactor Source Interface

2019-07-26 Thread Biao Liu
Hi Stephan, Thank you for feedback! Will take a look at your branch before public discussing. On Fri, Jul 26, 2019 at 12:01 AM Stephan Ewen wrote: > Hi Biao! > > Thanks for reviving this. I would like to join this discussion, but am > quite occupied with the 1.9 release, so can we maybe pause

Re:confirm subscribe to dev@flink.apache.org

2019-07-26 Thread lyz
confirmed At 2019-07-26 16:45:20, dev-h...@flink.apache.org wrote: >Hi! This is the ezmlm program. I'm managing the >dev@flink.apache.org mailing list. > >I'm working for my owner, who can be reached >at dev-ow...@flink.apache.org. > >To confirm that you would like > > xcliyaz...@163.com

[jira] [Created] (FLINK-13431) NameNode HA configuration was not loaded when running HiveConnector on Yarn

2019-07-26 Thread Hongtao Zhang (JIRA)
Hongtao Zhang created FLINK-13431: - Summary: NameNode HA configuration was not loaded when running HiveConnector on Yarn Key: FLINK-13431 URL: https://issues.apache.org/jira/browse/FLINK-13431 Project

Re: REST API / JarRunHandler: More flexibility for launching jobs

2019-07-26 Thread Till Rohrmann
Hi Thomas, quick question: Why do you wanna use the JarRunHandler? If another process is building the JobGraph, then one could use the JobSubmitHandler which expects a JobGraph and then starts executing it. Cheers, Till On Thu, Jul 25, 2019 at 7:45 PM Thomas Weise wrote: > Hi, > > While consid

Re: Fwd: Kafka Summit listing on ASF events site

2019-07-26 Thread Fabian Hueske
Hi Austin, Thanks for asking to add Flink Forward to the ASF event calendar! I had asked for this a few weeks ago but we had to get brand approval first. We got the approval and I asked again on dev@community.a.o just a few minutes ago. Thanks, Fabian Am Fr., 26. Juli 2019 um 01:21 Uhr schrieb

[jira] [Created] (FLINK-13432) Set max polling interval threshold for job result polling

2019-07-26 Thread Jeff Zhang (JIRA)
Jeff Zhang created FLINK-13432: -- Summary: Set max polling interval threshold for job result polling Key: FLINK-13432 URL: https://issues.apache.org/jira/browse/FLINK-13432 Project: Flink Issue T

[jira] [Created] (FLINK-13433) Do not fetch data from LookupableTableSource if the JoinKey in left side of LookupJoin contains null value

2019-07-26 Thread Jing Zhang (JIRA)
Jing Zhang created FLINK-13433: -- Summary: Do not fetch data from LookupableTableSource if the JoinKey in left side of LookupJoin contains null value Key: FLINK-13433 URL: https://issues.apache.org/jira/browse/FLINK-1

[jira] [Created] (FLINK-13434) Add E2E test for stop-with-savepoint

2019-07-26 Thread Till Rohrmann (JIRA)
Till Rohrmann created FLINK-13434: - Summary: Add E2E test for stop-with-savepoint Key: FLINK-13434 URL: https://issues.apache.org/jira/browse/FLINK-13434 Project: Flink Issue Type: Test

[jira] [Created] (FLINK-13435) Remove ShuffleDescriptor.ReleaseType and make release semantics fixed per partition type

2019-07-26 Thread Andrey Zagrebin (JIRA)
Andrey Zagrebin created FLINK-13435: --- Summary: Remove ShuffleDescriptor.ReleaseType and make release semantics fixed per partition type Key: FLINK-13435 URL: https://issues.apache.org/jira/browse/FLINK-13435

[jira] [Created] (FLINK-13436) Add TPC-H queries as E2E tests

2019-07-26 Thread Till Rohrmann (JIRA)
Till Rohrmann created FLINK-13436: - Summary: Add TPC-H queries as E2E tests Key: FLINK-13436 URL: https://issues.apache.org/jira/browse/FLINK-13436 Project: Flink Issue Type: Test C

[jira] [Created] (FLINK-13437) Add Hive SQL E2E test

2019-07-26 Thread Till Rohrmann (JIRA)
Till Rohrmann created FLINK-13437: - Summary: Add Hive SQL E2E test Key: FLINK-13437 URL: https://issues.apache.org/jira/browse/FLINK-13437 Project: Flink Issue Type: Test Components

[jira] [Created] (FLINK-13438) Fix Hive connector with DataTypes.DATE/TIME/TIMESTAMP support

2019-07-26 Thread Caizhi Weng (JIRA)
Caizhi Weng created FLINK-13438: --- Summary: Fix Hive connector with DataTypes.DATE/TIME/TIMESTAMP support Key: FLINK-13438 URL: https://issues.apache.org/jira/browse/FLINK-13438 Project: Flink

[jira] [Created] (FLINK-13439) Run existing SQL/Table API E2E tests with blink runner

2019-07-26 Thread Till Rohrmann (JIRA)
Till Rohrmann created FLINK-13439: - Summary: Run existing SQL/Table API E2E tests with blink runner Key: FLINK-13439 URL: https://issues.apache.org/jira/browse/FLINK-13439 Project: Flink Issu

Re: [DISCUSS] Create a Flink ecosystem website

2019-07-26 Thread Oytun Tez
I agree with Robert – localization (this is what we do at MotaWord) is a maintenance work. If not maintained as well as mainstream, it will only damage and distant devs that use those local websites. Re: comments, I don't think people will really discuss furiously. But we at least need a system wh

Re: [Requirement] CI report

2019-07-26 Thread Chesnay Schepler
Noted, I'll see what I can do. On 23/07/2019 10:15, Zili Chen wrote: Hi, Currently, our flinkbot updates CI report on status changing. However, it updates via editing GitHub comment, which would not send a notification to pr creator once status updated. Said the "PENDING" status is not quite

[jira] [Created] (FLINK-13440) Add test that fails job when sync savepoint is discarded.

2019-07-26 Thread Kostas Kloudas (JIRA)
Kostas Kloudas created FLINK-13440: -- Summary: Add test that fails job when sync savepoint is discarded. Key: FLINK-13440 URL: https://issues.apache.org/jira/browse/FLINK-13440 Project: Flink

Re: REST API / JarRunHandler: More flexibility for launching jobs

2019-07-26 Thread Thomas Weise
Hi Till, Thanks for taking a look! The Beam job server does not currently have the ability to just output the job graph (and related artifacts) that could then be used with the JobSubmitHandler. It is itself using StreamExecutionEnvironment, which in turn will lead to a REST API submission. Here

[jira] [Created] (FLINK-13441) Add batch E2E test which runs with fewer slots than parallelism

2019-07-26 Thread Till Rohrmann (JIRA)
Till Rohrmann created FLINK-13441: - Summary: Add batch E2E test which runs with fewer slots than parallelism Key: FLINK-13441 URL: https://issues.apache.org/jira/browse/FLINK-13441 Project: Flink

[jira] [Created] (FLINK-13442) Remove unnecessary notifySubpartitionConsumed method from view reader

2019-07-26 Thread zhijiang (JIRA)
zhijiang created FLINK-13442: Summary: Remove unnecessary notifySubpartitionConsumed method from view reader Key: FLINK-13442 URL: https://issues.apache.org/jira/browse/FLINK-13442 Project: Flink

Re: [DISCUSS] Flink project bylaws

2019-07-26 Thread Yu Li
Hi Becket, Thanks for noticing and resolving my comment around PMC removal and ASF rules of PMC membership change process, which you seem to neglect in the summary of updates (smile). Best Regards, Yu On Wed, 24 Jul 2019 at 04:32, Becket Qin wrote: > Hi folks, > > Thanks for all the feedback.

[jira] [Created] (FLINK-13443) Support/Map Hive INTERVAL type in Hive connector

2019-07-26 Thread Xuefu Zhang (JIRA)
Xuefu Zhang created FLINK-13443: --- Summary: Support/Map Hive INTERVAL type in Hive connector Key: FLINK-13443 URL: https://issues.apache.org/jira/browse/FLINK-13443 Project: Flink Issue Type: Im

Re: Request for permission as a contributor

2019-07-26 Thread Yu Li
Correct me if I'm wrong, but one still needs the contributor identity to be assigned to JIRA, although contributors could no longer assign the JIRA to themselves. On the other hand, due to the recent JIRA permissions change, I'd suggest to only apply for the contributor permission after you have s

Re: Fine Grained Recovery / FLIP-1

2019-07-26 Thread Thomas Weise
Guowei and Stephan, thanks for the reply! The biggest gain that FLIP-1 will deliver for streaming is that parallel processing can continue accept for those parallel paths affected by the failure, even when all tasks in an affected path need to be reset. Assuming task manager process failure as mos

[jira] [Created] (FLINK-13444) Translate English content of FLINK-13396 into Chinese

2019-07-26 Thread Jark Wu (JIRA)
Jark Wu created FLINK-13444: --- Summary: Translate English content of FLINK-13396 into Chinese Key: FLINK-13444 URL: https://issues.apache.org/jira/browse/FLINK-13444 Project: Flink Issue Type: Task

Re: [ANNOUNCE] Progress updates for Apache Flink 1.9.0 release

2019-07-26 Thread Tzu-Li (Gordon) Tai
Hi all, It's been a while since our last update for the release testing of 1.9.0, so I want to bring attention to the current status of the release. We are approaching RC1 soon, waiting on the following specific last ongoing threads to be closed: - FLINK-13241: This fixes a problem where when usi