Re: Re: [DISCUSS] FLIP-134: DataStream Semantics for Bounded Input

2020-08-17 Thread Yun Gao
Hi, Very thanks for bringing up this discussion! One more question is that does the BATCH and STREAMING mode also decides the shuffle types and operators? I'm asking so because that even for blocking mode, it should also benefit from keeping some edges to be pipeline if the resources

[jira] [Created] (FLINK-18982) InputFormat should remove unnecessary override methods

2020-08-17 Thread Ryan Tao (Jira)
Ryan Tao created FLINK-18982: Summary: InputFormat should remove unnecessary override methods Key: FLINK-18982 URL: https://issues.apache.org/jira/browse/FLINK-18982 Project: Flink Issue Type: Im

Re: Re: [DISCUSS] Removing deprecated methods from DataStream API

2020-08-17 Thread Yun Gao
+1 for removing the methods that are deprecated for a while & have alternative methods. One specific thing is that if we remove the DataStream#split, do we consider enabling side-output in more operators in the future ? Currently it should be only available in ProcessFunctions, but not availabl

Re: [DISCUSS] FLIP-132: Temporal Table DDL

2020-08-17 Thread Rui Li
Thanks Leonard for the clarifications! On Mon, Aug 17, 2020 at 9:17 PM Leonard Xu wrote: > > > But are we still able to track different views of such a > > table through time, as rows are added/deleted to/from the table? > > Yes, in fact we support temporal table from changlog which contains all

[jira] [Created] (FLINK-18981) Support column comment for Hive tables

2020-08-17 Thread Rui Li (Jira)
Rui Li created FLINK-18981: -- Summary: Support column comment for Hive tables Key: FLINK-18981 URL: https://issues.apache.org/jira/browse/FLINK-18981 Project: Flink Issue Type: Task Compone

Decompose failure recovery time

2020-08-17 Thread Zhinan Cheng
Hi all, I am working on measuring the failure recovery time of Flink and I want to decompose the recovery time into different parts, say the time to detect the failure, the time to restart the job, and the time to restore the checkpointing. Unfortunately, I cannot find any information in Flink d

[jira] [Created] (FLINK-18980) "Avro Confluent Schema Registry nightly end-to-end test" hangs

2020-08-17 Thread Dian Fu (Jira)
Dian Fu created FLINK-18980: --- Summary: "Avro Confluent Schema Registry nightly end-to-end test" hangs Key: FLINK-18980 URL: https://issues.apache.org/jira/browse/FLINK-18980 Project: Flink Issue T

Re: [DISCUSS] Removing deprecated methods from DataStream API

2020-08-17 Thread Kostas Kloudas
+1 for removing them. >From a quick look, most of them (not all) have been deprecated a long time ago. Cheers, Kostas On Mon, Aug 17, 2020 at 9:37 PM Dawid Wysakowicz wrote: > > @David Yes, my idea was to remove any use of fold method and all related > classes including WindowedStream#fold > >

Re: [DISCUSS] Removing deprecated methods from DataStream API

2020-08-17 Thread Dawid Wysakowicz
@David Yes, my idea was to remove any use of fold method and all related classes including WindowedStream#fold @Klou Good idea to also remove the deprecated enableCheckpointing() & StreamExecutionEnvironment#readFile and alike. I did another pass over some of the classes and thought we could also

[jira] [Created] (FLINK-18979) Update statefun docs to better emphasize remote modules

2020-08-17 Thread Seth Wiesman (Jira)
Seth Wiesman created FLINK-18979: Summary: Update statefun docs to better emphasize remote modules Key: FLINK-18979 URL: https://issues.apache.org/jira/browse/FLINK-18979 Project: Flink Issue

Re: [DISCUSS] FLIP-134: DataStream Semantics for Bounded Input

2020-08-17 Thread Kostas Kloudas
Hi Kurt and David, Thanks a lot for the insightful feedback! @Kurt: For the topic of checkpointing with Batch Scheduling, I totally agree with you that it requires a lot more work and careful thinking on the semantics. This FLIP was written under the assumption that if the user wants to have chec

[jira] [Created] (FLINK-18978) Support full table scan of key and namespace from statebackend

2020-08-17 Thread Seth Wiesman (Jira)
Seth Wiesman created FLINK-18978: Summary: Support full table scan of key and namespace from statebackend Key: FLINK-18978 URL: https://issues.apache.org/jira/browse/FLINK-18978 Project: Flink

[jira] [Created] (FLINK-18977) Extract WindowOperator construction into a builder class

2020-08-17 Thread Seth Wiesman (Jira)
Seth Wiesman created FLINK-18977: Summary: Extract WindowOperator construction into a builder class Key: FLINK-18977 URL: https://issues.apache.org/jira/browse/FLINK-18977 Project: Flink Iss

Re: [DISCUSS] Planning Flink 1.12

2020-08-17 Thread Seth Wiesman
@David Anderson I think we can get that into 1.12. I have a branch that just needs to be cleaned up. +1 for https://issues.apache.org/jira/browse/FLINK-13095 On Mon, Aug 17, 2020 at 2:03 AM Till Rohrmann wrote: > Hi Rodrigo, > > FLINK-10407 has not been up to date with the actual progress on t

Re: [VOTE] Release 1.10.2, release candidate #2

2020-08-17 Thread Jeff Zhang
+1 (non-binding) Verified it on Zeppelin, it works well. Robert Metzger 于2020年8月17日周一 下午9:41写道: > Thanks a lot for creating another release candidate! > > +1 (binding) > > - checked diff: > https://github.com/apache/flink/compare/release-1.10.1...release-1.10.2-rc2 > - kubernetes client was

Re: [VOTE] Release 1.10.2, release candidate #2

2020-08-17 Thread Robert Metzger
Thanks a lot for creating another release candidate! +1 (binding) - checked diff: https://github.com/apache/flink/compare/release-1.10.1...release-1.10.2-rc2 - kubernetes client was upgraded from 4.5.2 to 4.9.2 + some shading changes > verified NOTICE file with shade output - source compile

Re: [DISCUSS] FLIP-132: Temporal Table DDL

2020-08-17 Thread Leonard Xu
> But are we still able to track different views of such a > table through time, as rows are added/deleted to/from the table? Yes, in fact we support temporal table from changlog which contains all possible message types(INSERT/UPDATE/DELETE). > For > example, suppose I have an append-only tabl

[jira] [Created] (FLINK-18976) Support user defined query in JDBC Source

2020-08-17 Thread Leonard Xu (Jira)
Leonard Xu created FLINK-18976: -- Summary: Support user defined query in JDBC Source Key: FLINK-18976 URL: https://issues.apache.org/jira/browse/FLINK-18976 Project: Flink Issue Type: Improvement

Re: [DISCUSS] FLIP-132: Temporal Table DDL

2020-08-17 Thread Rui Li
Hey Leonard, Thanks for summarizing the document. I have one quick question. I understand a temporal table w/o version means each row in the table only has one version. But are we still able to track different views of such a table through time, as rows are added/deleted to/from the table? For exa

Re: [DISCUSS] FLIP-134: DataStream Semantics for Bounded Input

2020-08-17 Thread David Anderson
Kostas, I'm pleased to see some concrete details in this FLIP. I wonder if the current proposal goes far enough in the direction of recognizing the need some users may have for "batch" and "bounded streaming" to be treated differently. If I've understood it correctly, the section on scheduling al

[VOTE] Release 1.10.2, release candidate #2

2020-08-17 Thread Zhu Zhu
Hi everyone, Please review and vote on the release candidate #2 for the version 1.10.2, as follows: [ ] +1, Approve the release [ ] -1, Do not approve the release (please provide specific comments) The complete staging area is available for your review, which includes: * JIRA release notes [1],

Re: [DISCUSS] Removing deprecated methods from DataStream API

2020-08-17 Thread David Anderson
I assume that along with DataStream#fold you would also remove WindowedStream#fold. I'm in favor of going ahead with all of these. David On Mon, Aug 17, 2020 at 10:53 AM Dawid Wysakowicz wrote: > Hi devs and users, > > I wanted to ask you what do you think about removing some of the > deprecat

Re: [DISCUSS] Removing deprecated methods from DataStream API

2020-08-17 Thread Kostas Kloudas
Thanks a lot for starting this Dawid, Big +1 for the proposed clean-up, and I would also add the deprecated methods of the StreamExecutionEnvironment like: enableCheckpointing(long interval, CheckpointingMode mode, boolean force) enableCheckpointing() isForceCheckpointing() readFile(FileInputFor

[jira] [Created] (FLINK-18975) Translate the 'Execution Plans' page of "Application Development's Managing Execution" into Chinese

2020-08-17 Thread Roc Marshal (Jira)
Roc Marshal created FLINK-18975: --- Summary: Translate the 'Execution Plans' page of "Application Development's Managing Execution" into Chinese Key: FLINK-18975 URL: https://issues.apache.org/jira/browse/FLINK-18975

[jira] [Created] (FLINK-18974) Translate the 'User-Defined Functions' page of "Application Development's DataStream API" into Chinese

2020-08-17 Thread Roc Marshal (Jira)
Roc Marshal created FLINK-18974: --- Summary: Translate the 'User-Defined Functions' page of "Application Development's DataStream API" into Chinese Key: FLINK-18974 URL: https://issues.apache.org/jira/browse/FLINK-189

[jira] [Created] (FLINK-18973) Translate the 'History Server' page of 'Debugging & Monitoring' into Chinese

2020-08-17 Thread Roc Marshal (Jira)
Roc Marshal created FLINK-18973: --- Summary: Translate the 'History Server' page of 'Debugging & Monitoring' into Chinese Key: FLINK-18973 URL: https://issues.apache.org/jira/browse/FLINK-18973 Project: F

[jira] [Created] (FLINK-18972) Unfulfillable slot requests of Blink planner batch jobs never timeout

2020-08-17 Thread Zhu Zhu (Jira)
Zhu Zhu created FLINK-18972: --- Summary: Unfulfillable slot requests of Blink planner batch jobs never timeout Key: FLINK-18972 URL: https://issues.apache.org/jira/browse/FLINK-18972 Project: Flink

[DISCUSS] Removing deprecated methods from DataStream API

2020-08-17 Thread Dawid Wysakowicz
Hi devs and users, I wanted to ask you what do you think about removing some of the deprecated APIs around the DataStream API. The APIs I have in mind are: * RuntimeContext#getAllAccumulators (deprecated in 0.10) * DataStream#fold and all related classes and methods such as FoldFunction,

[jira] [Created] (FLINK-18971) Support to mount kerberos conf as ConfigMap and Keytab as Secrete

2020-08-17 Thread Yangze Guo (Jira)
Yangze Guo created FLINK-18971: -- Summary: Support to mount kerberos conf as ConfigMap and Keytab as Secrete Key: FLINK-18971 URL: https://issues.apache.org/jira/browse/FLINK-18971 Project: Flink

Re: [DISCUSS] Planning Flink 1.12

2020-08-17 Thread Till Rohrmann
Hi Rodrigo, FLINK-10407 has not been up to date with the actual progress on the feature. We will update it soon. I think that the community won't actively work on FLINK-12002 in this release. However, we will work on FLINK-16430 [1] and once this is done continue with the clean up of the legacy s