Re: Review of pull request

2019-10-04 Thread Rishindra Kumar
Hi Gordon, I modified the change according to your suggestions and updated the pull request. I am awaiting your review. Could you please review it when you get time? -- *Maddila Rishindra Kumar* *Software Engineer* *Walmartlabs India* *Contact No: +919967379528 | Alternate E-mail ID: rishindra.

Re: [DISCUSS] FLIP-73: Introducing Executors for job submission

2019-10-04 Thread Aljoscha Krettek
Hi, I think the end goal is to have only one environment per API, but I think we won’t be able to achieve that in the short-term because of backwards compatibility. This is most notable with the context environment, preview environments etc. To keep this FLIP very slim we can make this only ab

Re: [DISCUSS] FLIP-74: Flink JobClient API

2019-10-04 Thread Aljoscha Krettek
This makes sense to me, yes! > On 2. Oct 2019, at 20:35, Zili Chen wrote: > > Hi all, > > Narrow the scope to FLIP-74 we aimed at introduce a useful and extensible > user-facing public interface JobClient. Let me reemphasize two major works > under this thread. > > 1. standard interface > > A

Re: [DISCUSS] FLIP-73: Introducing Executors for job submission

2019-10-04 Thread Aljoscha Krettek
Do you all think we could agree on the basic executor primitives and start voting on this FLIP? There are still some implementation details but I think we can discuss/tackle them when we get to them and the various people implementing this should be in close collaboration. Best, Aljoscha > On

Re: [DISCUSS] FLIP-73: Introducing Executors for job submission

2019-10-04 Thread Zili Chen
Hi Aljoscha, After clearly narrow the scope of this FLIP it looks good to me the interface Executor and its discovery so that I'm glad to see the vote thread. As you said, we should still discuss on implementation details but I don't think it should be a blocker of the vote thread because a vote

Re: [DISCUSS] FLIP-73: Introducing Executors for job submission

2019-10-04 Thread Aljoscha Krettek
Hi Tison, I agree, for now the async Executor.execute() is an internal detail but during your work for FLIP-74 it will probably also reach the public API. Best, Aljoscha > On 4. Oct 2019, at 11:39, Zili Chen wrote: > > Hi Aljoscha, > > After clearly narrow the scope of this FLIP it looks goo

Re: [CODE STYLE] Parameters of method are always final

2019-10-04 Thread Aljoscha Krettek
I actually think that doing this the other way round would be correct. Removing final everywhere and relying on humans to assume that everything is final doesn’t seem maintainable to me. The benefit of having final on parameters/fields is that the compiler/IDE actually checks that you don’t mod

Re: [DISCUSS] FLIP-67: Global partitions lifecycle

2019-10-04 Thread Chesnay Schepler
/Till: In the FLIP you wrote "The set of partitions to release may contain local and/or global partitions; the promotion set must only refer to local partitions." to describe the `releasePartitions`. I think the JM should never be in the situation to release a global partition. Moreover, I beli

Re: [CODE STYLE] Parameters of method are always final

2019-10-04 Thread Zili Chen
Hi Aljoscha, I totally agree with you on field topic. Of course it makes significant difference whether or not a field is final and IDE/compiler can help on checking. Here are several thoughts about final modifier on parameters and why I propose this one: 1. parameters should be always final As

Re: [VOTE] FLIP-57: Rework FunctionCatalog

2019-10-04 Thread Aljoscha Krettek
Hi, I see there was quite some discussion and changes on the FLIP after this VOTE was started. I would suggest to start a new voting thread on the current state of the FLIP (keeping in mind that a FLIP vote needs at least three committer/PMC votes). For the future, we should probably keep disc

Re: [VOTE] FLIP-57: Rework FunctionCatalog

2019-10-04 Thread Timo Walther
Hi, I agree with Aljoscha. It is not transparent to me which votes are binding to the current status of the FLIP. Some other minor comments from my side: - We don't need to deprecate methods in FunctionCatalog. This class is internal. We can simply change the method signatures. - `String nam

Re: [DISCUSS] FLIP-67: Global partitions lifecycle

2019-10-04 Thread Till Rohrmann
On Fri, Oct 4, 2019 at 12:37 PM Chesnay Schepler wrote: > *Till: In the FLIP you wrote "The set of partitions to release may contain > local > and/or global partitions; the promotion set must only refer to local > partitions." to describe the `releasePartitions`. I think the JM should > never be

[jira] [Created] (FLINK-14319) Register user jar files in {Stream}ExecutionEnvironment

2019-10-04 Thread Leo Zhang (Jira)
Leo Zhang created FLINK-14319: - Summary: Register user jar files in {Stream}ExecutionEnvironment Key: FLINK-14319 URL: https://issues.apache.org/jira/browse/FLINK-14319 Project: Flink Issue Type

Re: [DISCUSS] FLIP-68: Extend Core Table System with Modular Plugins

2019-10-04 Thread Timo Walther
Hi everyone, first, I was also questioning my proposal. But Bowen's proposal of `tEnv.offloadToYaml()` would not work with the current design because we don't know how to serialize a catalog or module into properties. Currently, there is no converter from instance to properties. It is a one w

Re: [DISCUSS] FLIP-67: Global partitions lifecycle

2019-10-04 Thread Chesnay Schepler
I have updated the FLIP. - consistently use "local"/"global" terminology; this incidentally should make it easier to update the terminology if we decide on other names - inform RM via heartbeats from TE about available global partitions - add dedicated method for releasing global partitions - a

Re: [VOTE] FLIP-66: Support Time Attribute in SQL DDL (#2)

2019-10-04 Thread Timo Walther
+1 for the syntax and their semantics I think the implementation part is still a bit unclear to me because it only ensures the current status but still does not solve future requirements such as per-partition watermarks that need to be pushed into a connector such as Kafka. We can also discuss

Re: [DISCUSS] FLIP-67: Global partitions lifecycle

2019-10-04 Thread Till Rohrmann
Thanks for updating the FLIP. I think the RM does not need to have access to a full fledged ShuffleMaster implementation. Instead it should enough to give it a leaner interface which only supports to delete result partitions and list available global partitions. This might entail that one will hav

Re: [DISCUSS] FLIP-74: Flink JobClient API

2019-10-04 Thread Kostas Kloudas
I also agree @Zili Chen ! On Fri, Oct 4, 2019 at 10:17 AM Aljoscha Krettek wrote: > > This makes sense to me, yes! > > > On 2. Oct 2019, at 20:35, Zili Chen wrote: > > > > Hi all, > > > > Narrow the scope to FLIP-74 we aimed at introduce a useful and extensible > > user-facing public interface J

[VOTE] FLIP-73: Introducing Executors for job submission

2019-10-04 Thread Kostas Kloudas
Hi all, I would like to start the vote for FLIP-73 [1], which is discussed and reached a consensus in the discussion thread[2]. Given that it is Friday, the vote will be open until Oct. 9th (72h starting on Monday), unless there is an objection or not enough votes. Thanks, Kostas [1] https://c

Re: [DISCUSS] FLIP-74: Flink JobClient API

2019-10-04 Thread Zili Chen
Thanks for your replies. Since no objection to Konstantin's proposal so far, I'd like to update the FLIP correspondingly. They are naming issue and exclusion of deprecated functionality. I'm hereby infer that our community generally agree on the introduction of the JobClient and its interfaces pr

[DISCUSS] FLIP-65: New type inference for Table API UDFs

2019-10-04 Thread Timo Walther
Hi everyone, I would like to propose FLIP-65 that describes how we want to deal with data types and their inference/extraction in the Table API in the future. I have collected many comments, shortcomings, issues from users and trainings in last years that went into the design. It completes the

Re: [VOTE] FLIP-73: Introducing Executors for job submission

2019-10-04 Thread Zili Chen
Thanks for your works Kostas! +1 for FLIP-73 Best, tison Kostas Kloudas 于2019年10月4日周五 下午11:40写道: > Hi all, > > I would like to start the vote for FLIP-73 [1], which is discussed and > reached a consensus in the discussion thread[2]. > > Given that it is Friday, the vote will be open until Oct

Re: [VOTE] FLIP-73: Introducing Executors for job submission

2019-10-04 Thread Thomas Weise
+1 On Fri, Oct 4, 2019 at 8:56 AM Zili Chen wrote: > Thanks for your works Kostas! > > +1 for FLIP-73 > > Best, > tison > > > Kostas Kloudas 于2019年10月4日周五 下午11:40写道: > > > Hi all, > > > > I would like to start the vote for FLIP-73 [1], which is discussed and > > reached a consensus in the disc

Re: [DISCUSS] FLIP-73: Introducing Executors for job submission

2019-10-04 Thread Thomas Weise
It might be useful to mention on FLIP-73 that the intention for Executor.execute is to be an asynchronous API once it becomes public and also refer to FLIP-74 as such. On Fri, Oct 4, 2019 at 2:52 AM Aljoscha Krettek wrote: > Hi Tison, > > I agree, for now the async Executor.execute() is an inte

ApacheCon Europe 2019 talks which are relevant to Apache Flink

2019-10-04 Thread myrle
Dear Apache Flink committers, In a little over 2 weeks time, ApacheCon Europe is taking place in Berlin. Join us from October 22 to 24 for an exciting program and lovely get-together of the Apache Community. We are also planning a hackathon.  If your project is interested in participating, p

Re: [DISCUSS] FLIP-68: Extend Core Table System with Modular Plugins

2019-10-04 Thread Xuefu Z
I agree with Timo that the new table APIs need to be consistent. I'd go further that an name (or id) is needed for module definition in YAML file. In the current design, name is skipped and type has binary meanings. Thanks, Xuefu On Fri, Oct 4, 2019 at 5:24 AM Timo Walther wrote: > Hi everyone,

Re: [VOTE] FLIP-66: Support Time Attribute in SQL DDL (#2)

2019-10-04 Thread Rong Rong
Oops.. Sorry for the confusion. I thought only PMC votes are binding. Thanks for the clarification @Timo :-D -- Rong On Fri, Oct 4, 2019 at 7:12 AM Timo Walther wrote: > +1 for the syntax and their semantics > > I think the implementation part is still a bit unclear to me because it > only ensu

Re: [VOTE] FLIP-66: Support Time Attribute in SQL DDL (#2)

2019-10-04 Thread Jark Wu
+1 from my side. Best, Jark On Sat, 5 Oct 2019 at 09:33, Rong Rong wrote: > Oops.. Sorry for the confusion. I thought only PMC votes are binding. > Thanks for the clarification @Timo :-D > > -- > Rong > > On Fri, Oct 4, 2019 at 7:12 AM Timo Walther wrote: > > > +1 for the syntax and their sem

Re: [VOTE] FLIP-66: Support Time Attribute in SQL DDL (#2)

2019-10-04 Thread Jark Wu
Thanks all for the voting. I'm closing the vote now. So far, the vote received: * +1 votes (4 binding, 2 non-binding): - Kurt Young (binding) - Danny Chan - Jingsong Lee - Rong Rong (binding) - Timo (binding) - Jark (binding) * 0/-1 votes: none Thereby, the community accepted FLIP-66. I'll

[jira] [Created] (FLINK-14320) [FLIP-66] Support Time Attribute in SQL DDL

2019-10-04 Thread Jark Wu (Jira)
Jark Wu created FLINK-14320: --- Summary: [FLIP-66] Support Time Attribute in SQL DDL Key: FLINK-14320 URL: https://issues.apache.org/jira/browse/FLINK-14320 Project: Flink Issue Type: New Feature

[jira] [Created] (FLINK-14321) Support to parse watermark statement in SQL DDL

2019-10-04 Thread Jark Wu (Jira)
Jark Wu created FLINK-14321: --- Summary: Support to parse watermark statement in SQL DDL Key: FLINK-14321 URL: https://issues.apache.org/jira/browse/FLINK-14321 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-14322) Add watermark information in TableSchema

2019-10-04 Thread Jark Wu (Jira)
Jark Wu created FLINK-14322: --- Summary: Add watermark information in TableSchema Key: FLINK-14322 URL: https://issues.apache.org/jira/browse/FLINK-14322 Project: Flink Issue Type: Sub-task

[jira] [Created] (FLINK-14323) Serialize Expression to String and parse String to Expression

2019-10-04 Thread Jark Wu (Jira)
Jark Wu created FLINK-14323: --- Summary: Serialize Expression to String and parse String to Expression Key: FLINK-14323 URL: https://issues.apache.org/jira/browse/FLINK-14323 Project: Flink Issue Ty

[jira] [Created] (FLINK-14324) Convert SqlCreateTable with SqlWatermark to CatalogTable

2019-10-04 Thread Jark Wu (Jira)
Jark Wu created FLINK-14324: --- Summary: Convert SqlCreateTable with SqlWatermark to CatalogTable Key: FLINK-14324 URL: https://issues.apache.org/jira/browse/FLINK-14324 Project: Flink Issue Type: Su

[jira] [Created] (FLINK-14325) Convert CatalogTable to TableSourceTable with additional watermark information

2019-10-04 Thread Jark Wu (Jira)
Jark Wu created FLINK-14325: --- Summary: Convert CatalogTable to TableSourceTable with additional watermark information Key: FLINK-14325 URL: https://issues.apache.org/jira/browse/FLINK-14325 Project: Flink

[jira] [Created] (FLINK-14326) Support to apply watermark assigner according to the WatermarkSpec in TableSourceTable

2019-10-04 Thread Jark Wu (Jira)
Jark Wu created FLINK-14326: --- Summary: Support to apply watermark assigner according to the WatermarkSpec in TableSourceTable Key: FLINK-14326 URL: https://issues.apache.org/jira/browse/FLINK-14326 Project:

[jira] [Created] (FLINK-14327) Getting "Could not forward element to next operator" error

2019-10-04 Thread ASK5 (Jira)
ASK5 created FLINK-14327: Summary: Getting "Could not forward element to next operator" error Key: FLINK-14327 URL: https://issues.apache.org/jira/browse/FLINK-14327 Project: Flink Issue Type: Bug