Re: [VOTE] Flink Project Bylaws

2019-08-27 Thread Becket Qin
Thanks for voting, Jincheng, Kostas and Marton! By now the vote counts as following: +1 (Binding): 16 (Aljoscha, Fabian, Kurt, Till, Timo, Max, Stephan, Gordon, Robert, Ufuk, Chesnay, Shaoxuan, Henry, Jincheng, Kostas, Marton) +0 (Binding): 1 (Thomas) +1 (Non-Binding): 11 (Hequn, Vino, Piotr, Da

Re: [VOTE] FLIP-58: Flink Python User-Defined Function for Table API

2019-08-27 Thread jincheng sun
Hi Dian, +1, Thanks for your great job! Best, Jincheng Dian Fu 于2019年8月28日周三 上午11:04写道: > Hi all, > > I'd like to start a voting thread for FLIP-58 [1] since that we have > reached an agreement on the design in the discussion thread [2], > > This vote will be open for at least 72 hours. Unless

[jira] [Created] (FLINK-13880) The behavior of JobExecutionResult.getAccumulatorResult does not match its java doc

2019-08-27 Thread Caizhi Weng (Jira)
Caizhi Weng created FLINK-13880: --- Summary: The behavior of JobExecutionResult.getAccumulatorResult does not match its java doc Key: FLINK-13880 URL: https://issues.apache.org/jira/browse/FLINK-13880 Pro

Re: [ANNOUNCE] Apache Flink 1.9.0 released

2019-08-27 Thread Fabian Hueske
Hi everyone, The Docker images for Flink 1.9.0 with Scala 2.12 are available now :-) Cheers, Fabian Oytun Tez schrieb am Di., 27. Aug. 2019, 21:18: > Thank you, Fabian! We are migrating soon once 2.12 is available. > > Cheers, > > --- > Oytun Tez > > *M O T A W O R D* > The World's Fastest Hum

Re: [DISCUSS] Flink Python User-Defined Function for Table API

2019-08-27 Thread jincheng sun
Thanks for the feedback Bowen! Great thanks for create the FLIP and bring up the VOTE Dian! Best, Jincheng Dian Fu 于2019年8月28日周三 上午11:32写道: > Hi all, > > I have started a voting thread [1]. Thanks a lot for your help during > creating the FLIP @Jincheng. > > > Hi Bowen, > > Very appreciated fo

Re: [DISCUSS] Flink Python User-Defined Function for Table API

2019-08-27 Thread Dian Fu
Hi all, I have started a voting thread [1]. Thanks a lot for your help during creating the FLIP @Jincheng. Hi Bowen, Very appreciated for your comments. I have replied you in the design doc. As it seems that the comments doesn't affect the overall design, I'll not cancel the vote for now and

Re: [DISCUSS] Flink Python User-Defined Function for Table API

2019-08-27 Thread Bowen Li
Hi Jincheng and Dian, Sorry for being late to the party. I took a glance at the proposal, LGTM in general, and I left only a couple comments. Thanks, Bowen On Mon, Aug 26, 2019 at 8:05 PM Dian Fu wrote: > Hi Jincheng, > > Thanks! It works. > > Thanks, > Dian > > > 在 2019年8月27日,上午10:55,jinchen

[VOTE] FLIP-58: Flink Python User-Defined Function for Table API

2019-08-27 Thread Dian Fu
Hi all, I'd like to start a voting thread for FLIP-58 [1] since that we have reached an agreement on the design in the discussion thread [2], This vote will be open for at least 72 hours. Unless there is an objection, I will try to close it by Sept 2, 2019 00:00 UTC if we have received sufficie

[jira] [Created] (FLINK-13879) It seems that there might be some problem with the DataStream.keyBy(xx).maxBy(yy)

2019-08-27 Thread liusong (Jira)
liusong created FLINK-13879: --- Summary: It seems that there might be some problem with the DataStream.keyBy(xx).maxBy(yy) Key: FLINK-13879 URL: https://issues.apache.org/jira/browse/FLINK-13879 Project: Flin

Is there Go client for Flink?

2019-08-27 Thread Dagang Wei
Hi folks, Seems Flink only provides REST API, wondering is there Go client available? Dagang

Re: [DISCUSS] FLIP-55: Introduction of a Table API Java Expression DSL

2019-08-27 Thread David Anderson
TImo, While it's not exactly pretty, I don't mind the $("field") construct. It's not particularly surprising. The v() method troubles me more; it looks mysterious. I think we would do better to have something more explicit. val() isn't much better -- val("foo") could be interpreted to mean the val

[DISCUSS] FLIP-57 - Rework FunctionCatalog

2019-08-27 Thread Bowen Li
Hi folks, I'd like to kick off a discussion on reworking Flink's FunctionCatalog. It's critically helpful to improve function usability in SQL. https://docs.google.com/document/d/1w3HZGj9kry4RsKVCduWp82HkW6hhgi2unnvOAUS72t8/edit?usp=sharing In short, it: - adds support for precise function refer

[jira] [Created] (FLINK-13878) Flink CLI fails (TimeoutException) for unknown reason

2019-08-27 Thread Mitch Wasson (Jira)
Mitch Wasson created FLINK-13878: Summary: Flink CLI fails (TimeoutException) for unknown reason Key: FLINK-13878 URL: https://issues.apache.org/jira/browse/FLINK-13878 Project: Flink Issue T

Re: Customize StreamingFileSink: Enable extending StreamingFileSink class

2019-08-27 Thread Kailash Dayanand
Apologies for the delay on this one. I have created the jira ticket and will continue discussion there: https://issues.apache.org/jira/browse/FLINK-13864. On Sat, Aug 24, 2019 at 7:48 AM Thomas Weise wrote: > Hi Kailash, > > Any update on this? > > Thanks, > Thomas > > > On Wed, Aug 7, 2019 at 7

[jira] [Created] (FLINK-13877) Support Hive version 2.1.0 and 2.1.1

2019-08-27 Thread Xuefu Zhang (Jira)
Xuefu Zhang created FLINK-13877: --- Summary: Support Hive version 2.1.0 and 2.1.1 Key: FLINK-13877 URL: https://issues.apache.org/jira/browse/FLINK-13877 Project: Flink Issue Type: Improvement

Re: [DISCUSS] FLIP-49: Unified Memory Configuration for TaskExecutors

2019-08-27 Thread Stephan Ewen
One note on the Environment Variables and Configuration discussion. My understanding is that passed ENV variables are added to the configuration in the "GlobalConfiguration.loadConfig()" method (or similar). For all the code inside Flink, it looks like the data was in the config to start with, jus

Re: [DISCUSS] FLIP-55: Introduction of a Table API Java Expression DSL

2019-08-27 Thread Timo Walther
Hi David, thanks for your feedback. With the current design, the DSL would be free of any ambiguity but it is definitely more verbose esp. around defining values. I would be happy about further suggestions that make the DSL more readable. I'm also not sure if we go for `$()` and `v()` instea

Re: [DISCUSS] FLIP-55: Introduction of a Table API Java Expression DSL

2019-08-27 Thread David Anderson
In general I'm in favor of anything that is going to make the Table API easier to learn and more predictable in its behavior. This proposal kind of falls in the middle. As someone who has spent hours in the crevices between the various flavors of the current implementations, I certainly view keepin

Re: [DISCUSS] FLIP-54: Evolve ConfigOption and Configuration

2019-08-27 Thread Timo Walther
Hi everyone, I updated the FLIP proposal one more time as mentioned in the voting thread. If there are no objections, I will start a new voting thread tomorrow at 9am Berlin time. Thanks, Timo On 22.08.19 14:19, Timo Walther wrote: Hi everyone, thanks for all the feedback we have received

[jira] [Created] (FLINK-13876) Remove ExecutionConfig field from PojoSerializer

2019-08-27 Thread Dawid Wysakowicz (Jira)
Dawid Wysakowicz created FLINK-13876: Summary: Remove ExecutionConfig field from PojoSerializer Key: FLINK-13876 URL: https://issues.apache.org/jira/browse/FLINK-13876 Project: Flink Issu

[jira] [Created] (FLINK-13875) Add missing redirects to the flink documentation

2019-08-27 Thread Seth Wiesman (Jira)
Seth Wiesman created FLINK-13875: Summary: Add missing redirects to the flink documentation Key: FLINK-13875 URL: https://issues.apache.org/jira/browse/FLINK-13875 Project: Flink Issue Type:

[jira] [Created] (FLINK-13874) StreamingFileSink fails to recover (truncate) properly

2019-08-27 Thread Gyula Fora (Jira)
Gyula Fora created FLINK-13874: -- Summary: StreamingFileSink fails to recover (truncate) properly Key: FLINK-13874 URL: https://issues.apache.org/jira/browse/FLINK-13874 Project: Flink Issue Type

Re: [VOTE] FLIP-54: Evolve ConfigOption and Configuration

2019-08-27 Thread Timo Walther
Hi Dawid, I'm a big fan of immutability and your suggestion makes sense to me. Furthermore, I thought about the class name clashes of ConfigOptionGroup and the existing ConfigGroup annotation. Maybe we should come up with a better name. I will make some additional adjustments to the FLIP. I

Re: [DISCUSS] FLIP-53: Fine Grained Resource Management

2019-08-27 Thread Xintong Song
Thanks for the correction, Till. Regarding your comments: - You are right, we should not change the edge type for streaming jobs. Then I think we can change the option 'allSourcesInSamePipelinedRegion' in step 2 to 'isStreamingJob', and implement the current step 2 before the current step 1 so we

Re: [VOTE] FLIP-54: Evolve ConfigOption and Configuration

2019-08-27 Thread Dawid Wysakowicz
Actually I wanted to propose a slight change to the proposal. Therefore I want to change my vote to -1 for now. I suggest to change the Configurable interface to ConfigurableFactory: public interface ConfigurableFactory {     /**     * Creates an instance from the given configuration.     */

Re: [DISCUSS] FLIP-49: Unified Memory Configuration for TaskExecutors

2019-08-27 Thread Xintong Song
Thanks for the comments, Till. I've also seen your comments on the wiki page, but let's keep the discussion here. - Regarding 'TaskExecutorSpecifics', how do you think about naming it 'TaskExecutorResourceSpecifics'. - Regarding passing memory configurations into task executors, I'm in favor of d

Re: [VOTE] FLIP-49: Unified Memory Configuration for TaskExecutors

2019-08-27 Thread Xintong Song
Alright, then let's keep the discussion in the DISCUSS mailing thread, and see whether we need to restart the vote. Thank you~ Xintong Song On Tue, Aug 27, 2019 at 8:12 PM Till Rohrmann wrote: > I had a couple of comments concerning the implementation plan. I've posted > them to the original

Re: [DISCUSS] FLIP-53: Fine Grained Resource Management

2019-08-27 Thread Till Rohrmann
Thanks for creating the implementation plan Xintong. Overall, the implementation plan looks good. I had a couple of comments: - What will happen if a user has defined a streaming job with two slot sharing groups? Would the code insert a blocking data exchange between these two groups? If yes, then

Re: [VOTE] FLIP-54: Evolve ConfigOption and Configuration

2019-08-27 Thread JingsongLee
+1 non-binding Best, Jingsong Lee -- From:Dawid Wysakowicz Send Time:2019年8月27日(星期二) 13:55 To:dev ; Timo Walther Subject:Re: [VOTE] FLIP-54: Evolve ConfigOption and Configuration +1 to the FLIP Also I think we should mention tha

Re: [CODE-STYLE] Builder pattern

2019-08-27 Thread Till Rohrmann
Hi all, I would be in favour of the following convention 1. a) static method for builder creation 2. No strict rule because especially for boolean flags it might make sense to have something like `enableX()` or `withY()` where one doesn't specify a concrete value. 3. Mutable builders but if there

Re: [DISCUSS] FLIP-53: Fine Grained Resource Management

2019-08-27 Thread Till Rohrmann
I guess there is a typo since the link to the FLIP-53 is https://cwiki.apache.org/confluence/display/FLINK/FLIP-53%3A+Fine+Grained+Operator+Resource+Management Cheers, Till On Tue, Aug 27, 2019 at 1:42 PM Xintong Song wrote: > Added implementation steps for this FLIP on the wiki page [1]. > > >

Re: [VOTE] FLIP-49: Unified Memory Configuration for TaskExecutors

2019-08-27 Thread Till Rohrmann
I had a couple of comments concerning the implementation plan. I've posted them to the original discussion thread. Depending on the outcome of this discussion we might need to restart the vote. Cheers, Till On Tue, Aug 27, 2019 at 11:14 AM Xintong Song wrote: > Hi all, > > I would like to start

Re: [ANNOUNCE] Apache Flink 1.9.0 released

2019-08-27 Thread Fabian Hueske
Hi all, Flink 1.9 Docker images are available at Docker Hub [1] now. Due to some configuration issue, there are only Scala 2.11 issues at the moment but this was fixed [2]. Flink 1.9 Scala 2.12 images should be available soon. Cheers, Fabian [1] https://hub.docker.com/_/flink [2] https://github.

Re: [DISCUSS] FLIP-49: Unified Memory Configuration for TaskExecutors

2019-08-27 Thread Till Rohrmann
Hi Xintong, thanks for addressing the comments and adding a more detailed implementation plan. I have a couple of comments concerning the implementation plan: - The name `TaskExecutorSpecifics` is not really descriptive. Choosing a different name could help here. - I'm not sure whether I would pa

Re: [VOTE] FLIP-54: Evolve ConfigOption and Configuration

2019-08-27 Thread Dawid Wysakowicz
+1 to the FLIP Also I think we should mention that the voting will last at least 72 hours as requested by the bylaws until 30 Aug 14:00 CEST. (Correct me if I am wrong Timo) On 27/08/2019 13:32, Jark Wu wrote: > +1 to the FLIP. > > > Regards, > Jark > >> 在 2019年8月27日,19:28,Timo Walther 写道: >> >>

Re: [DISCUSS] Builder dedicated for testing

2019-08-27 Thread Till Rohrmann
+1 for trying to avoid cluttering production code with testing code. Whenever possible we should add a testing utility to fulfill our testing requirements instead of adding the code to the production class. Cheers, Till On Tue, Aug 27, 2019 at 11:07 AM SHI Xiaogang wrote: > Hi Tison, > > Thanks

[jira] [Created] (FLINK-13873) Change the colum family as tags for influxdb reporter

2019-08-27 Thread ouyangwulin (Jira)
ouyangwulin created FLINK-13873: --- Summary: Change the colum family as tags for influxdb reporter Key: FLINK-13873 URL: https://issues.apache.org/jira/browse/FLINK-13873 Project: Flink Issue Typ

Re: [CODE-STYLE] Builder pattern

2019-08-27 Thread Timo Walther
Hi all, great to put this code style discussion on the mailing list because I also have found this style inconsistent in the past. Regarding Gyula's suggestions: 1. a static method `builder()` I think IDEs are also hightlight methods with this name 2. I would vote for a more declarative `prop

Re: [DISCUSS] FLIP-53: Fine Grained Resource Management

2019-08-27 Thread Xintong Song
Added implementation steps for this FLIP on the wiki page [1]. Thank you~ Xintong Song [1] https://cwiki.apache.org/confluence/display/FLINK/FLIP-49%3A+Unified+Memory+Configuration+for+TaskExecutors On Mon, Aug 19, 2019 at 10:29 PM Xintong Song wrote: > Hi everyone, > > As Till suggested, t

Re: [VOTE] FLIP-54: Evolve ConfigOption and Configuration

2019-08-27 Thread Jark Wu
+1 to the FLIP. Regards, Jark > 在 2019年8月27日,19:28,Timo Walther 写道: > > Hi everyone, > > thanks for the great feedback we have received for the draft of FLIP-54. The > discussion seems to have reached an agreement. Of course this doesn't mean > that we can't propose further improvements on

[VOTE] FLIP-54: Evolve ConfigOption and Configuration

2019-08-27 Thread Timo Walther
Hi everyone, thanks for the great feedback we have received for the draft of FLIP-54. The discussion seems to have reached an agreement. Of course this doesn't mean that we can't propose further improvements on ConfigOption's and Flink configuration in general in the future. It is just one st

[jira] [Created] (FLINK-13872) Translate Operations Playground to Chinese

2019-08-27 Thread Fabian Hueske (Jira)
Fabian Hueske created FLINK-13872: - Summary: Translate Operations Playground to Chinese Key: FLINK-13872 URL: https://issues.apache.org/jira/browse/FLINK-13872 Project: Flink Issue Type: Task

[jira] [Created] (FLINK-13871) Consolidate isRunning, canceled, isFinished fields in StreamTask and SourceStreamTask

2019-08-27 Thread Alex (Jira)
Alex created FLINK-13871: Summary: Consolidate isRunning, canceled, isFinished fields in StreamTask and SourceStreamTask Key: FLINK-13871 URL: https://issues.apache.org/jira/browse/FLINK-13871 Project: Flink

Re: How to use Flink Command flink run -yt -C?

2019-08-27 Thread LakeShen
I Have solve this problem , thanks. LakeShen 于2019年8月27日周二 下午4:55写道: > Hi community , I have a question about flink command, when I use flink run > command to submit my flink job to yarn , I use -yt to upload my function > jar , but I set -C file:xxxfunction.jar , the flink job throw the excepti

[VOTE] FLIP-49: Unified Memory Configuration for TaskExecutors

2019-08-27 Thread Xintong Song
Hi all, I would like to start the voting process for FLIP-49 [1], which is discussed and reached consensus in this thread [2]. This voting will be open for at least 72 hours. I'll try to close it Aug. 30 10:00 UTC, unless there is an objection or not enough votes. Thank you~ Xintong Song [1]

Re: [DISCUSS] Builder dedicated for testing

2019-08-27 Thread SHI Xiaogang
Hi Tison, Thanks for bringing this up to discussion. I think it's helpful to reducing unnecessary constructors with instance builders in test scope. Now certain classes, e.g., Execution, ExecutionVertex and StateHandle, are instantiated (including mocking and spying) here and there in the test co

How to use Flink Command flink run -yt -C?

2019-08-27 Thread LakeShen
Hi community , I have a question about flink command, when I use flink run command to submit my flink job to yarn , I use -yt to upload my function jar , but I set -C file:xxxfunction.jar , the flink job throw the exception like this : The main method caused an error: Could not find class 'com.you

Re: [DISCUSS] Enhance Support for Multicast Communication Pattern

2019-08-27 Thread Jark Wu
Hi all, Thanks Yun for bringing this topic. I missed this discussion because of the "multicast" title. After reading the design, if I understand correctly, it is proposing a custom event mach mechanism, i.e. broadcasting custom event. It is a orthogonality topic with multicasting. So I would sugge

[jira] [Created] (FLINK-13870) within method should applied in every individual pattern

2019-08-27 Thread YufeiLiu (Jira)
YufeiLiu created FLINK-13870: Summary: within method should applied in every individual pattern Key: FLINK-13870 URL: https://issues.apache.org/jira/browse/FLINK-13870 Project: Flink Issue Type:

Re: [VOTE] Flink Project Bylaws

2019-08-27 Thread Márton Balassi
+1 On Fri, Aug 23, 2019 at 11:40 AM Kostas Tzoumas wrote: > +1 > > On Thu, Aug 22, 2019 at 5:29 PM jincheng sun > wrote: > >> +1 >> >> Becket Qin 于2019年8月22日 周四16:22写道: >> >> > Hi All, so far the votes count as following: >> > >> > +1 (Binding): 13 (Aljoscha, Fabian, Kurt, Till, Timo, Max, Step

Re: [CODE-STYLE] Builder pattern

2019-08-27 Thread Kostas Kloudas
Hi all, I agree with Arvid, although for point 2 I would be less strict. @Piotr, for the side note you mentioned, and from the description you mention in the mail for example I, it seems that the need to pass parameters in the build() is not an inherent need of the build pattern but it can be mit

[jira] [Created] (FLINK-13869) Hive built-in function can not work in blink planner stream mode

2019-08-27 Thread Terry Wang (Jira)
Terry Wang created FLINK-13869: -- Summary: Hive built-in function can not work in blink planner stream mode Key: FLINK-13869 URL: https://issues.apache.org/jira/browse/FLINK-13869 Project: Flink

Re: [CODE-STYLE] Builder pattern

2019-08-27 Thread Arvid Heise
Hi all, I'd like to differentiate between API level builder usage and "internal" builder usage (for example, test harness). For API level builder, in general everything goes, as long as it aligns with user expectations. API level usages are also much more discussed, such that I'd expect them to b

Re: [DISCUSS] Enhance Support for Multicast Communication Pattern

2019-08-27 Thread Piotr Nowojski
Hi, Before starting a work on the design doc, I would suggest to find someone to shepherd this project. Otherwise this effort might drown among other parallel things. I could take care of that from the runtime perspective, however most of the changes are about the API and changes, which are out