Re: [DISCUSS] Flink client api enhancement for downstream project

2019-08-20 Thread Zili Chen
path). > > > In the YARN, Mesos and Kubernetes scenarios you have the jar but you > > could > > > start a cluster that has the jar on the Job Manager as well (but this > is > > > the only case where I think you can assume that the client has the jar > on > >

Re: [DISCUSS][CODE STYLE] Breaking long function argument lists and chained method calls

2019-08-20 Thread Zili Chen
Implement question: how to apply the line length rules? If we just turn on checkstyle rule "LineLength" then a huge effort is required to break lines those break the rule. If we use an auto-formatter here then it possibly break line "just at the position" awfully. Is it possible we require only t

[SURVEY] How do you use high-availability services in Flink?

2019-08-21 Thread Zili Chen
Hi guys, We want to have an accurate idea of how users actually use high-availability services in Flink, especially how you customize high-availability services by HighAvailabilityServicesFactory. Basically there are standalone impl., zookeeper impl., embedded impl. used in MiniCluster, YARN impl

Re: [SURVEY] How do you use high-availability services in Flink?

2019-08-21 Thread Zili Chen
In addition, FLINK-13750[1] also likely introduce breaking change on high-availability services. So it is highly encouraged you who might be affected by the change share your cases :-) Best, tison. [1] https://issues.apache.org/jira/browse/FLINK-13750 Zili Chen 于2019年8月21日周三 下午3:32写道: >

Re: [DISCUSS][CODE STYLE] Usage of Java Optional

2019-08-21 Thread Zili Chen
; > >>>>>>>>> > > >>>>>>>>> It is often the case that domain objects hang about in memory > > >> for a > > >>>>>> fair > > >>>>>>>>> while, as processing in the app

Re: [DISCUSS][CODE STYLE] Breaking long function argument lists and chained method calls

2019-08-21 Thread Zili Chen
> I suggest we spawn separate discussion threads (can do as a follow-up) > about: > >- the hard line length limit in Java, possibly to confirm it also for >Scala (cc @Tison) >- indentation rules for the broken list of a declared function arguments > > If there ar

Re: CiBot Update

2019-08-21 Thread Zili Chen
Thanks for your announcement. Nice work! Best, tison. vino yang 于2019年8月22日周四 上午8:14写道: > +1 for "@flinkbot run travis", it is very convenient. > > Chesnay Schepler 于2019年8月21日周三 下午9:12写道: > > > Hi everyone, > > > > this is an update on recent changes to the CI bot. > > > > > > The bot now ca

Re: [DISCUSS][CODE STYLE] Breaking long function argument lists and chained method calls

2019-08-22 Thread Zili Chen
because parentheses are aligned in a similar way, as well as the border between declaration and function body is clear. Zili Chen 于2019年8月22日周四 上午9:53写道: > Thanks Andrey for driving the discussion. Just for clarification, > what we conclude here are several guidelines without automatic >

Re: [ANNOUNCE] Apache Flink 1.9.0 released

2019-08-22 Thread Zili Chen
Congratulations! Thanks Gordon and Kurt for being the release manager. Thanks all the contributors who have made this release possible. Best, tison. Jark Wu 于2019年8月22日周四 下午8:11写道: > Congratulations! > > Thanks Gordon and Kurt for being the release manager and thanks a lot to > all the contr

Re: [DISCUSS] Flink client api enhancement for downstream project

2019-08-22 Thread Zili Chen
loaded by user class loader > > to avoid class conflict with framework. > > > > Anyway, both of the two submission modes are useful. > > We just need to clarify the concepts. > > > > > Best, > > Yang > > Zili Chen 于2019年8月20日周二 下午5:58写道: > >

Re: [ANNOUNCE] Apache Flink 1.9.0 released

2019-08-23 Thread Zili Chen
Hi Gavin, I also find a problem in shell if the directory contain whitespace then the final command to run is incorrect. Could you check the final command to be executed? FYI, here is the ticket[1]. Best, tison. [1] https://issues.apache.org/jira/browse/FLINK-13827 Gavin Lee 于2019年8月23日周五 下午

Re: [ANNOUNCE] Apache Flink 1.9.0 released

2019-08-23 Thread Zili Chen
ectory under root folder of flink distribution. > > > On Fri, Aug 23, 2019 at 4:10 PM Zili Chen wrote: > >> Hi Gavin, >> >> I also find a problem in shell if the directory contain whitespace >> then the final command to run is incorrect. Could you check the >>

Re: [DISCUSS] Flink client api enhancement for downstream project

2019-08-23 Thread Zili Chen
ication to his running job. > > > > On the topic of not hijacking the "env.execute()" in order to get the > > Plan, I definitely agree but > > for the proposal of having a "compile()" method in the env, I would > > like to have a better look at >

Re: [ANNOUNCE] Apache Flink 1.9.0 released

2019-08-23 Thread Zili Chen
ss was removed in flink-dist_2.12-1.9 >> jar >> file. >> Seems broken here for scala 2.12, right? >> >> [1] >> >> http://mirror.bit.edu.cn/apache/flink/flink-1.9.0/flink-1.9.0-bin-scala_2.12.tgz >> >> On Fri, Aug 23, 2019 at 4:37 PM Zili Chen

Re: [DISCUSS] Use Java's Duration instead of Flink's Time

2019-08-23 Thread Zili Chen
Hi Stephan, I like the idea unify usage of time/duration api. We actually use at least five different classes for this purposes(see below). One thing I'd like to pick up is that duration configuration in Flink is almost in pattern as "60 s" that fits in the pattern parsed by scala.concurrent.dura

Re: [DISCUSS] Use Java's Duration instead of Flink's Time

2019-08-23 Thread Zili Chen
with the Duration class in Java. Java and Scala have > different types of systems. Currently, Duration (scala) and FineDuration > (scala) work well. In addition, this work brings additional complexity and > cost compared to the gains obtained. > > Best, > Vino > > Zili Chen 于201

Re: [DISSCUSS] Tolerate temporarily suspended ZooKeeper connections

2019-08-25 Thread Zili Chen
Hi Till, I'd like to revive this thread since 1.9.0 has been released. IMHO we already reached a consensus on JIRA and if you can review the pull request we hopefully address the issue in next release. Best, tison. Zili Chen 于2019年7月29日周一 下午11:05写道: > Hi Till, > > T

Re: [ANNOUNCE] Apache Flink 1.9.0 released

2019-08-26 Thread Zili Chen
Hi Oytun, I think it intents to publish flink-queryable-state-client-java without scala suffix since it is scala-free. An artifact without scala suffix has been published [2]. See also [1]. Best, tison. [1] https://issues.apache.org/jira/browse/FLINK-12602 [2] https://mvnrepository.com/artifact

[DISCUSS] Builder dedicated for testing

2019-08-26 Thread Zili Chen
Hi devs, I'd like to share an observation that we have too many @VisibleForTesting constructors that only used in test scope such as ExecutionGraph and RestClusterClient. It would be helpful if we introduce Builders in test scope for build such instance and remain the production code only necessa

Re: [VOTE] FLIP-54: Evolve ConfigOption and Configuration

2019-08-28 Thread Zili Chen
The design looks good to me. +1 go ahead! Best, tison. Jark Wu 于2019年8月28日周三 下午6:08写道: > Hi Timo, > > The new changes looks good to me. > > +1 to the FLIP. > > > Cheers, > Jark > > On Wed, 28 Aug 2019 at 16:02, Timo Walther wrote: > > > Hi everyone, > > > > after some last minute changes yes

Re: [DISCUSS] FLIP-50: Spill-able Heap Keyed State Backend

2019-08-29 Thread Zili Chen
Hi Yu, Notice that the wiki is still marked as "*Under Discussion*" state. I think you can update it correspondingly. Best, tison. Yu Li 于2019年8月20日周二 下午10:28写道: > Sorry for the lag but since we've got a consensus days ago, I started a > vote thread which will have a result by EOD, thus I'm

[PROPOSAL] Force rebase on master before merge

2019-08-29 Thread Zili Chen
Hi devs, GitHub provides a mechanism which is able to require branches to be up to date before merged[1](point 6). I can see several advantages enabling it. Thus propose our project to turn on this switch. Below are my concerns. Looking forward to your insights. 1. Avoid CI failures in pr which f

Re: [PROPOSAL] Force rebase on master before merge

2019-08-29 Thread Zili Chen
ntee the testing time. > > To summarize: I think this should be decided by the committer who is > merging the PR, > but not be forced. > > Best, > Kurt > > > On Thu, Aug 29, 2019 at 11:07 PM Zili Chen wrote: > > > Hi devs, > > > > GitHub provide

Re: [DISCUSS] Flink client api enhancement for downstream project

2019-08-30 Thread Zili Chen
link run -d -p 5 -ynm perjob-cluster1 -m yarn-cluster > > > examples/streaming/WindowJoin.jar > > > *session cluster* > > > ./bin/flink run -p 5 -ynm session-cluster1 -m yarn-cluster > > > examples/streaming/WindowJoin.jar > > > > > > What will it look like

Re: [PROPOSAL] Force rebase on master before merge

2019-09-01 Thread Zili Chen
mistakes out of the equation. > >> > >> Do you know how these status checks work concretely? Will Github reject > >> commits for which there is no passed Travis run? How would hotfix > commits > >> being distinguished from PR commits for which a Travis run s

Re: [DISCUSS] Flink client api enhancement for downstream project

2019-09-04 Thread Zili Chen
that we join forces of our community. Best, tison. Zili Chen 于2019年8月31日周六 下午12:52写道: > Great Kostas! Looking forward to your POC! > > Best, > tison. > > > Jeff Zhang 于2019年8月30日周五 下午11:07写道: > >> Awesome, @Kostas Looking forward your POC. >> >> Kost

Re: [VOTE] FLIP-61 Simplify Flink's cluster level RestartStrategy configuration

2019-09-04 Thread Zili Chen
+1 zhijiang 于2019年9月5日周四 上午12:36写道: > +1 > -- > From:Till Rohrmann > Send Time:2019年9月4日(星期三) 13:39 > To:dev > Cc:Zhu Zhu > Subject:Re: [VOTE] FLIP-61 Simplify Flink's cluster level RestartStrategy > configuration > > +1 (bindin

Re: [ANNOUNCE] Kostas Kloudas joins the Flink PMC

2019-09-06 Thread Zili Chen
Congrats Klou! Best, tison. Till Rohrmann 于2019年9月6日周五 下午9:23写道: > Congrats Klou! > > Cheers, > Till > > On Fri, Sep 6, 2019 at 3:00 PM Dian Fu wrote: > >> Congratulations Kostas! >> >> Regards, >> Dian >> >> > 在 2019年9月6日,下午8:58,Wesley Peng 写道: >> > >> > On 2019/9/6 8:55 下午, Fabian Hueske w

[DISCUSS] Retrieval services in non-high-availability scenario

2019-09-09 Thread Zili Chen
Hi devs, I'd like to start a discussion thread on the topic how we provide retrieval services in non-high-availability scenario. To clarify terminology, non-high-availability scenario refers to StandaloneHaServices and EmbeddedHaServices. ***The problem*** We notice that retrieval services of cu

Re: [DISCUSS] Retrieval services in non-high-availability scenario

2019-09-10 Thread Zili Chen
HAServices is actually orthogonal to > introducing the LeaderServer HAServices implementation and could also be > done for the existing HAServices. > > 4. Clean up of HAServices implementations: > > You are right that some of the existing HAServices implementations are > "d

Re: [DISCUSS] Retrieval services in non-high-availability scenario

2019-09-10 Thread Zili Chen
converges to be stable. Best, tison. Till Rohrmann 于2019年9月10日周二 下午6:06写道: > Hi Tison, > > thanks for the detailed response. I put some comments inline: > > On Tue, Sep 10, 2019 at 10:51 AM Zili Chen wrote: > > > Hi Till, > > > > Thanks for your reply. I agree poin

Re: [DISCUSS] Flink client api enhancement for downstream project

2019-09-11 Thread Zili Chen
think? >> >> P.S. Committers/PMCs should should be able to login with their apache ID. >> >> Best, >> Aljoscha >> >> > On 6. Sep 2019, at 14:24, Zili Chen wrote: >> > >> > Hi Aljoscha, >> > >> > I'd like to gather

Re: Interested in contributing and looking for good first issues

2019-09-12 Thread Zili Chen
Hi Anoop, Welcome to the Flick community. Although we don't maintain a list of starter issues so far, it would be helpful if you provide more information about which topic you are interested in. Flink servers stateful computations over data streams based on a series of layered concepts, such as S

Re: [DISCUSS] Use Java's Duration instead of Flink's Time

2019-09-12 Thread Zili Chen
I Xiaogang > > > > > wrote: > > > > > > > > > +1 to replace Flink's time with Java's Duration. > > > > > > > > > > Besides, i also suggest to use Java's Instant for "point-in-time". > > >

Re: [ANNOUNCE] Zili Chen becomes a Flink committer

2019-09-12 Thread Zili Chen
Thanks a lot everyone for the warm welcome. Happy Mid-autumn Festival! Best, tison. Leonard Xu 于2019年9月12日周四 上午11:05写道: > Congratulations Zili Chen ! ! > > Best, > Leonard Xu > > On 2019年9月12日, at 上午11:02, Yun Tang wrote: > > > > Congratulations

[PROPOSAL] Merge NewClusterClient into ClusterClient

2019-09-17 Thread Zili Chen
Hi devs, FLINK-14096[1] was created yesterday. It is aimed at merge the bridge class NewClusterClient into ClusterClient because with the effort under FLINK-10392 this bridge class is no longer necessary. Technically in current codebase all implementation of interface NewClusterClient is subclass

Re: [PROPOSAL] Merge NewClusterClient into ClusterClient

2019-09-17 Thread Zili Chen
tice there're many implementation details in > ClusterClient. I think we should just expose a thin interface, so maybe we > can create interface ClusterClient which includes as less methods as > possible, and move all the implementation to AbstractClusterClient. > > > > &

Re: [PROPOSAL] Merge NewClusterClient into ClusterClient

2019-09-18 Thread Zili Chen
lity for > > non-well-designed api, otherwise it will bring lots of unnecessary > > overhead. > > > > 2. Another concern is that I notice there're many implementation details > in > > ClusterClient. I think we should just expose a thin interface, so maybe >

Confluence permission for FLIP creation

2019-09-19 Thread Zili Chen
Hi devs, I'd like to create a page about the ongoing JobClient FLIP. Could you grant me Confluence permission for FLIP creation? My Confluence ID is tison. Best, tison.

Re: Confluence permission for FLIP creation

2019-09-19 Thread Zili Chen
Thanks! Best, tison. Till Rohrmann 于2019年9月19日周四 下午5:34写道: > Granted. You should now be able to create pages Tison. > > Cheers, > Till > > On Thu, Sep 19, 2019 at 11:01 AM Zili Chen wrote: > > > Hi devs, > > > > I'd like to create a page about the

Re: [PROPOSAL] Merge NewClusterClient into ClusterClient

2019-09-19 Thread Zili Chen
what Till (cc’ed) says, maybe he has an opinion on why > > the separation should be kept. > > > > Best, > > Aljoscha > > > > > On 18. Sep 2019, at 11:54, Zili Chen wrote: > > > > > > Hi Xiaogang, > > > > > > Thanks for your rep

Re: How to prevent from launching 2 jobs at the same time

2019-09-22 Thread Zili Chen
The situation is as Dian said. Flink identifies jobs by job id instead of job name. However, I think it is still a valid question if it is an alternative Flink identifies jobs by job name and leaves the work to distinguish jobs by name to users. The advantages in this way includes a readable displ

Re: [DISCUSS] FLIP-73: Introducing Executors for job submission

2019-09-24 Thread Zili Chen
Hi Thomas, >Should the new Executor execute method be defined as asynchronous? It could >return a job handle to interact with the job and the legacy environments >can still block to retain their semantics. During our discussion there will be a method executeAsync(...): CompletableFuture where J

Re: [DISCUSS] FLIP-73: Introducing Executors for job submission

2019-09-24 Thread Zili Chen
eise 于2019年9月25日周三 上午2:39写道: > Since Exceutor is a new interface, why is backward compatibility a concern? > > I could see that become an issue later when replacing Executor execute with > executeAsync. Or are both targeted for 1.10? > > > On Tue, Sep 24, 2019 at 10

[DISCUSS] Expose multiple level clients

2019-09-25 Thread Zili Chen
Hi all, While drafting FLIP-74 I notice that a job level client(called JobClient) is always retrieved from a Flink application cluster level client(called ClusterClient), which is then always retrieved from a extern cluster(YARN, mesos, k8s, etc.) level client(called ClusterDescriptor). A Flink j

[DISCUSS] FLIP-74: Flink JobClient API

2019-09-25 Thread Zili Chen
Hi all, Summary from the discussion about introducing Flink JobClient API[1] we draft FLIP-74[2] to gather thoughts and towards a standard public user-facing interfaces. This discussion thread aims at standardizing job level client API. But I'd like to emphasize that how to retrieve JobClient pos

Re: CiBot Update

2019-09-25 Thread Zili Chen
t; Zhu Zhu > > >>>>>>>> > > >>>>>>>> zhijiang > wangzhijiang...@aliyun.com.invalid>> 于2019年8月22日周四 下午4:18写道: > > >>>>>>>> > > >>>>>>>>> It is really very convenient no

[DISCUSS] Remove uncompleted YARNHighAvailabilityService

2019-09-26 Thread Zili Chen
Hi devs, Noticed that there are several stale & uncompleted high-availability services implementations, I start this thread in order to see whether or not we can remove them for a clean codebase work on the ongoing high-availability refactor effort[1]. Below are all of classes I noticed. - YarnH

Re: [DISCUSS] Expose multiple level clients

2019-09-26 Thread Zili Chen
t; > I shared some thoughts on this topic in the FLIP-74 thread, as some of > the questions are related to that. > > Cheers, > Kostas > > On Wed, Sep 25, 2019 at 12:30 PM Zili Chen wrote: > > > > Hi all, > > > > While drafting FLIP-74 I notice that a job l

Re: [DISCUSS] FLIP-74: Flink JobClient API

2019-09-26 Thread Zili Chen
out is how > to get a JobClient for a job that is already running. As you mentioned in > the document. Currently I’m thinking that its ok to add a method to > Executor for retrieving a JobClient for a running job by providing an ID. > Let’s see what Kostas has to say on the topic. > &

Re: [DISCUSS] FLIP-74: Flink JobClient API

2019-09-27 Thread Zili Chen
up to user code. [1] https://issues.apache.org/jira/browse/FLINK-14051?focusedCommentId=16931388&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16931388 [2] https://lists.apache.org/x/thread.html/e8f14a381be6c027e8945f884c3cfcb309ce49c1ba557d3749fca495@%3Cdev.f

Re: [DISCUSS] FLIP-74: Flink JobClient API

2019-09-27 Thread Zili Chen
ClusterEntrypoint. Zili Chen 于2019年9月27日周五 下午3:13写道: > About JobCluster > > Actually I am not quite sure what we gains from DETACHED configuration on > cluster side. > We don't have a NON-DETACHED JobCluster in fact in our codebase, right? > > It comes to me one major questio

Re: [DISCUSS] FLIP-73: Introducing Executors for job submission

2019-09-27 Thread Zili Chen
gt; (Stream)ExecutionEnvironment#execute(). > > > > > > This method has to stay and keep having the same (blocking) semantics, > > > but we can > > > add a new one, sth along the lines of executeAsync() that will return > > > the JobClient and > &

Re: REST API / JarRunHandler: More flexibility for launching jobs

2019-09-29 Thread Zili Chen
t; other projects can rely on at some point. If I remember correctly, then > >> we > >>> wanted to define a proto buf definition for the JobGraph so that > clients > >>> written in a different language can submit JobGraphs and we could > extend > >>> t

Re: [DISCUSS] Remove uncompleted YARNHighAvailabilityService

2019-09-30 Thread Zili Chen
Any suggestion? IMO it is exactly inactive for quite some time, we can remove these uncompleted codes at least for now and re-introduce if needed. Best, tison. Zili Chen 于2019年9月27日周五 上午9:23写道: > Hi devs, > > Noticed that there are several stale & uncompleted high-availabili

Re: [DISCUSS] Remove uncompleted YARNHighAvailabilityService

2019-09-30 Thread Zili Chen
If we should need them in the future, then we can still get them by > going back a bit in time. Hence +1 for removing unused HA implementations. > > Cheers, > Till > > On Mon, Sep 30, 2019 at 10:42 AM Zili Chen wrote: > > > Any suggestion? > > > > IMO it is exactl

[CODE STYLE] Parameters of method are always final

2019-10-01 Thread Zili Chen
Hi devs, Coming from this discussion[1] I'd like to propose that in Flink codebase we suggest a code style that parameters of method are always final. Almost everywhere parameters of method are final already and if we have such consensus we can prevent redundant final modifier in method declaratio

Re: [DISCUSS] FLIP-74: Flink JobClient API

2019-10-02 Thread Zili Chen
to > > > > > execute Flink jobs but IMHO it would ease A LOT the work of UI > > > developers > > > > > that could have a way to show the users all available jobs inside a > > > jar + > > > > > their configurable parameters. > > > > &

Re: [DISCUSS] FLIP-73: Introducing Executors for job submission

2019-10-02 Thread Zili Chen
Thanks for your thoughts Kostas! I agree Executor to be a concept on clients now. And sincerely second the description Now the Executor simply uses a client, e.g. a ClusterClient, to submit the job (JobGraph) that it will create from the user program. In that sense, the Executor is one level of a

Re: [DISCUSS] FLIP-73: Introducing Executors for job submission

2019-10-02 Thread Zili Chen
important 2. Session cluster already share resource between jobs, if a job want to have its dedicate resource, use per-job mode. Does this way fits your requirement Thomas? Zili Chen 于2019年10月2日周三 下午4:57写道: > Thanks for your thoughts Kostas! > > I agree Executor to be a concept on cl

Re: [DISCUSS] FLIP-73: Introducing Executors for job submission

2019-10-02 Thread Zili Chen
becomes more unclear what per-job looks like in user perspective. But the original purpose is towards a concrete PerJobExecutor and I want to save bandwidth by reduce concurrent coupled threads a bit. Zili Chen 于2019年10月2日周三 下午5:33写道: > Hi Till, > > The purpose to post thoughts abov

Re: [DISCUSS] FLIP-73: Introducing Executors for job submission

2019-10-02 Thread Zili Chen
't it > be possible to introduce the Executors without changing Flink's deployment > options in the first step? I don't fully understand where this > need/requirement comes from. > > Cheers, > Till > > On Wed, Oct 2, 2019 at 10:58 AM Zili Chen wrote: >

Re: [DISCUSS] FLIP-73: Introducing Executors for job submission

2019-10-02 Thread Zili Chen
Thanks for your clarification Till. I agree with the current semantics of the per-job mode, one should deploy a new cluster for each part of the job. Apart from the performance concern it also means that PerJobExecutor knows how to deploy a cluster actually, which is different from the description

Re: [CODE STYLE] Parameters of method are always final

2019-10-02 Thread Zili Chen
he > parameters). Did I understand this correctly? > > Piotrek > > > On 1 Oct 2019, at 21:44, Zili Chen wrote: > > > > Hi devs, > > > > Coming from this discussion[1] I'd like to propose that in Flink codebase > > we suggest a code style > >

Re: [DISCUSS] FLIP-73: Introducing Executors for job submission

2019-10-02 Thread Zili Chen
straction and makes it usable via API. We should closely follow up on the > above points though because I think they are also important. > > Best, > Aljoscha > > > On 2. Oct 2019, at 12:08, Zili Chen wrote: > > > > Thanks for your clarification Till. > > >

Re: [DISCUSS] FLIP-73: Introducing Executors for job submission

2019-10-02 Thread Zili Chen
as Till said, changing the semantics is beyond the scope of this > > FLIP and as Tison mentioned we should work towards decoupling > > discussions rather than the opposite. So let's discuss about the > > future of the per-job and detached modes in a separate thread. This > &

Re: [DISCUSS] FLIP-73: Introducing Executors for job submission

2019-10-02 Thread Zili Chen
since this one does not affect current behavior at all. Best, tison. Zili Chen 于2019年10月3日周四 上午2:02写道: > Hi Kostas, > > It seems does no harm we have a configuration parameter of Executor#execute > since we can merge this one with the one configured on Executor created and > let thi

Re: [DISCUSS] FLIP-73: Introducing Executors for job submission

2019-10-02 Thread Zili Chen
ent* it seems a bit diverge with you statement above. Or we say a single Environment as a possible advantage after the introduction of Executor so that we exclude it from this pass. Best, tison. Zili Chen 于2019年10月3日周四 上午2:07写道: > BTW, correct me if I misunderstand, now I learn more about our

Re: [DISCUSS] FLIP-74: Flink JobClient API

2019-10-02 Thread Zili Chen
might have some impact so I think it's better to reach a community consensus as prerequisite. Note that with all client methods are asynchronous, no matter whether or not we remove client side detach option it is no power. Let me know your ideas on these topic and keep moving forward :-) Best

Re: [DISCUSS] FLIP-73: Introducing Executors for job submission

2019-10-03 Thread Zili Chen
ute() method is described, there are details > about parts of the > integration with the existing Flink code-base. > > So I am not sure what do you mean by making the "integration a > follow-up discussion". > > Cheers, > Kostas > > On Wed, Oct 2, 2019 at 8:1

Re: [DISCUSS] FLIP-74: Flink JobClient API

2019-10-03 Thread Zili Chen
s/on the context if/on the context of/ s/dummy/dumb/

Re: [DISCUSS] FLIP-74: Flink JobClient API

2019-10-03 Thread Zili Chen
te actually has a dependency of FLIP-73. 2. retrieve JobClient of an existing job directly lead to the discussion of the retrieval chains which I started as [DISCUSS] Expose multiple level clients. Best, tison. Zili Chen 于2019年10月3日周四 上午2:35写道: > Hi all, > > Narrow the scope to FLIP-74

Re: [DISCUSS] FLIP-74: Flink JobClient API

2019-10-03 Thread Zili Chen
gt; > In the future we could consider exposing it through a > ClusterClientFactory (or sth similar). > > What do you think > Kostas > > On Thu, Oct 3, 2019 at 10:12 AM Zili Chen wrote: > > > > s/on the context if/on the context of/ > > s/dummy/dumb/ >

Re: [DISCUSS] FLIP-73: Introducing Executors for job submission

2019-10-04 Thread Zili Chen
inction between “detached” and “attached” but we can still do it in > the CLI (via the ContextEnvironment) to support the existing “detached” > behaviour of the CLI that users expect. What do you think about this? > > > > Best, > > Aljoscha > > > >> O

Re: [CODE STYLE] Parameters of method are always final

2019-10-04 Thread Zili Chen
2019, at 13:31, Piotr Nowojski wrote: > > > > +1 from my side. > > > >> On 2 Oct 2019, at 13:07, Zili Chen wrote: > >> > >> Yes exactly. > >> > >> > >> Piotr Nowojski 于2019年10月2日周三 下午7:03写道: > >> > >>> Hi

Re: [DISCUSS] FLIP-74: Flink JobClient API

2019-10-04 Thread Zili Chen
its interfaces proposed in the FLIP. If there are other concerns, please thrown into this thread. Otherwise I'm going to start a vote thread later. Best, tison. Kostas Kloudas 于2019年10月4日周五 下午11:33写道: > I also agree @Zili Chen ! > > On Fri, Oct 4, 2019 at 10:17 AM Aljoscha Krettek

Re: [VOTE] FLIP-73: Introducing Executors for job submission

2019-10-04 Thread Zili Chen
Thanks for your works Kostas! +1 for FLIP-73 Best, tison Kostas Kloudas 于2019年10月4日周五 下午11:40写道: > Hi all, > > I would like to start the vote for FLIP-73 [1], which is discussed and > reached a consensus in the discussion thread[2]. > > Given that it is Friday, the vote will be open until Oct

[VOTE] FLIP-74: Flink JobClient API

2019-10-07 Thread Zili Chen
Hi all, I would like to start the vote for FLIP-74[1], which is discussed and reached a consensus in the discussion thread[2]. The vote will be open util Oct. 9th(72h starting on Oct.7th), unless there is an objection or not enough votes. Best, tison. [1] https://cwiki.apache.org/confluence/di

Re: [CODE STYLE] Parameters of method are always final

2019-10-07 Thread Zili Chen
rule would probably fail on a good portion of > the > > current code base. But we would also remove reassignment to parameters > > (which I consider an anti-pattern). > > > > If we opt not to enforce it, then -1 for removing final keywords from my > > side. &

Re: [VOTE] Release 1.9.1, release candidate #1

2019-10-07 Thread Zili Chen
Hi Jark, I notice a critical bug[1] is marked resolved in 1.9.1 but given 1.9.1 has been cut I'd like to throw the issue here so that we're sure whether or not it is included in 1.9.1. Best, tison. [1] https://issues.apache.org/jira/browse/FLINK-14315 Jark Wu 于2019年9月30日周一 下午3:25写道: > Hi ev

Re: [VOTE] FLIP-74: Flink JobClient API

2019-10-08 Thread Zili Chen
Given the ongoing FlinkForward Berlin event, I'm going to extend this vote thread with a bit of period, said until Oct. 11th(Friday). Best, tison. Zili Chen 于2019年10月7日周一 下午4:15写道: > Hi all, > > I would like to start the vote for FLIP-74[1], which is discussed and > reached a

Re: [DISCUSS] FLIP-73: Introducing Executors for job submission

2019-10-09 Thread Zili Chen
cutor.execute() is an internal detail but > > during your work for FLIP-74 it will probably also reach the public API. > > > > Best, > > Aljoscha > > > > > On 4. Oct 2019, at 11:39, Zili Chen wrote: > > > > > > Hi Aljoscha, > > > &g

Re: [DISCUSS] FLIP-73: Introducing Executors for job submission

2019-10-10 Thread Zili Chen
rom the Executors, as the Executors simply use the > clients to submit jobs and return a cluster client. > > Cheers, > Kostas > > On Wed, Oct 9, 2019 at 7:01 PM Zili Chen wrote: > > > > Hi Kostas & Aljoscha, > > > > I'm drafting a plan exposing mu

Re: [VOTE] FLIP-74: Flink JobClient API

2019-10-11 Thread Zili Chen
> Thanks, > > Biao /'bɪ.aʊ/ > > > > > > > > On Fri, 11 Oct 2019 at 14:44, Jeff Zhang wrote: > > > > > +1, overall design make sense to me > > > > > > SHI Xiaogang 于2019年10月11日周五 上午11:15写道: > > > > > > > +1. The inte

Re: [VOTE] FLIP-74: Flink JobClient API

2019-10-11 Thread Zili Chen
ointDir) of the > clusterClient should change to return a CompletableFuture. I believe > that this change is redundant as we will not need it for the > JobClient. I should have been more clear on what I meant before. > > Cheers, > Kostas > > On Fri, Oct 11, 2019 at 11:51 AM Zili

Re: [PROPOSAL] Contribute Stateful Functions to Apache Flink

2019-10-14 Thread Zili Chen
+1 to add Stateful Function to FLINK core repository. Best, tison. Becket Qin 于2019年10月14日周一 下午4:16写道: > +1 to adding Stateful Function to Flink. It is a very useful addition to > the Flink ecosystem. > > Given this is essentially a new top-level / first-citizen API of Flink, it > seems better

Re: [VOTE] FLIP-74: Flink JobClient API

2019-10-14 Thread Zili Chen
Hi all, +1 from my side. Given the current state of this voting thread, FLIP-74 is accepted with 3 binding vote and 2 non-binding vote. Thanks for your participation! I will update the wiki to reflect that the result of the vote. Best, tison. Zili Chen 于2019年10月11日周五 下午8:48写道: > Well. T

Re: [DISCUSS] FLIP policy for introducing config option keys

2019-10-15 Thread Zili Chen
Hi Aljoscha & Dawid & Kostas, I agree that changes on config option keys deserve a FLIP and it is reasonable we commit the changes with a standard FLIP process so that ensure the change given proper visibility. My concern is about naming. Given FLIP-73 as an example, if FLIPs associated to FLIP-7

Re: [DISCUSS] FLIP policy for introducing config option keys

2019-10-15 Thread Zili Chen
59 > > > On 15. Oct 2019, at 14:31, Zili Chen wrote: > > > > Hi Aljoscha & Dawid & Kostas, > > > > I agree that changes on config option keys deserve a FLIP and it is > > reasonable > > we commit the changes with a standard FLIP process so that ens

Re: [DISCUSS] FLIP policy for introducing config option keys

2019-10-15 Thread Zili Chen
IRA label to track these changes. > > What do you think? > > Best, Hequn > > On Tue, Oct 15, 2019 at 8:43 PM Zili Chen wrote: > > > The naming concern above can be a separated issue since it looks also > > affect FLIP-54 and isn't limited for config option cha

Re: [VOTE] FLIP-77: Introduce ConfigOptions with Data Types

2019-10-16 Thread Zili Chen
+1 Best, tison. Aljoscha Krettek 于2019年10月16日周三 下午6:25写道: > +1 > > > On 16. Oct 2019, at 10:30, Timo Walther wrote: > > > > +1 > > > > Thanks, > > Timo > > > > On 15.10.19 17:07, Till Rohrmann wrote: > >> Sorry for the confusion. I should have checked with an external mail > >> client. Thanks

[DISCUSS] What do we gain by supporting customized High-Availability services

2019-10-17 Thread Zili Chen
Hi devs, Recently the community excludes customize support on new restart strategies[1], which reminds me to think of which kind of customized support a framework like Flink should provides. The key idea is pluggable is not customizable. We might handle a series of implementation of restart stra

Re: [DISCUSS] What do we gain by supporting customized High-Availability services

2019-10-17 Thread Zili Chen
A challenge is how we ensure the support for customized implementation. When we introduce JobGraphStore#releaseJobGraph we actually change quite a bit codepath in Dispatcher. While we are unable to test arbitrarily customized implementation our compatibility promise is actually no more than compila

Re: [NOTICE] Binary licensing is now auto-generated

2019-10-17 Thread Zili Chen
Thanks for this improvement Chesnay ;-) Best, tison. Chesnay Schepler 于2019年10月17日周四 下午9:37写道: > Hello, > > I just merged FLINK-14008 to 1.8, 1.9 and 1.10, which means that from > now on the tricky part of the binary licensing (NOTICE-binary, > licenses-binary) is automatically generated durin

Re: [DISCUSS] What do we gain by supporting customized High-Availability services

2019-10-17 Thread Zili Chen
ulsar/wiki/PIP-45%3A-Pluggable-metadata-interface Zili Chen 于2019年10月17日周四 下午8:13写道: > A challenge is how we ensure the support for customized implementation. > When > we introduce JobGraphStore#releaseJobGraph we actually change quite a bit > codepath > in Dispatcher. While w

Re: [ANNOUNCE] Apache Flink 1.9.1 released

2019-10-19 Thread Zili Chen
Thanks a lot for being release manager Jark. Great work! Best, tison. Till Rohrmann 于2019年10月19日周六 下午10:15写道: > Thanks a lot for being our release manager Jark and thanks to everyone who > has helped to make this release possible. > > Cheers, > Till > > On Sat, Oct 19, 2019 at 3:26 PM Jark Wu

Re: [VOTE] Accept Stateful Functions into Apache Flink

2019-10-21 Thread Zili Chen
+1 (non-binding) Best, tison. Kostas Kloudas 于2019年10月21日周一 下午11:49写道: > +1 (binding) > > On Mon, Oct 21, 2019 at 5:18 PM Aljoscha Krettek > wrote: > > > > +1 (binding) > > > > Aljoscha > > > > > On 21. Oct 2019, at 16:18, Thomas Weise wrote: > > > > > > +1 (binding) > > > > > > > > > On Mon

Re: [VOTE] FLIP-81: Executor-related new ConfigOptions

2019-10-23 Thread Zili Chen
Thanks for starting this voting process. I have looked at the FLIP and the discussion thread. These options added make sense to me. +1 from my side. Best, tison. Kostas Kloudas 于2019年10月23日周三 下午4:12写道: > Hi all, > > This is the voting thread for FLIP-81, as the title says. > > The FLIP can be

Re: [DISCUSS] What do we gain by supporting customized High-Availability services

2019-10-23 Thread Zili Chen
own HA services at the > moment. This underlines as well that this interface is an internal API. > > Cheers, > Till > > On Fri, Oct 18, 2019 at 5:56 AM Zili Chen wrote: > > > Another perspective is that a stable, carefully-designed interface with > > clear semant

Re: [ANNOUNCE] Becket Qin joins the Flink PMC

2019-10-28 Thread Zili Chen
Congratulations Becket! Best, tison. Congxian Qiu 于2019年10月29日周二 上午9:53写道: > Congratulations Becket! > > Best, > Congxian > > > Wei Zhong 于2019年10月29日周二 上午9:42写道: > > > Congratulations Becket! > > > > Best, > > Wei > > > > > 在 2019年10月29日,09:36,Paul Lam 写道: > > > > > > Congrats Becket! > > >

  1   2   3   >