Hi, @Aljoscha, I believe that it is better to be done like this so that we do not step on each-other's feet. If the executor already "knew" about the JobClient, then we should also know about how we expect the JobExecutionResult is retrieved (which is part of FLIP-74). I think it is nice to have each discussion self-contained.
Cheers, Kostas On Wed, Sep 25, 2019 at 2:13 PM Aljoscha Krettek <aljos...@apache.org> wrote: > > Hi, > > I’m fine with either signature for the new execute() method but I think we > should focus on the executor discovery and executor configuration part in > this FLIP while FLIP-74 is about the evolution of the method signature to > return a future. > > I understand that it’s a bit weird, that this FLIP introduces a new interface > only to be changed within the same Flink release in a follow-up FLIP. But I > think we can still do it. What do you think? > > Best, > Aljoscha > > > On 25. Sep 2019, at 10:11, Kostas Kloudas <kklou...@gmail.com> wrote: > > > > Hi Thomas and Zili, > > > > As you both said the Executor is a new addition so there are no > > compatibility concerns. > > Backwards compatibility comes into play on the > > (Stream)ExecutionEnvironment#execute(). > > > > This method has to stay and keep having the same (blocking) semantics, > > but we can > > add a new one, sth along the lines of executeAsync() that will return > > the JobClient and > > will allow the caller to interact with the job. > > > > Cheers, > > Kostas > > > > On Wed, Sep 25, 2019 at 2:44 AM Zili Chen <wander4...@gmail.com> wrote: > >> > >>> Since Exceutor is a new interface, why is backward compatibility a > >>> concern? > >> > >> For backward compatibility, it is on (Stream)ExecutionEnvironment#execute. > >> You're right that we don't stick to blocking to return a > >> JobExecutionResult in > >> Executor aspect but implementing env.execute with a unique > >> > >> Executor#execute(or with suffix Async): CompletableFuture<JobClient> > >> > >> what do you think @Kostas Kloudas? > >> > >>> I could see that become an issue later when replacing Executor execute > >>> with > >>> executeAsync. Or are both targeted for 1.10? > >> > >> IIUC both Executors and JobClient are targeted for 1.10. > >> > >> > >> Thomas Weise <t...@apache.org> 于2019年9月25日周三 上午2:39写道： > >>> > >>> Since Exceutor is a new interface, why is backward compatibility a > >>> concern? > >>> > >>> I could see that become an issue later when replacing Executor execute > >>> with > >>> executeAsync. Or are both targeted for 1.10? > >>> > >>> > >>> On Tue, Sep 24, 2019 at 10:24 AM Zili Chen <wander4...@gmail.com> wrote: > >>> > >>>> Hi Thomas, > >>>> > >>>>> Should the new Executor execute method be defined as asynchronous? It > >>>> could > >>>>> return a job handle to interact with the job and the legacy environments > >>>>> can still block to retain their semantics. > >>>> > >>>> During our discussion there will be a method > >>>> > >>>> executeAsync(...): CompletableFuture<JobClient> > >>>> > >>>> where JobClient can be regarded as job handle in your context. > >>>> > >>>> I think we remain > >>>> > >>>> execute(...): JobExecutionResult > >>>> > >>>> just for backward compatibility because this effort towards 1.10 which is > >>>> not a > >>>> major version bump. > >>>> > >>>> BTW, I am drafting details of JobClient(as FLIP-74). Will start a > >>>> separated > >>>> discussion > >>>> thread on that interface as soon as I finish an early version. > >>>> > >>>> Best, > >>>> tison. > >>>> > >>>> > >>>> Thomas Weise <t...@apache.org> 于2019年9月25日周三 上午1:17写道： > >>>> > >>>>> Thanks for the proposal. These changes will make it significantly easier > >>>> to > >>>>> programmatically use Flink in downstream frameworks. > >>>>> > >>>>> Should the new Executor execute method be defined as asynchronous? It > >>>> could > >>>>> return a job handle to interact with the job and the legacy environments > >>>>> can still block to retain their semantics. > >>>>> > >>>>> (The blocking execution has also made things more difficult in Beam, we > >>>>> could simply switch to use Executor directly.) > >>>>> > >>>>> Thomas > >>>>> > >>>>> > >>>>> On Tue, Sep 24, 2019 at 6:48 AM Kostas Kloudas <kklou...@apache.org> > >>>>> wrote: > >>>>> > >>>>>> Hi all, > >>>>>> > >>>>>> In the context of the discussion about introducing the Job Client API > >>>>> [1], > >>>>>> there was a side-discussion about refactoring the way users submit jobs > >>>>> in > >>>>>> Flink. There were many different interesting ideas on the topic and 3 > >>>>>> design documents that were trying to tackle both the issue about code > >>>>>> submission and the Job Client API. > >>>>>> > >>>>>> This discussion thread aims at the job submission part and proposes the > >>>>>> approach of introducing the Executor abstraction which will abstract > >>>> the > >>>>>> job submission logic from the Environments and will make it API > >>>> agnostic. > >>>>>> > >>>>>> The FLIP can be found at [2]. > >>>>>> > >>>>>> Please keep the discussion here, in the mailing list. > >>>>>> > >>>>>> Looking forward to your opinions, > >>>>>> Kostas > >>>>>> > >>>>>> [1] > >>>>>> > >>>>>> > >>>>> > >>>> https://lists.apache.org/thread.html/ce99cba4a10b9dc40eb729d39910f315ae41d80ec74f09a356c73938@%3Cdev.flink.apache.org%3E > >>>>>> [2] > >>>>>> > >>>>>> > >>>>> > >>>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-73%3A+Introducing+Executors+for+job+submission > >>>>>> > >>>>> > >>>> >
