>
>>> Besides, by adding an compiling option, we need to consider more things
>>> when submitting a job like "Is my program including multiple job?" or "Does
>>> the program need to be initialized before submitting to a remote cluster?",
>>> which l
l vote for the per-program new concept but I may not
>> prefer the opt-in option mentioned in the mailing list or maybe we need to
>> reconsider a better concept and definition which is easy to understand.
>>
>>
>>
>> Best,
>>
>> Jiayi Liao
>>
>
tioned in the mailing list or maybe we need to
> reconsider a better concept and definition which is easy to understand.
>
>
>
> Best,
>
> Jiayi Liao
>
> Original Message
> *Sender:* Rong Rong
> *Recipient:* Regina"
> *Cc:* Theo Diefenthal;
> user@flink.apache.org
&
and we have found flink (as os 1.3/1.6.4) cannot handle
graphs of that size. We’re currently testing 1.9.1 but have not retested the
large graph scenario.
Billy
From: Paul Lam [mailto:paullin3...@gmail.com]
Sent: Wednesday, October 30, 2019 8:41 AM
To: SHI Xiaogang
Cc: tison; dev; user
regards
>
>
> --
>
> *Von: *"Flavio Pompermaier"
> *An: *"Yang Wang"
> *CC: *"tison" , "Newport, Billy" <
> billy.newp...@gs.com>, "Paul Lam" , "SHI Xiaogang"
> , "dev&quo
requests. In the
model with multiple jobs in a program, there’s at least the opportunity to
reuse idle taskmanagers.
From: Theo Diefenthal
Sent: Thursday, October 31, 2019 10:56 AM
To: user@flink.apache.org
Subject: Re: [DISCUSS] Semantic and implementation of per-job mode
I agree with Gyula
etested the
large graph scenario.
Billy
From: Paul Lam [mailto: [ mailto:paullin3...@gmail.com | paullin3...@gmail.com
] ]
Sent: Wednesday, October 30, 2019 8:41 AM
To: SHI Xiaogang
Cc: tison; dev; user
Subject: Re: [DISCUSS] Semantic and implementation of per-job mode
Hi,
T
s in it, we do a job graph per table rather than a
>>> single job graph that does all tables instead. A single job graph can be in
>>> the tens of thousands of nodes in our largest cases and we have found flink
>>> (as os 1.3/1.6.4) cannot handle graphs of that size. We’r
y
>>
>>
>>
>>
>>
>> *From:* Paul Lam [mailto:paullin3...@gmail.com]
>> *Sent:* Wednesday, October 30, 2019 8:41 AM
>> *To:* SHI Xiaogang
>> *Cc:* tison; dev; user
>> *Subject:* Re: [DISCUSS] Semantic and implementation of per-job mode
>>
>&g
d flink
>> (as os 1.3/1.6.4) cannot handle graphs of that size. We’re currently
>> testing 1.9.1 but have not retested the large graph scenario.
>>
>>
>>
>> Billy
>>
>>
>>
>>
>>
>> *From:* Paul Lam [mailto:paullin3...@gmail.com]
>> *Sent:*
gang
> *Cc:* tison; dev; user
> *Subject:* Re: [DISCUSS] Semantic and implementation of per-job mode
>
>
>
> Hi,
>
>
>
> Thanks for starting the discussion.
>
>
>
> WRT the per-job semantic, it looks natural to me that per-job means
> per-job-graph,
>
>
[mailto:paullin3...@gmail.com]
Sent: Wednesday, October 30, 2019 8:41 AM
To: SHI Xiaogang
Cc: tison; dev; user
Subject: Re: [DISCUSS] Semantic and implementation of per-job mode
Hi,
Thanks for starting the discussion.
WRT the per-job semantic, it looks natural to me that per-job means
per-job-graph
Hi,
Thanks for starting the discussion.
WRT the per-job semantic, it looks natural to me that per-job means
per-job-graph,
because in my understanding JobGraph is the representation of a job. Could you
share some use case in which a user program should contain multiple job graphs?
WRT the per-
Hi
Thanks for bringing this.
The design looks very nice to me in that
1. In the new per-job mode, we don't need to compile user programs in the
client and can directly run user programs with user jars. That way, it's
easier for resource isolation in multi-tenant platforms and is much safer.
2. Th
(CC user list because I think users may have ideas on how per-job mode
should look like)
Hi all,
In the discussion about Flink on k8s[1] we encounter a problem that opinions
diverge in how so-called per-job mode works. This thread is aimed at stating
a dedicated discussion about per-job semantic
15 matches
Mail list logo
