Hi,
Well yes, I could probably make it work with a constant number of operators
(and consequently buffers) by developing specific input and output classes, and
that way I'd have a workaround for that buffers issue.
The size of my job is input-dependent mostly because my code creates one full
p
Original Message-
> From: Nico Kruber [mailto:n...@data-artisans.com]
> Sent: vendredi 18 août 2017 14:58
> To: Gwenhael Pasquiers
> Cc: Ufuk Celebi ; user@flink.apache.org
> Subject: Re: Great number of jobs and numberOfBuffers
>
> Hi Gwenhael,
> the effect you describ
Hello,
Sorry to ask you again, but no idea on this ?
-Original Message-
From: Gwenhael Pasquiers [mailto:gwenhael.pasqui...@ericsson.com]
Sent: lundi 21 août 2017 12:04
To: Nico Kruber
Cc: Ufuk Celebi ; user@flink.apache.org
Subject: RE: Great number of jobs and numberOfBuffers
Hi,
1
and 4) in order to
limit the numbers of files on HDFS.
-Original Message-
From: Nico Kruber [mailto:n...@data-artisans.com]
Sent: vendredi 18 août 2017 14:58
To: Gwenhael Pasquiers
Cc: Ufuk Celebi ; user@flink.apache.org
Subject: Re: Great number of jobs and numberOfBuffers
Hi Gwe
t; From: Ufuk Celebi [mailto:u...@apache.org]
> Sent: jeudi 17 août 2017 11:24
> To: Ufuk Celebi
> Cc: Gwenhael Pasquiers ;
> user@flink.apache.org; Nico Kruber
Subject: Re:
> Great number of jobs and numberOfBuffers
>
> PS: Also pulling in Nico (CC'd) who is working on
rg]
Sent: jeudi 17 août 2017 11:24
To: Ufuk Celebi
Cc: Gwenhael Pasquiers ;
user@flink.apache.org; Nico Kruber
Subject: Re: Great number of jobs and numberOfBuffers
PS: Also pulling in Nico (CC'd) who is working on the network stack.
On Thu, Aug 17, 2017 at 11:23 AM, Ufuk Celebi wro
PS: Also pulling in Nico (CC'd) who is working on the network stack.
On Thu, Aug 17, 2017 at 11:23 AM, Ufuk Celebi wrote:
> Hey Gwenhael,
>
> the network buffers are recycled automatically after a job terminates.
> If this does not happen, it would be quite a major bug.
>
> To help debug this:
>
Hey Gwenhael,
the network buffers are recycled automatically after a job terminates.
If this does not happen, it would be quite a major bug.
To help debug this:
- Which version of Flink are you using?
- Does the job fail immediately after submission or later during execution?
- Is the following
