RE: Distribution of sinks among the nodes

Thanks, One more thing to expect from the next version ! -Original Message- From: Aljoscha Krettek [mailto:aljos...@apache.org] Sent: lundi 8 février 2016 13:18 To: user@flink.apache.org Subject: Re: Distribution of sinks among the nodes Hi, I just merged the new feature, so once this

Re: Distribution of sinks among the nodes

ap will have it’s own >> sink and there will be no exchanges between flink instances. >> >> From: ewenstep...@gmail.com [mailto:ewenstep...@gmail.com] On Behalf Of >> Stephan Ewen >> Sent: jeudi 4 février 2016 15:13 >> To: user@flink.apache.org >> Subject: Re

Re: Distribution of sinks among the nodes

ances. > > From: ewenstep...@gmail.com [mailto:ewenstep...@gmail.com] On Behalf Of > Stephan Ewen > Sent: jeudi 4 février 2016 15:13 > To: user@flink.apache.org > Subject: Re: Distribution of sinks among the nodes > > To your other question, there are two things in F

RE: Distribution of sinks among the nodes

. From: ewenstep...@gmail.com [mailto:ewenstep...@gmail.com] On Behalf Of Stephan Ewen Sent: jeudi 4 février 2016 15:13 To: user@flink.apache.org Subject: Re: Distribution of sinks among the nodes To your other question, there are two things in Flink: (1) Chaining. Tasks are folded together into

Re: Distribution of sinks among the nodes

eep the >> standard behavior, don’t have to make a complicated generic-optimized >> algorithm, and let the user make it’s choices. *Should I create a JIRA ?* >> >> >> >> For the time being we could start the application 4 time : one time per >> node, put th

Re: Distribution of sinks among the nodes

being we could start the application 4 time : one time per > node, put that’s not pretty at all J > > > > B.R. > > > > *From:* Till Rohrmann [mailto:trohrm...@apache.org] > *Sent:* mercredi 3 février 2016 17:58 > > *To:* user@flink.apache.org > *Subject:* Re: Distri

RE: Distribution of sinks among the nodes

at 24 if there was no chaining … From: Gwenhael Pasquiers [mailto:gwenhael.pasqui...@ericsson.com] Sent: jeudi 4 février 2016 09:55 To: user@flink.apache.org Subject: RE: Distribution of sinks among the nodes Don’t we need to set the number of slots to 24 (4 sources + 16 mappers + 4 sinks

RE: Distribution of sinks among the nodes

application 4 time : one time per node, put that’s not pretty at all ☺ B.R. From: Till Rohrmann [mailto:trohrm...@apache.org] Sent: mercredi 3 février 2016 17:58 To: user@flink.apache.org Subject: Re: Distribution of sinks among the nodes Hi Gwenhäel, if you set the number of slots for each TaskManager

Re: Distribution of sinks among the nodes

> > > -Original Message- > From: Aljoscha Krettek [mailto:aljos...@apache.org] > Sent: mercredi 3 février 2016 16:26 > To: user@flink.apache.org > Subject: Re: Distribution of sinks among the nodes > > Hi Gwenhäel, > when you say 16 maps, are we talking about one m

RE: Distribution of sinks among the nodes

nd we have only 4 kafka partitions. That's why we need more mappers than sources / sinks -Original Message- From: Aljoscha Krettek [mailto:aljos...@apache.org] Sent: mercredi 3 février 2016 16:26 To: user@flink.apache.org Subject: Re: Distribution of sinks among the nodes Hi Gwen

Re: Distribution of sinks among the nodes

Hi Gwenhäel, when you say 16 maps, are we talking about one mapper with parallelism 16 or 16 unique map operators? Regards, Aljoscha > On 03 Feb 2016, at 15:48, Gwenhael Pasquiers > wrote: > > Hi, > > We try to deploy an application with the following “architecture” : > > 4 kafka sources =

Distribution of sinks among the nodes

Hi, We try to deploy an application with the following “architecture” : 4 kafka sources => 16 maps => 4 kafka sinks, on 4 nodes, with 24 slots (we disabled operator chaining). So we’d like on each node : 1x source => 4x map => 1x sink That way there are no exchanges between different instances