>>> > > > As far as I know, the reason why the broadcast variables are
> >>> > implemented
> >>> > > that way is that the senders would have to know which sub-tasks are
> >>> > > deployed to which TMs.
> >>> > >
> >>> > > As the broadcast var
uming that the same behavior will apply for
>> broadcast
>> > > joins as well.
>> > >
>> > > Is this the case?
>> > >
>> > > Regards,
>> > > Alexander
>> > >
>> > >
>> > >
Felix Neutatz created FLINK-4175:
Summary: Broadcast data sent increases with # slots per TM
Key: FLINK-4175
URL: https://issues.apache.org/jira/browse/FLINK-4175
Project: Flink
Issue Type
> > >
> > > Regards,
> > > Alexander
> > >
> > >
> > > 2016-06-08 17:13 GMT+02:00 Kunft, Andreas <andreas.ku...@tu-berlin.de
> >:
> > >
> > > > Hi Till,
> > > >
> > > > thanks for the
; > thanks for the fast answer.
> > > I'll think about a concrete way of implementing and open an JIRA.
> > >
> > > Best
> > > Andreas
> > >
> > > Von: Till Rohrmann <trohrm...@apache.org>
> > &
; > I'll think about a concrete way of implementing and open an JIRA.
> >
> > Best
> > Andreas
> >
> > Von: Till Rohrmann <trohrm...@apache.org>
> > Gesendet: Mittwoch, 8. Juni 2016 15:53
> > An: dev@flink.apache.org
> >
pen an JIRA.
>
> Best
> Andreas
>
> Von: Till Rohrmann <trohrm...@apache.org>
> Gesendet: Mittwoch, 8. Juni 2016 15:53
> An: dev@flink.apache.org
> Betreff: Re: Broadcast data sent increases with # slots per TM
>
> Hi Andreas,
>
> your obs
t data sent increases with # slots per TM
Hi Andreas,
your observation is correct. The data is sent to each slot and the
receiving TM only materializes one copy of the data. The rest of the data
is discarded.
As far as I know, the reason why the broadcast variables are implemented
th
Hi Andreas,
your observation is correct. The data is sent to each slot and the
receiving TM only materializes one copy of the data. The rest of the data
is discarded.
As far as I know, the reason why the broadcast variables are implemented
that way is that the senders would have to know which
Hi,
we experience some unexpected increase of data sent over the network for
broadcasts with increasing number of slots per Taskmanager.
We provided a benchmark [1]. It not only increases the size of data sent over
the network but also hurts performance as seen in the preliminary results
10 matches
Mail list logo