Great :-)
On Tue, Jun 8, 2021 at 1:11 PM Yingjie Cao wrote:
> Hi Till,
>
> Thanks for the suggestion. The blog post is already on the way.
>
> Best,
> Yingjie
>
> Till Rohrmann 于2021年6月8日周二 下午5:30写道：
>
>> Thanks for the update Yingjie. Would it make sense to write a short blog
>> post about thi
Hi Till,
Thanks for the suggestion. The blog post is already on the way.
Best,
Yingjie
Till Rohrmann 于2021年6月8日周二 下午5:30写道：
> Thanks for the update Yingjie. Would it make sense to write a short blog
> post about this feature including some performance improvement numbers? I
> think this could
Thanks for the update Yingjie. Would it make sense to write a short blog
post about this feature including some performance improvement numbers? I
think this could be interesting to our users.
Cheers,
Till
On Mon, Jun 7, 2021 at 4:49 AM Jingsong Li wrote:
> Thanks Yingjie for the great effort!
Thanks Yingjie for the great effort!
This is really helpful to Flink Batch users!
Best,
Jingsong
On Mon, Jun 7, 2021 at 10:11 AM Yingjie Cao wrote:
> Hi devs & users,
>
> The FLIP-148[1] has been released with Flink 1.13 and the final
> implementation has some differences compared with the ini
Hi devs & users,
The FLIP-148[1] has been released with Flink 1.13 and the final
implementation has some differences compared with the initial proposal in
the FLIP document. To avoid potential misunderstandings, I have updated the
FLIP document[1] accordingly and I also drafted another document[2]
oolean type option for easy
>> use and the default value can be false in MVP. Then it will not bring any
>> effects for users after upgrade to new version by default,
>> and sort-merge option can be enabled to try out if users willing in
>> desired scenarios.
>>
>> Best,
>> Zhijiang
>> --
nd sort-merge option can be enabled to try out if users willing in
> desired scenarios.
>
> Best,
> Zhijiang
> ------------------
> From:Till Rohrmann
> Send Time:2020年10月16日(星期五) 15:42
> To:dev
> Subject:Re: [DISCUSS] FLIP-148: Introduce Sort-Merge Based Blocking
> Shuffle
: [DISCUSS] FLIP-148: Introduce Sort-Merge Based Blocking Shuffle to
Flink
Thanks for sharing the preliminary numbers with us Yingjie. The numbers
look quite impressive :-)
Cheers,
Till
On Thu, Oct 15, 2020 at 5:25 PM Yingjie Cao wrote:
> Hi Till,
>
> Thanks for your reply and comments.
&g
Thanks for sharing the preliminary numbers with us Yingjie. The numbers
look quite impressive :-)
Cheers,
Till
On Thu, Oct 15, 2020 at 5:25 PM Yingjie Cao wrote:
> Hi Till,
>
> Thanks for your reply and comments.
>
> You are right, the proposed sort-merge based shuffle is an extension of the
>
Hi Till,
Thanks for your reply and comments.
You are right, the proposed sort-merge based shuffle is an extension of the
existing blocking shuffle and does not change any default behavior of Flink.
As for the performance, according to our previous experience, sort-merge
based implementation can
Hi Yingjie,
thanks for proposing the sort-merge based blocking shuffle. I like the
proposal and it does not seem to change the internals of Flink. Instead it
is an extension of existing interfaces which makes it a
non-invasive addition.
Do you have any numbers comparing the performance of the sor
Hi devs,
Currently, Flink adopts a hash-style blocking shuffle implementation which
writes data sent to different reducer tasks into separate files
concurrently. Compared to sort-merge based approach writes those data
together into a single file and merges those small files into bigger ones,
hash-
12 matches
Mail list logo
