Thank you for writing this blog post, Daisy and Kevin! It helps me to understand what sort-based shuffle is and how to use it. Looking forward to your future improvements!
On Wed, Nov 3, 2021 at 6:32 PM Yuxin Tan <tanyuxinw...@gmail.com> wrote: > Thanks Daisy and Kevin! The IO scheduling idea of the sequential reading > and the benchmark result look really great! Looking forward to the next > work. > > Best, > > Yuxin > > weijie guo <guoweijieres...@gmail.com> 于2021年11月3日周三 下午5:24写道： > >> It's really an amazing job to fill in the defects of flink in batch >> shuffle. I really appreciate the work done in io scheduling, the sequential >> reading of the shuffle reader can greatly improve the disk IO performance >> and stability. Sort-based shuffle realizes this feature in a concise and >> efficient way. By the way, the default shuffle implementation in the batch >> mode of flink is still hash-based, maybe we can consider using the new >> shuffle implementation by default later. Last but not least, thank Yingjie >> Cao (Kevin) and Daisy Tsang for publishing this blog. >> >> Lijie Wang <wangdachui9...@gmail.com> 于2021年11月3日周三 下午4:17写道： >> >>> Thanks Daisy and Kevin for bringing this blog, it is very helpful for >>> understanding the principle of sort shuffle. >>> >>> >>> Best, >>> >>> Lijie >>> >>> Guowei Ma <guowei....@gmail.com> 于2021年11月3日周三 下午2:57写道： >>> >>>> >>>> Thank Daisy& Kevin much for your introduction to the improvement of TM >>>> blocking shuffle, credit base+io scheduling is indeed a very interesting >>>> thing. At the same time, I look forward to this as a default setting for tm >>>> blocking shuffle. >>>> >>>> Best, >>>> Guowei >>>> >>>> >>>> On Wed, Nov 3, 2021 at 2:46 PM Gen Luo <luogen...@gmail.com> wrote: >>>> >>>>> Thanks Daisy and Kevin! The benchmark results look really exciting! >>>>> >>>>> On Tue, Nov 2, 2021 at 4:38 PM David Morávek <d...@apache.org> wrote: >>>>> >>>>>> Thanks Daisy and Kevin for a great write up! ;) Especially the 2nd >>>>>> part was really interesting, I really like the idea of the single spill >>>>>> file with a custom scheduling of read requests. >>>>>> >>>>>> Best, >>>>>> D. >>>>>> >>>>>> On Mon, Nov 1, 2021 at 10:01 AM Daisy Tsang <da...@ververica.com> >>>>>> wrote: >>>>>> >>>>>>> Hey everyone, we have a new two-part post published on the Apache >>>>>>> Flink blog about the sort-based blocking shuffle implementation in >>>>>>> Flink. >>>>>>> It covers benchmark results, design and implementation details, and >>>>>>> more! >>>>>>> We hope you like it and welcome any sort of feedback on it. :) >>>>>>> >>>>>>> >>>>>>> https://flink.apache.org/2021/10/26/sort-shuffle-part1.html >>>>>>> https://flink.apache.org/2021/10/26/sort-shuffle-part2.html >>>>>>> >>>>>>
