Hi, Can you provide a bit more info please?
How are you running this job and what is the streaming framework (kafka, files etc)? HTH Mich view my Linkedin profile <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/> *Disclaimer:* Use it at your own risk. Any and all responsibility for any loss, damage or destruction of data or any other property which may arise from relying on this email's technical content is explicitly disclaimed. The author will in no case be liable for any monetary damages arising from such loss, damage or destruction. On Sat, 10 Apr 2021 at 14:28, András Kolbert <kolbertand...@gmail.com> wrote: > hi, > > I have a streaming job and quite often executors die (due to memory > errors/ "unable to find location for shuffle etc) during the processing. I > started digging and found that some of the tasks are concentrated to one > executor, just as below: > [image: image.png] > > Can this be the reason? > Should I repartition the underlying data before I execute a groupby on the > top of it? > > Any advice is welcome > > Thanks > Andras >
