Re: Spark 3.1.2 full thread dumps

2022-02-11 Thread Maksim Grinman
get the dreaded thread dump > if your tasks are taking few seconds > > > > *From: *Maksim Grinman > *Date: *Thursday, February 10, 2022 at 7:21 PM > *To: *"Lalwani, Jayesh" > *Cc: *Mich Talebzadeh , Holden Karau < > hol...@pigscanfly.ca>, Sean Owen , "

Re: Spark 3.1.2 full thread dumps

2022-02-04 Thread Maksim Grinman
location.minExecutors=2 \ >>>>>--conf spark.dynamicAllocation.maxExecutors=10 \ >>>>>--conf spark.dynamicAllocation.initialExecutors=4 \ >>>>> >>>>> It is not very clear to me how Spark distributes tasks

Re: Spark 3.1.2 full thread dumps

2022-02-03 Thread Maksim Grinman
bility for any > loss, damage or destruction of data or any other property which may arise > from relying on this email's technical content is explicitly disclaimed. > The author will in no case be liable for any monetary damages arising from > such loss, damage or destruction. > &g

Spark 3.1.2 full thread dumps

2022-02-03 Thread Maksim Grinman
event the thread dumps? -- Maksim Grinman VP Engineering Resolute AI