subject:"Spark Tasks Progress"

RE: Spark Tasks Progress

2019-09-23 Thread Ilya Matiach

.scala#L347 You could probably do something similar, and send the partitionId or some other id to the driver. Hope this helps! Thank you, Ilya From: Sultan Alamro Sent: Friday, September 20, 2019 8:26 PM To: dev@spark.apache.org Subject: Spark Tasks Progress Hi all, I am trying to do some a

Spark Tasks Progress

2019-09-20 Thread Sultan Alamro

Hi all, I am trying to do some actions at the Driver side in Spark while an application is running. The Driver needs to know the tasks progress before making any decision. I know that tasks progress can be accessed within each executor or task from RecordReader class by calling getProgress().