Hello everyone, I am implementing a caching mechanism for analytic workloads running on top of Spark and I need to retrieve the Spark DAG right after it is generated and the DAG scheduler. I would appreciate it if you could give me some hints or reference me to some documents about where the DAG is generated and inputs assigned to it. I found the DAG Scheduler class <https://github.com/apache/spark/blob/55dea9be62019d64d5d76619e1551956c8bb64d0/core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala> but I am not sure if it is a good starting point.
Regards Mania
