milenkovicm commented on PR #1309: URL: https://github.com/apache/datafusion-ballista/pull/1309#issuecomment-3414552929
Sorry for late reply @danielhumanmod I think your breakdown makes sense, I just wonder what would be the easiest way to implement this. Scheduler emits job completion events, so we could react on job completion, but this approach would be a bit messy with observers, as evens will be emitted for all jobs. An alternative would be to add ability to register job completion callback which triggers change of job B once the job A finishes. Trigger can change UnresolvedShuffle to actual Exec (we could use same logic like we use for simple EXPLAIN). I believe job finish callback would be useful for other cases, I'm not sure at this point how complicated would it be to implement, may have a look this weekend. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
