killzoner commented on code in PR #1698:
URL:
https://github.com/apache/datafusion-ballista/pull/1698#discussion_r3237217761
##########
ballista/scheduler/src/scheduler_server/query_stage_scheduler.rs:
##########
@@ -88,6 +88,10 @@ impl<T: 'static + AsLogicalPlan, U: 'static +
AsExecutionPlan>
time_recorder = Some((Instant::now(), event.clone()));
};
let event_sender = EventSender::new(tx_event.clone());
+ self.metrics_collector.set_pending_tasks_queue_size(
+ self.state.task_manager.pending_task_number().await as u64,
Review Comment:
At first I tried to put it in specific stages, but I couldn't figure exactly
what stages would make it reliable and not potentially stale (i think even
executor lost can yield back tasks in pending?)
Ended with this out of simplicity, will have a second look to be more
selective
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
