ump45nose commented on code in PR #2698:
URL: 
https://github.com/apache/datafusion-ballista/pull/2698#discussion_r4586306714


##########
ballista/scheduler/src/state/mod.rs:
##########
@@ -926,6 +938,14 @@ mod tests {
         let (tx_event, _rx_event) = tokio::sync::mpsc::channel(100);
         let sender = EventSender::new(tx_event);
         state.task_manager.record_task_reservations(&bound);
+
+        let available_before =

Review Comment:
   Done in b656e66, after merging current `main` in 10d8c1e. The test now sums 
`global_input_partition_ids.len()` over `bound` and asserts `available_after == 
available_before + bound_partitions`, so it no longer assumes bind takes every 
pending partition. Locally it is 8 before / 16 after with the fix; with the 
reset disabled it fails (8 vs 16 for AQE, 0 vs 2 for the static graph). 
(AI-assisted reply.)



##########
ballista/scheduler/src/state/execution_graph.rs:
##########
@@ -316,6 +330,27 @@ pub trait ExecutionGraph: Debug {
     }
 }
 
+/// Reset a still-Running task_info slot so its partitions return to pending.
+pub(crate) fn reset_unlaunched_task_in_stages(
+    stages: &mut HashMap<usize, ExecutionStage>,
+    key: &TaskKey,
+) -> bool {
+    let Some(ExecutionStage::Running(stage)) = stages.get_mut(&key.stage_id) 
else {
+        return false;
+    };

Review Comment:
   Done in b656e66. `reset_unlaunched_task` now takes the `TaskDescription` 
plus `executor_id`, and `reset_unlaunched_task_in_stages` only resets when the 
running stage is still on `task.stage_attempt_num` and the slot is `Running { 
executor_id }` for that executor. The test also checks that a reset for another 
executor, or for a bumped stage attempt, is a no-op and leaves the pending 
count unchanged. (AI-assisted reply.)



##########
ballista/scheduler/src/state/execution_graph.rs:
##########
@@ -1394,6 +1429,10 @@ impl ExecutionGraph for StaticExecutionGraph {
         }
     }
 
+    fn reset_unlaunched_task(&mut self, key: &TaskKey) -> bool {

Review Comment:
   Added in b656e66: the test body moved into 
`launch_tasks_metadata_miss_resets_and_refunds(adaptive)`, and 
`launch_tasks_refunds_vcores_when_executor_metadata_missing_static_graph` runs 
it with `with_ballista_adaptive_query_planner(false)`, so the 
`StaticExecutionGraph` override is exercised too. (AI-assisted reply.)



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to