kumarUjjawal commented on code in PR #27584:
URL: https://github.com/apache/datafusion/pull/27584#discussion_r4633202589


##########
datafusion/physical-plan/src/sorts/partitioned_topk.rs:
##########
@@ -347,7 +347,10 @@ impl PartitionedTopKExec {
             }
         };
         let partition_exprs = sort_exprs[..partition_prefix_len].to_vec();
-        eq_properties.reorder(sort_exprs)?;
+
+        eq_properties.clear_groupings();
+        eq_properties.clear_orderings();
+        eq_properties.add_ordering(sort_exprs);

Review Comment:
   Add a regression test for inherited ordering suffixes



##########
datafusion/physical-plan/src/sorts/partitioned_topk.rs:
##########
@@ -347,7 +347,10 @@ impl PartitionedTopKExec {
             }
         };
         let partition_exprs = sort_exprs[..partition_prefix_len].to_vec();
-        eq_properties.reorder(sort_exprs)?;
+
+        eq_properties.clear_groupings();

Review Comment:
   This patches one operator, but the real cause is shared: 
`EquivalenceProperties::reorder` keeps all existing orderings, including longer 
ones, whenever the requested ordering is already satisfied. 
`SortExec::compute_properties` (sort.rs:1722) and `PartialSortExec` 
(partial_sort.rs:284) also use it after physically re-sorting, with unstable 
sorts and TopK heaps.



##########
datafusion/physical-plan/src/sorts/partitioned_topk.rs:
##########


Review Comment:
   
https://github.com/apache/datafusion/blob/417b7b8fd506224e56ba63ab38ed0c834956d26a/datafusion/physical-plan/src/sorts/partitioned_topk.rs#L370
   
   This comment now refers to code that no longer exists: it says "`reorder` 
above already established that prefix", but the `reorder` call was replaced by 
`clear_orderings()` + `add_ordering(sort_exprs)`.



##########
datafusion/physical-plan/src/sorts/partitioned_topk.rs:
##########


Review Comment:
   
https://github.com/apache/datafusion/blob/417b7b8fd506224e56ba63ab38ed0c834956d26a/datafusion/physical-plan/src/sorts/partitioned_topk.rs#L346
   
   
`EquivalenceProperties::new(..).extend(input.equivalence_properties().clone())` 
re-normalizes every input ordering and grouping and recursively derives implied 
orderings (`add_orderings` → `implied_by`). Two lines later, 
`clear_orderings()`/`clear_groupings()` throws all of that away.**



##########
datafusion/physical-plan/src/sorts/partitioned_topk.rs:
##########


Review Comment:
   
https://github.com/apache/datafusion/blob/417b7b8fd506224e56ba63ab38ed0c834956d26a/datafusion/physical-plan/src/sorts/partitioned_topk.rs#L341
   
   In the `None` branch, the full input `EquivalenceProperties` is cloned, 
including `oeq_class`, `oeq_cache`, `geq_class`, and `geq_cache`, only to be 
cleared immediately. Now that the operator throws away all input orderings, 
this clone keeps state the code never uses.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to