saadtajwar commented on code in PR #23824:
URL: https://github.com/apache/datafusion/pull/23824#discussion_r3634684230
##########
datafusion/common/src/config.rs:
##########
@@ -1390,6 +1390,16 @@ config_namespace! {
/// cause regressions in both memory usage and runtime.
pub enable_window_topn: bool, default = false
+ /// When set to true, the optimizer will rewrite a "top-1 per group"
+ /// pattern of the form `Filter(row_number() = 1)` over a `PARTITION
BY` window into an
+ /// `Aggregate(first_value(... ORDER BY ...) GROUP BY partition)`.
+ /// This avoids buffering / fully sorting the input, which is
especially beneficial on wide payloads.
+ /// Disabled by default because on nested/wide value types the
aggregate only becomes memory
+ /// efficient once the columnar nested-type `GroupsAccumulator` fast
+ /// path is available; enabling it without that support can route wide
+ /// payloads onto the slow per-group accumulator path.
+ pub enable_row_number_to_aggregate: bool, default = false
Review Comment:
I assume we wanted a config for this - and can change the default to true
and remove the "disabled by default..." comment when #23601 is closed out - but
if not happy to remove!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
