[ https://issues.apache.org/jira/browse/ARROW-12763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Keane resolved ARROW-12763. ------------------------------------ Resolution: Fixed Issue resolved by pull request 11405 [https://github.com/apache/arrow/pull/11405] > [R] Optimize dplyr queries that use head/tail after arrange > ----------------------------------------------------------- > > Key: ARROW-12763 > URL: https://issues.apache.org/jira/browse/ARROW-12763 > Project: Apache Arrow > Issue Type: Improvement > Components: R > Reporter: Ian Cook > Assignee: Neal Richardson > Priority: Major > Labels: pull-request-available, query-engine > Fix For: 6.0.0 > > Time Spent: 1h > Remaining Estimate: 0h > > Use the Arrow C++ function {{partition_nth_indices}} to optimize dplyr > queries like this: > {code:r} > iris %>% > Table$create() %>% > arrange(desc(Sepal.Length)) %>% > head(10) %>% > collect() > {code} > This query sorts the full table even though it doesn't need to. It could use > {{partition_nth_indices}} to find the rows containing the top 10 values of > {{Sepal.Length}} and only collect and sort those 10 rows. > Test to see if this improves performance in practice on larger data. -- This message was sent by Atlassian Jira (v8.3.4#803005)