[ https://issues.apache.org/jira/browse/ARROW-12763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17421598#comment-17421598 ]
Neal Richardson commented on ARROW-12763: ----------------------------------------- See discussion on ARROW-13767. I think this means we should use the top_k node when that is added (ARROW-13973) > [R] Optimize dplyr queries that use head/tail after arrange > ----------------------------------------------------------- > > Key: ARROW-12763 > URL: https://issues.apache.org/jira/browse/ARROW-12763 > Project: Apache Arrow > Issue Type: Improvement > Components: R > Reporter: Ian Cook > Priority: Major > > Use the Arrow C++ function {{partition_nth_indices}} to optimize dplyr > queries like this: > {code:r} > iris %>% > Table$create() %>% > arrange(desc(Sepal.Length)) %>% > head(10) %>% > collect() > {code} > This query sorts the full table even though it doesn't need to. It could use > {{partition_nth_indices}} to find the rows containing the top 10 values of > {{Sepal.Length}} and only collect and sort those 10 rows. > Test to see if this improves performance in practice on larger data. -- This message was sent by Atlassian Jira (v8.3.4#803005)