[jira] [Commented] (ARROW-15069) [R] open_dataset very slow on heavily partitioned parquet dataset

2021-12-14 Thread Andy Teucher (Jira)
[ https://issues.apache.org/jira/browse/ARROW-15069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17459442#comment-17459442 ] Andy Teucher commented on ARROW-15069: -- Thanks so much! Only ~300 rows per file doe

[jira] [Commented] (ARROW-15069) [R] open_dataset very slow on heavily partitioned parquet dataset

2021-12-13 Thread Weston Pace (Jira)
[ https://issues.apache.org/jira/browse/ARROW-15069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17458872#comment-17458872 ] Weston Pace commented on ARROW-15069: - If I only group by year then it takes ~20 sec

[jira] [Commented] (ARROW-15069) [R] open_dataset very slow on heavily partitioned parquet dataset

2021-12-13 Thread Weston Pace (Jira)
[ https://issues.apache.org/jira/browse/ARROW-15069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17458871#comment-17458871 ] Weston Pace commented on ARROW-15069: - Thanks for the great reproducer. This is an

[jira] [Commented] (ARROW-15069) [R] open_dataset very slow on heavily partitioned parquet dataset

2021-12-13 Thread Andy Teucher (Jira)
[ https://issues.apache.org/jira/browse/ARROW-15069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17458712#comment-17458712 ] Andy Teucher commented on ARROW-15069: -- Thanks very much [~westonpace] - that defin

[jira] [Commented] (ARROW-15069) [R] open_dataset very slow on heavily partitioned parquet dataset

2021-12-13 Thread Weston Pace (Jira)
[ https://issues.apache.org/jira/browse/ARROW-15069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17458583#comment-17458583 ] Weston Pace commented on ARROW-15069: - I'm going to try and take a look at this soon

[jira] [Commented] (ARROW-15069) [R] open_dataset very slow on heavily partitioned parquet dataset

2021-12-10 Thread Andy Teucher (Jira)
[ https://issues.apache.org/jira/browse/ARROW-15069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17457394#comment-17457394 ] Andy Teucher commented on ARROW-15069: -- Fantastic, thank you! I hesitated to call i

[jira] [Commented] (ARROW-15069) [R] open_dataset very slow on heavily partitioned parquet dataset

2021-12-10 Thread Jonathan Keane (Jira)
[ https://issues.apache.org/jira/browse/ARROW-15069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17457367#comment-17457367 ] Jonathan Keane commented on ARROW-15069: Thanks for the detailed report! I kno