[jira] [Commented] (ARROW-9458) [Python] Dataset singlethreaded only

[ https://issues.apache.org/jira/browse/ARROW-9458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17157390#comment-17157390 ] Joris Van den Bossche commented on ARROW-9458: -- How do you release the GIL w

[jira] [Commented] (ARROW-9458) [Python] Dataset singlethreaded only

[ https://issues.apache.org/jira/browse/ARROW-9458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17157387#comment-17157387 ] Maarten Breddels commented on ARROW-9458: - let me know if you want to do the hono

[jira] [Commented] (ARROW-9458) [Python] Dataset singlethreaded only

[ https://issues.apache.org/jira/browse/ARROW-9458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17157374#comment-17157374 ] Maarten Breddels commented on ARROW-9458: - Indeed, seeing a massive speedup. Too

[jira] [Commented] (ARROW-9458) [Python] Dataset singlethreaded only

[ https://issues.apache.org/jira/browse/ARROW-9458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17157372#comment-17157372 ] Maarten Breddels commented on ARROW-9458: - Yes, in my case, the row groups are 1_

[jira] [Commented] (ARROW-9458) [Python] Dataset singlethreaded only

[ https://issues.apache.org/jira/browse/ARROW-9458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17157365#comment-17157365 ] Joris Van den Bossche commented on ARROW-9458: -- It might be we are not relea

[jira] [Commented] (ARROW-9458) [Python] Dataset singlethreaded only

[ https://issues.apache.org/jira/browse/ARROW-9458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17157349#comment-17157349 ] Joris Van den Bossche commented on ARROW-9458: -- > Did you set ? batch_size=1

[jira] [Commented] (ARROW-9458) [Python] Dataset singlethreaded only

[ https://issues.apache.org/jira/browse/ARROW-9458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17157340#comment-17157340 ] Maarten Breddels commented on ARROW-9458: - Did you set ? batch_size=1_000_000 >

[jira] [Commented] (ARROW-9458) [Python] Dataset singlethreaded only

[ https://issues.apache.org/jira/browse/ARROW-9458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17157337#comment-17157337 ] Joris Van den Bossche commented on ARROW-9458: -- [~maartenbreddels] how big a

[jira] [Commented] (ARROW-9458) [Python] Dataset singlethreaded only

[ https://issues.apache.org/jira/browse/ARROW-9458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17157338#comment-17157338 ] Maarten Breddels commented on ARROW-9458: -   Running this (now with all columns)

[jira] [Commented] (ARROW-9458) [Python] Dataset singlethreaded only

[ https://issues.apache.org/jira/browse/ARROW-9458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17157308#comment-17157308 ] Joris Van den Bossche commented on ARROW-9458: -- Ah, and forgot to note: we a

[jira] [Commented] (ARROW-9458) [Python] Dataset singlethreaded only

[ https://issues.apache.org/jira/browse/ARROW-9458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17157306#comment-17157306 ] Joris Van den Bossche commented on ARROW-9458: -- That it doesn't do this in p