[jira] [Commented] (SPARK-37185) DataFrame.take() only uses one worker

2021-11-02 Thread mathieu longtin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17437585#comment-17437585 ] mathieu longtin commented on SPARK-37185: - It seems to try to optimize for a simple query, but

[jira] [Commented] (SPARK-37185) DataFrame.take() only uses one worker

2021-11-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17437182#comment-17437182 ] Hyukjin Kwon commented on SPARK-37185: -- can you show the perf diff between both codes? >

[jira] [Commented] (SPARK-37185) DataFrame.take() only uses one worker

2021-11-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17437180#comment-17437180 ] Hyukjin Kwon commented on SPARK-37185: -- isn't it more optimized to use only one partition on one

[jira] [Commented] (SPARK-37185) DataFrame.take() only uses one worker

2021-11-01 Thread mathieu longtin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17437009#comment-17437009 ] mathieu longtin commented on SPARK-37185: - Additional note: if there's a "group by" in the