[GitHub] [spark] HyukjinKwon commented on pull request #38613: [SPARK-41005][CONNECT][PYTHON][FOLLOW-UP] Fetch/send partitions in parallel for Arrow based collect

2022-11-11 Thread GitBox
HyukjinKwon commented on PR #38613: URL: https://github.com/apache/spark/pull/38613#issuecomment-1311621739 Actually let's just go with https://github.com/apache/spark/pull/38614 approach which is simpler. This approach can't easily dedup the codes anyway because of ordering anyway. --

[GitHub] [spark] HyukjinKwon commented on pull request #38613: [SPARK-41005][CONNECT][PYTHON][FOLLOW-UP] Fetch/send partitions in parallel for Arrow based collect

2022-11-10 Thread GitBox
HyukjinKwon commented on PR #38613: URL: https://github.com/apache/spark/pull/38613#issuecomment-1311232973 It collects all results first because of synced `runJob` that waits all results to arrive. -- This is an automated message from the Apache Git Service. To respond to the message,