Re: [PR] [SPARK-47252][DOCS] Clarify that pivot may trigger an eager computation [spark]

2024-03-04 Thread via GitHub
HyukjinKwon closed pull request #45363: [SPARK-47252][DOCS] Clarify that pivot may trigger an eager computation URL: https://github.com/apache/spark/pull/45363 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] [SPARK-47252][DOCS] Clarify that pivot may trigger an eager computation [spark]

2024-03-04 Thread via GitHub
HyukjinKwon commented on PR #45363: URL: https://github.com/apache/spark/pull/45363#issuecomment-1977745985 Merged to master. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] [SPARK-47252][DOCS] Clarify that pivot may trigger an eager computation [spark]

2024-03-04 Thread via GitHub
nchammas commented on PR #45363: URL: https://github.com/apache/spark/pull/45363#issuecomment-1976753715 I've updated the screenshots in the PR description to reflect the latest changes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on

Re: [PR] [SPARK-47252][DOCS] Clarify that pivot may trigger an eager computation [spark]

2024-03-03 Thread via GitHub
HyukjinKwon commented on code in PR #45363: URL: https://github.com/apache/spark/pull/45363#discussion_r1510598001 ## python/pyspark/sql/group.py: ## @@ -450,6 +446,10 @@ def pivot(self, pivot_col: str, values: Optional[List["LiteralType"]] = None) -> values : list, op

Re: [PR] [SPARK-47252][DOCS] Clarify that pivot may trigger an eager computation [spark]

2024-03-03 Thread via GitHub
nchammas commented on code in PR #45363: URL: https://github.com/apache/spark/pull/45363#discussion_r151059 ## connector/connect/client/jvm/src/main/scala/org/apache/spark/sql/RelationalGroupedDataset.scala: ## @@ -259,18 +259,19 @@ class RelationalGroupedDataset private[sql

Re: [PR] [SPARK-47252][DOCS] Clarify that pivot may trigger an eager computation [spark]

2024-03-03 Thread via GitHub
HyukjinKwon commented on code in PR #45363: URL: https://github.com/apache/spark/pull/45363#discussion_r1510453526 ## connector/connect/client/jvm/src/main/scala/org/apache/spark/sql/RelationalGroupedDataset.scala: ## @@ -259,18 +259,19 @@ class RelationalGroupedDataset private[

Re: [PR] [SPARK-47252][DOCS] Clarify that pivot may trigger an eager computation [spark]

2024-03-02 Thread via GitHub
nchammas commented on code in PR #45363: URL: https://github.com/apache/spark/pull/45363#discussion_r1510094130 ## sql/core/src/main/scala/org/apache/spark/sql/RelationalGroupedDataset.scala: ## @@ -324,18 +324,18 @@ class RelationalGroupedDataset protected[sql]( /** * Pi