HyukjinKwon commented on code in PR #38579:
URL: https://github.com/apache/spark/pull/38579#discussion_r1018705370
##########
python/pyspark/sql/dataframe.py:
##########
@@ -4217,15 +4217,18 @@ def cov(self, col1: str, col2: str) -> float:
def crosstab(self, col1: str, col2: str) -> "DataFrame":
"""
Computes a pair-wise frequency table of the given columns. Also known
as a contingency
- table. The number of distinct values for each column should be less
than 1e4. At most 1e6
- non-zero pair frequencies will be returned.
Review Comment:
I think we can just remove this and don't add `.. versionchanged:: 3.4.0`.
It's sort of improvement, and trivial to mention.
##########
python/pyspark/sql/dataframe.py:
##########
@@ -4217,15 +4217,18 @@ def cov(self, col1: str, col2: str) -> float:
def crosstab(self, col1: str, col2: str) -> "DataFrame":
"""
Computes a pair-wise frequency table of the given columns. Also known
as a contingency
- table. The number of distinct values for each column should be less
than 1e4. At most 1e6
- non-zero pair frequencies will be returned.
+ table.
The first column of each row will be the distinct values of `col1` and
the column names
will be the distinct values of `col2`. The name of the first column
will be `$col1_$col2`.
Pairs that have no occurrences will have zero as their counts.
:func:`DataFrame.crosstab` and :func:`DataFrameStatFunctions.crosstab`
are aliases.
.. versionadded:: 1.4.0
+ .. versionchanged:: 3.4.0
+ The underlying implementation was optimized, then there is no
limitaions on number of
+ distinct values or pairs.
+
Review Comment:
```suggestion
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
