Github user dongjoon-hyun commented on a diff in the pull request:
https://github.com/apache/spark/pull/13109#discussion_r67811795
--- Diff: R/pkg/R/stats.R ---
@@ -134,9 +129,7 @@ setMethod("freqItems", signature(x = "SparkDataFrame",
cols = "character"),
collect(dataFrame(sct))
})
-#' approxQuantile
-#'
-#' Calculates the approximate quantiles of a numerical column of a
SparkDataFrame.
+#' approxQuantile - Calculates the approximate quantiles of a numerical
column of a SparkDataFrame.
--- End diff --
Unfortunately, this line is ignored. We need `@description` here, too.
```
#' @description
#' approxQuantile - Calculates the approximate quantiles of a numerical
column of a SparkDataFrame.
```
After adding that, the description depth will look differently. I mean only
`approxQuantile` has a detail description like the following.
```
crosstab - Computes a pair-wise frequency table of the given columns. Also
known as a contingency table. The number of distinct values for each column
should be less than 1e4. At most 1e6 non-zero pair frequencies will be returned.
freqItems - Finding frequent items for columns, possibly with false
positives. Using the frequent element count algorithm described in
http://dx.doi.org/10.1145/762471.762473, proposed by Karp, Schenker, and
Papadimitriou.
approxQuantile - Calculates the approximate quantiles of a numerical column
of a SparkDataFrame.
The result of this algorithm has the following deterministic bound: If the
SparkDataFrame has N elements and if we request the quantile at probability 'p'
up to error 'err', then the algorithm will return a sample 'x' from the
SparkDataFrame so that the *exact* rank of 'x' is close to (p * N). More
precisely, floor((p - err) * N) <= rank(x) <= ceil((p + err) * N).
This method implements a variation of the Greenwald-Khanna algorithm (with
some speed optimizations). The algorithm was first present in
[[http://dx.doi.org/10.1145/375663.375670 Space-efficient Online Computation of
Quantile Summaries]] by Greenwald and Khanna.
sampleBy - Returns a stratified sample without replacement based on the
fraction given on each stratum.
```
I'm not sure about balancing them by removing `The result of this
algorithm~~~Khanna.`. If you think that is okay, we can keep that.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
