Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/12955#issuecomment-217528845
Hm, then there are actually similar methods for key-value RDDs in
`PairRDDFunctions` and `JavaPairRDD` -- basically the non "distinct" ones.
I think we could also say a little bit more about what confidence means. I
believe it's fair to say that, if called repeatedly, this is the fraction of
results whose bounds are expected to contain the true count.
If you want to keep going ... man, I see that this value isn't even checked
to be in [0,1]. Really (0,1) is all that makes sense. Everywhere in the code
that computes an inverse CDF doesn't check this arg, and it will already throw
an exception if so.
If you're up for it, make a JIRA to describe the expanded scope. Otherwise
I'll do it at some point if you don't have bandwidth.
More points for describing in some more detail what "relativeSD" means for
the "approx distinct" count methods.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
