Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/5608#issuecomment-95904145
Sampling 1 might work for some cases; 100 for others; some may take 1000.
There's no way to know.
This change as it stands is needlessly complex because it duplicates the
loop among other things. You just want to take n samples of the array, and use
the largest as your base, and smallest as your multiplier. That would be OK,
and make n some small number like 2 or 3. At least it would be less hard-coded,
and would make for a better change, along with some comments about why you are
doing this.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
