[
https://issues.apache.org/jira/browse/SPARK-6522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
holdenk closed SPARK-6522.
--------------------------
Resolution: Fixed
Fix Version/s: 1.1.0
> Standardize Random Number Generation
> ------------------------------------
>
> Key: SPARK-6522
> URL: https://issues.apache.org/jira/browse/SPARK-6522
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Affects Versions: 1.3.0
> Reporter: RJ Nowling
> Priority: Minor
> Fix For: 1.1.0
>
>
> Generation of random numbers in Spark has to be handled carefully since
> references to RNGs copy the state to the workers. As such, a separate RNG
> needs to be seeded for each partition. Each time random numbers are used in
> Spark's libraries, the RNG seeding is re-implemented, leaving open the
> possibility of mistakes.
> It would be useful if RNG seeding was standardized through utility functions
> or random number generation functions that can be called in Spark pipelines.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
