@Koen,
If you meant to reply to my question on distributing matrices, could you
re-send as there was not content in your post.
Thanks,
On 08/07/2015 10:02 AM, Koen Vantomme wrote:
Verzonden vanaf mijn Sony Xperia™-smartphone
---- iceback schreef ----
Is this the sort of problem spark can accommodate?
I need to compare 10,000 matrices with each other (10^10 comparison). The
matrices are 100x100000 (10^7 int values).
I have 10 machines with 2 to 8 cores (8-32 "processors").
All machines have to
- contribute to matrices generation (a simulation, takes seconds)
- see all matrices
- compare matrices (takes very little time compared to simulation)
I expect to persist the simulations, have spark push them to processors.
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/distributing-large-matrices-tp24174.html
Sent from the Apache Spark User List mailing list archive at
Nabble.com <http://Nabble.com>.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
<mailto:user-unsubscr...@spark.apache.org>
For additional commands, e-mail: user-h...@spark.apache.org
<mailto:user-h...@spark.apache.org>
