Verzonden vanaf mijn Sony Xperia™-smartphone
---- iceback schreef ---- >Is this the sort of problem spark can accommodate? > >I need to compare 10,000 matrices with each other (10^10 comparison). The >matrices are 100x100000 (10^7 int values). >I have 10 machines with 2 to 8 cores (8-32 "processors"). >All machines have to > - contribute to matrices generation (a simulation, takes seconds) > - see all matrices > - compare matrices (takes very little time compared to simulation) > >I expect to persist the simulations, have spark push them to processors. > > > > >-- >View this message in context: >http://apache-spark-user-list.1001560.n3.nabble.com/distributing-large-matrices-tp24174.html >Sent from the Apache Spark User List mailing list archive at Nabble.com. > >--------------------------------------------------------------------- >To unsubscribe, e-mail: user-unsubscr...@spark.apache.org >For additional commands, e-mail: user-h...@spark.apache.org >
