Sorry for my late reply! I'm also very interested in the implementation of distributed matrix multiplication. As Shivaram mentioned, the communication is the concern here. But maybe we can start with a reasonable implementation and then iterate on its performance. It would be great if eventually we can implement an algorithm close to the 2.5D algorithm (http://www.netlib.org/lapack/lawnspdf/lawn248.pdf).
I created two JIRAs for this topic: 1. Distributed block matrix: https://issues.apache.org/jira/browse/SPARK-3434 2. Distributed matrix multiplication: https://issues.apache.org/jira/browse/SPARK-3435 We can move our discussion there. Rong, I'm really happy to see the Saury project. It would be great if you can share your design and experience (maybe on the JIRA page so it is easier to track). I will read the reports on CSDN and ping you if I ran into problems. Thanks! Best, Xiangrui On Sat, Sep 6, 2014 at 1:28 AM, Yu Ishikawa <yuu.ishikawa+sp...@gmail.com> wrote: > Hi Rong, > > Great job! Thank you for let me know your work. > I will read the source code of saury later. > > Although AMPLab is working to implement them, would you like to merge it > into Spark? > > Best, > > -- Yu Ishikawa > > > > > -- > View this message in context: > http://apache-spark-developers-list.1001551.n3.nabble.com/mllib-Add-multiplying-large-scale-matrices-tp8291p8310.html > Sent from the Apache Spark Developers List mailing list archive at Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org > For additional commands, e-mail: dev-h...@spark.apache.org > --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
