[GitHub] incubator-spark pull request: [Proposal] Adding sparse data suppor...

[srowen]

Github user srowen commented on the pull request:

    https://github.com/apache/incubator-spark/pull/575#issuecomment-34715135
  
    Wow nice writeup. (Is Breeze benchmarked too somewhere? don't see it 
there). Totally agree. That's why I would use JBlas at least for the complex 
operations. Although to keep it simple, using it for all dense operations isn't 
bad. And, the code already does that. (matei said it won't use native code for 
simple operations anyway?)
    
    I'm interested here in the sparse operations question, which can't be 
JBlas. Should the other library used just be used for sparse operations, or 
also for most dense operations except the ones where JBlas is better? Either 
makes sense to me. Previously I've gone with Commons Math, with a hint of JBlas 
optionally for expensive operations. 
    
    So here I suppose all I am suggesting is: need to be careful about mixing 
use of JBlas and another dense library -- should probably be all JBlas, or 90% 
the other library that's also being used for sparse. And then also voting for 
CM for sparse for at least the short term for other reasons.