[GitHub] incubator-spark pull request: [Proposal] Adding sparse data suppor...

[giyengar]

Github user giyengar commented on the pull request:

    https://github.com/apache/incubator-spark/pull/575#issuecomment-34988403
  
    @shivaram, @srowen:
    
    I am a new user of Spark and MLLib in particular. I love the clean 
interface that MLLib currently has. It is a joy to code machine learning 
algorithms in it. 
    
    To address shivaram's (3), I'd say about 80% of the needs of Regression and 
Classification tasks can be solved using basic linear algebra operations that 
can potentially be built in-house though it seems rather wasteful to build yet 
another sparse linear algebra library. I have used Commons Math and was a bit 
disappointed to see that sparse vector support is being deprecated. 
    
    I like Sean's top 3 ideas with (2) and (1) reversed. It might make sense to 
take Commons Math sparse implementations and repurpose it as MLLib-sparse. 
    
    Best Regards,