[GitHub] incubator-spark pull request: [Proposal] Adding sparse data suppor...

[mengxr]

Github user mengxr commented on the pull request:

    https://github.com/apache/incubator-spark/pull/575#issuecomment-34707127
  
    @sscdotopen @debasish83 , I'm okay with copying VectorWritable and remove 
mahout-core from dependencies.
    
    @srowen Just as you mentioned, the sparse vector in commons-math is 
deprecated and will be removed in 4.0, and it is annoying to implement another 
sparse linear algebra package and take care of it. I've been struggling between 
mahout-math and breeze-math for a while. I don't like the design of mahout-math 
either. I would like to go with breeze-math if it becomes more stable, which 
utilizes netlib-java for calling native BLAS/LAPACK/ARPACK routines, but now it 
has performance issues (see the benchmark I posted to the JIRA) and it has been 
merged into breeze. I don't think there is a clear choice at this time. This is 
also why we want to separate what we use in the implementation from what 
interfaces we provide to users.