Re: [mllib] Is there any bugs to divide a Breeze sparse vectors at Spark v1.3.0-rc3?

2015-03-18 Thread David Hall 
sure.

On Wed, Mar 18, 2015 at 12:19 AM, Debasish Das debasish.da...@gmail.com
wrote:

 Hi David,

 We are stress testing breeze.optimize.proximal and nnls...if you are
 cutting a release now, we will need another release soon once we get the
 runtime optimizations in place and merged to breeze.

 Thanks.
 Deb
  On Mar 15, 2015 9:39 PM, David Hall david.lw.h...@gmail.com wrote:

 snapshot is pushed. If you verify I'll publish the new artifacts.

 On Sun, Mar 15, 2015 at 1:14 AM, Yu Ishikawa 
 yuu.ishikawa+sp...@gmail.com
 wrote:

  David Hall who is a breeze creator told me that it's a bug. So, I made a
  jira
  ticket about this issue. We need to upgrade breeze from 0.11.1 to
 0.11.2 or
  later in order to fix the bug, when the new version of breeze will be
  released.
 
  [SPARK-6341] Upgrade breeze from 0.11.1 to 0.11.2 or later - ASF JIRA
  https://issues.apache.org/jira/browse/SPARK-6341
 
  Thanks,
  Yu Ishikawa
 
 
 
  -
  -- Yu Ishikawa
  --
  View this message in context:
 
 http://apache-spark-developers-list.1001551.n3.nabble.com/mllib-Is-there-any-bugs-to-divide-a-Breeze-sparse-vectors-at-Spark-v1-3-0-rc3-tp11056p11058.html
  Sent from the Apache Spark Developers List mailing list archive at
  Nabble.com.
 
  -
  To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org
  For additional commands, e-mail: dev-h...@spark.apache.org

Re: [mllib] Is there any bugs to divide a Breeze sparse vectors at Spark v1.3.0-rc3?

2015-03-18 Thread Yu Ishikawa 
Sorry for the delay in replying. I moved from Tokyo to New York in order to
attend Spark Summit East.
I verified the snapshot and the difference.
https://github.com/scalanlp/breeze/commit/f61d2f61137807651fc860404a244640e213f6d3

Thank you for your great work!
Yu Ishikawa



-
-- Yu Ishikawa
--
View this message in context: 
http://apache-spark-developers-list.1001551.n3.nabble.com/mllib-Is-there-any-bugs-to-divide-a-Breeze-sparse-vectors-at-Spark-v1-3-0-rc3-tp11056p11107.html
Sent from the Apache Spark Developers List mailing list archive at Nabble.com.

-
To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org
For additional commands, e-mail: dev-h...@spark.apache.org

Re: [mllib] Is there any bugs to divide a Breeze sparse vectors at Spark v1.3.0-rc3?

2015-03-18 Thread Debasish Das 
Hi David,

We are stress testing breeze.optimize.proximal and nnls...if you are
cutting a release now, we will need another release soon once we get the
runtime optimizations in place and merged to breeze.

Thanks.
Deb
 On Mar 15, 2015 9:39 PM, David Hall david.lw.h...@gmail.com wrote:

 snapshot is pushed. If you verify I'll publish the new artifacts.

 On Sun, Mar 15, 2015 at 1:14 AM, Yu Ishikawa yuu.ishikawa+sp...@gmail.com
 
 wrote:

  David Hall who is a breeze creator told me that it's a bug. So, I made a
  jira
  ticket about this issue. We need to upgrade breeze from 0.11.1 to 0.11.2
 or
  later in order to fix the bug, when the new version of breeze will be
  released.
 
  [SPARK-6341] Upgrade breeze from 0.11.1 to 0.11.2 or later - ASF JIRA
  https://issues.apache.org/jira/browse/SPARK-6341
 
  Thanks,
  Yu Ishikawa
 
 
 
  -
  -- Yu Ishikawa
  --
  View this message in context:
 
 http://apache-spark-developers-list.1001551.n3.nabble.com/mllib-Is-there-any-bugs-to-divide-a-Breeze-sparse-vectors-at-Spark-v1-3-0-rc3-tp11056p11058.html
  Sent from the Apache Spark Developers List mailing list archive at
  Nabble.com.
 
  -
  To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org
  For additional commands, e-mail: dev-h...@spark.apache.org

Re: [mllib] Is there any bugs to divide a Breeze sparse vectors at Spark v1.3.0-rc3?

2015-03-17 Thread David Hall 
ping?

On Sun, Mar 15, 2015 at 9:38 PM, David Hall david.lw.h...@gmail.com wrote:

 snapshot is pushed. If you verify I'll publish the new artifacts.

 On Sun, Mar 15, 2015 at 1:14 AM, Yu Ishikawa yuu.ishikawa+sp...@gmail.com
  wrote:

 David Hall who is a breeze creator told me that it's a bug. So, I made a
 jira
 ticket about this issue. We need to upgrade breeze from 0.11.1 to 0.11.2
 or
 later in order to fix the bug, when the new version of breeze will be
 released.

 [SPARK-6341] Upgrade breeze from 0.11.1 to 0.11.2 or later - ASF JIRA
 https://issues.apache.org/jira/browse/SPARK-6341

 Thanks,
 Yu Ishikawa



 -
 -- Yu Ishikawa
 --
 View this message in context:
 http://apache-spark-developers-list.1001551.n3.nabble.com/mllib-Is-there-any-bugs-to-divide-a-Breeze-sparse-vectors-at-Spark-v1-3-0-rc3-tp11056p11058.html
 Sent from the Apache Spark Developers List mailing list archive at
 Nabble.com.

 -
 To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org
 For additional commands, e-mail: dev-h...@spark.apache.org

[mllib] Is there any bugs to divide a Breeze sparse vectors at Spark v1.3.0-rc3?

2015-03-15 Thread Yu Ishikawa 
Hi all,

Is there any bugs to divide a Breeze sparse vector at Spark v1.3.0-rc3? When
I tried to divide a sparse vector at Spark v1.3.0-rc3, I got a wrong result
if the target vector has any zero values.

Spark v1.3.0-rc3 depends on Breeze v0.11.1. And Breeze v0.11.1 seems to have
any bugs to divide a sparse vector by a scalar value. When dividing a breeze
sparse vector which has any zero values, the result seems to be a zero
vector. However, we can run the same code on Spark v1.2.x.

However, there is no problem to multiply a breeze sparse vector. I asked the
breeze community this problem on the below issue.
https://github.com/scalanlp/breeze/issues/382

For example,
```
test(dividing a breeze spark vector) {
val vec = Vectors.sparse(6, Array(0, 4), Array(0.0, 10.0)).toBreeze
val n = 60.0
val answer1 = vec :/ n
val answer2 = vec.toDenseVector :/ n
println(vec)
println(answer1)
println(answer2)
assert(answer1.toDenseVector === answer2)
}

SparseVector((0,0.0), (4,10.0))
SparseVector()
DenseVector(0.0, 0.0, 0.0, 0.0, 0.1, 0.0)

DenseVector(0.0, 0.0, 0.0, 0.0, 0.0, 0.0) did not equal DenseVector(0.0,
0.0, 0.0, 0.0, 0.1, 0.0)
org.scalatest.exceptions.TestFailedException: DenseVector(0.0, 0.0, 0.0,
0.0, 0.0, 0.0) did not equal DenseVector(0.0, 0.0, 0.0, 0.0,
0.1, 0.0)
```

Thanks,
Yu Ishikawa



-
-- Yu Ishikawa
--
View this message in context: 
http://apache-spark-developers-list.1001551.n3.nabble.com/mllib-Is-there-any-bugs-to-divide-a-Breeze-sparse-vectors-at-Spark-v1-3-0-rc3-tp11056.html
Sent from the Apache Spark Developers List mailing list archive at Nabble.com.

-
To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org
For additional commands, e-mail: dev-h...@spark.apache.org

Re: [mllib] Is there any bugs to divide a Breeze sparse vectors at Spark v1.3.0-rc3?

2015-03-15 Thread Yu Ishikawa 
David Hall who is a breeze creator told me that it's a bug. So, I made a jira
ticket about this issue. We need to upgrade breeze from 0.11.1 to 0.11.2 or
later in order to fix the bug, when the new version of breeze will be
released.

[SPARK-6341] Upgrade breeze from 0.11.1 to 0.11.2 or later - ASF JIRA
https://issues.apache.org/jira/browse/SPARK-6341

Thanks,
Yu Ishikawa



-
-- Yu Ishikawa
--
View this message in context: 
http://apache-spark-developers-list.1001551.n3.nabble.com/mllib-Is-there-any-bugs-to-divide-a-Breeze-sparse-vectors-at-Spark-v1-3-0-rc3-tp11056p11058.html
Sent from the Apache Spark Developers List mailing list archive at Nabble.com.

-
To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org
For additional commands, e-mail: dev-h...@spark.apache.org

Re: [mllib] Is there any bugs to divide a Breeze sparse vectors at Spark v1.3.0-rc3?

2015-03-15 Thread DB Tsai 
It's a bug in breeze's side. Once David fixes it and publishes it to
maven, we can upgrade to breeze 0.11.2. Please file a jira ticket for
this issue. thanks.

Sincerely,

DB Tsai
---
Blog: https://www.dbtsai.com


On Sun, Mar 15, 2015 at 12:45 AM, Yu Ishikawa
yuu.ishikawa+sp...@gmail.com wrote:
 Hi all,

 Is there any bugs to divide a Breeze sparse vector at Spark v1.3.0-rc3? When
 I tried to divide a sparse vector at Spark v1.3.0-rc3, I got a wrong result
 if the target vector has any zero values.

 Spark v1.3.0-rc3 depends on Breeze v0.11.1. And Breeze v0.11.1 seems to have
 any bugs to divide a sparse vector by a scalar value. When dividing a breeze
 sparse vector which has any zero values, the result seems to be a zero
 vector. However, we can run the same code on Spark v1.2.x.

 However, there is no problem to multiply a breeze sparse vector. I asked the
 breeze community this problem on the below issue.
 https://github.com/scalanlp/breeze/issues/382

 For example,
 ```
 test(dividing a breeze spark vector) {
 val vec = Vectors.sparse(6, Array(0, 4), Array(0.0, 10.0)).toBreeze
 val n = 60.0
 val answer1 = vec :/ n
 val answer2 = vec.toDenseVector :/ n
 println(vec)
 println(answer1)
 println(answer2)
 assert(answer1.toDenseVector === answer2)
 }

 SparseVector((0,0.0), (4,10.0))
 SparseVector()
 DenseVector(0.0, 0.0, 0.0, 0.0, 0.1, 0.0)

 DenseVector(0.0, 0.0, 0.0, 0.0, 0.0, 0.0) did not equal DenseVector(0.0,
 0.0, 0.0, 0.0, 0.1, 0.0)
 org.scalatest.exceptions.TestFailedException: DenseVector(0.0, 0.0, 0.0,
 0.0, 0.0, 0.0) did not equal DenseVector(0.0, 0.0, 0.0, 0.0,
 0.1, 0.0)
 ```

 Thanks,
 Yu Ishikawa



 -
 -- Yu Ishikawa
 --
 View this message in context: 
 http://apache-spark-developers-list.1001551.n3.nabble.com/mllib-Is-there-any-bugs-to-divide-a-Breeze-sparse-vectors-at-Spark-v1-3-0-rc3-tp11056.html
 Sent from the Apache Spark Developers List mailing list archive at Nabble.com.

 -
 To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org
 For additional commands, e-mail: dev-h...@spark.apache.org


-
To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org
For additional commands, e-mail: dev-h...@spark.apache.org

Re: [mllib] Is there any bugs to divide a Breeze sparse vectors at Spark v1.3.0-rc3?

2015-03-15 Thread David Hall 
snapshot is pushed. If you verify I'll publish the new artifacts.

On Sun, Mar 15, 2015 at 1:14 AM, Yu Ishikawa yuu.ishikawa+sp...@gmail.com
wrote:

 David Hall who is a breeze creator told me that it's a bug. So, I made a
 jira
 ticket about this issue. We need to upgrade breeze from 0.11.1 to 0.11.2 or
 later in order to fix the bug, when the new version of breeze will be
 released.

 [SPARK-6341] Upgrade breeze from 0.11.1 to 0.11.2 or later - ASF JIRA
 https://issues.apache.org/jira/browse/SPARK-6341

 Thanks,
 Yu Ishikawa



 -
 -- Yu Ishikawa
 --
 View this message in context:
 http://apache-spark-developers-list.1001551.n3.nabble.com/mllib-Is-there-any-bugs-to-divide-a-Breeze-sparse-vectors-at-Spark-v1-3-0-rc3-tp11056p11058.html
 Sent from the Apache Spark Developers List mailing list archive at
 Nabble.com.

 -
 To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org
 For additional commands, e-mail: dev-h...@spark.apache.org