[jira] [Commented] (SPARK-12372) Unary operator "-" fails for MLlib vectors
[ https://issues.apache.org/jira/browse/SPARK-12372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15060699#comment-15060699 ] Joseph K. Bradley commented on SPARK-12372: --- There simply isn't a unary operation. There are ongoing discussions about turning MLlib vectors and matrices into a full-fledged local linear algebra library, but currently, you could convert to numpy/scipy and use those library for pyspark. > Unary operator "-" fails for MLlib vectors > -- > > Key: SPARK-12372 > URL: https://issues.apache.org/jira/browse/SPARK-12372 > Project: Spark > Issue Type: Bug > Components: MLlib, PySpark >Affects Versions: 1.5.2 >Reporter: Christos Iraklis Tsatsoulis > > Consider the following snippet in pyspark 1.5.2: > {code:none} > >>> from pyspark.mllib.linalg import Vectors > >>> x = Vectors.dense([0.0, 1.0, 0.0, 7.0, 0.0]) > >>> x > DenseVector([0.0, 1.0, 0.0, 7.0, 0.0]) > >>> -x > Traceback (most recent call last): > File "", line 1, in > TypeError: func() takes exactly 2 arguments (1 given) > >>> y = Vectors.dense([2.0, 0.0, 3.0, 4.0, 5.0]) > >>> y > DenseVector([2.0, 0.0, 3.0, 4.0, 5.0]) > >>> x-y > DenseVector([-2.0, 1.0, -3.0, 3.0, -5.0]) > >>> -y+x > Traceback (most recent call last): > File "", line 1, in > TypeError: func() takes exactly 2 arguments (1 given) > >>> -1*x > DenseVector([-0.0, -1.0, -0.0, -7.0, -0.0]) > {code} > Clearly, the unary operator {{-}} (minus) for vectors fails, giving errors > for expressions like {{-x}} and {{-y+x}}, despite the fact that {{x-y}} > behaves as expected. > The last operation, {{-1*x}}, although mathematically "correct", includes > minus signs for the zero entries, which again is normally not expected. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-12372) Unary operator "-" fails for MLlib vectors
[ https://issues.apache.org/jira/browse/SPARK-12372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15060858#comment-15060858 ] Christos Iraklis Tsatsoulis commented on SPARK-12372: - If this is the case, then a warning/clarification in the documentation wouldn't hurt - Spark users are not supposed to be aware of the internal "ongoing discussions" between Spark developers (BTW, any relevant link would be very welcome - I could not find any mention in MLlib & Breeze docs, neither in the recent preprint papers on linalg & MLlib). All in all, I suggest you re-open the issue with a different type (it's not a bug, as you say), and the required resolution being a notification in the relevant docs ("don't try this..., because..."). > Unary operator "-" fails for MLlib vectors > -- > > Key: SPARK-12372 > URL: https://issues.apache.org/jira/browse/SPARK-12372 > Project: Spark > Issue Type: Bug > Components: MLlib, PySpark >Affects Versions: 1.5.2 >Reporter: Christos Iraklis Tsatsoulis > > Consider the following snippet in pyspark 1.5.2: > {code:none} > >>> from pyspark.mllib.linalg import Vectors > >>> x = Vectors.dense([0.0, 1.0, 0.0, 7.0, 0.0]) > >>> x > DenseVector([0.0, 1.0, 0.0, 7.0, 0.0]) > >>> -x > Traceback (most recent call last): > File "", line 1, in > TypeError: func() takes exactly 2 arguments (1 given) > >>> y = Vectors.dense([2.0, 0.0, 3.0, 4.0, 5.0]) > >>> y > DenseVector([2.0, 0.0, 3.0, 4.0, 5.0]) > >>> x-y > DenseVector([-2.0, 1.0, -3.0, 3.0, -5.0]) > >>> -y+x > Traceback (most recent call last): > File "", line 1, in > TypeError: func() takes exactly 2 arguments (1 given) > >>> -1*x > DenseVector([-0.0, -1.0, -0.0, -7.0, -0.0]) > {code} > Clearly, the unary operator {{-}} (minus) for vectors fails, giving errors > for expressions like {{-x}} and {{-y+x}}, despite the fact that {{x-y}} > behaves as expected. > The last operation, {{-1*x}}, although mathematically "correct", includes > minus signs for the zero entries, which again is normally not expected. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-12372) Unary operator "-" fails for MLlib vectors
[ https://issues.apache.org/jira/browse/SPARK-12372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15061310#comment-15061310 ] Joseph K. Bradley commented on SPARK-12372: --- That's a good point. I'll reopen this and edit it to be for documenting the current limitations. Thanks! > Unary operator "-" fails for MLlib vectors > -- > > Key: SPARK-12372 > URL: https://issues.apache.org/jira/browse/SPARK-12372 > Project: Spark > Issue Type: Bug > Components: Documentation, MLlib >Affects Versions: 1.5.2 >Reporter: Christos Iraklis Tsatsoulis > > Consider the following snippet in pyspark 1.5.2: > {code:none} > >>> from pyspark.mllib.linalg import Vectors > >>> x = Vectors.dense([0.0, 1.0, 0.0, 7.0, 0.0]) > >>> x > DenseVector([0.0, 1.0, 0.0, 7.0, 0.0]) > >>> -x > Traceback (most recent call last): > File "", line 1, in > TypeError: func() takes exactly 2 arguments (1 given) > >>> y = Vectors.dense([2.0, 0.0, 3.0, 4.0, 5.0]) > >>> y > DenseVector([2.0, 0.0, 3.0, 4.0, 5.0]) > >>> x-y > DenseVector([-2.0, 1.0, -3.0, 3.0, -5.0]) > >>> -y+x > Traceback (most recent call last): > File "", line 1, in > TypeError: func() takes exactly 2 arguments (1 given) > >>> -1*x > DenseVector([-0.0, -1.0, -0.0, -7.0, -0.0]) > {code} > Clearly, the unary operator {{-}} (minus) for vectors fails, giving errors > for expressions like {{-x}} and {{-y+x}}, despite the fact that {{x-y}} > behaves as expected. > The last operation, {{-1*x}}, although mathematically "correct", includes > minus signs for the zero entries, which again is normally not expected. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org