[GitHub] spark pull request: SPARK-1428: MLlib should convert non-float64 N...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/356 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: SPARK-1428: MLlib should convert non-float64 N...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/356#issuecomment-40122134 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: SPARK-1428: MLlib should convert non-float64 N...
GitHub user techaddict reopened a pull request: https://github.com/apache/spark/pull/356 SPARK-1428: MLlib should convert non-float64 NumPy arrays to float64 instead of complaining You can merge this pull request into a Git repository by running: $ git pull https://github.com/techaddict/spark 1428 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/356.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #356 commit 3bdf5f60d57a423c76682e81775d19e64904a487 Author: Sandeep Date: 2014-04-08T06:44:09Z SPARK-1428: MLlib should convert non-float64 NumPy arrays to float64 instead of complaining --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: SPARK-1428: MLlib should convert non-float64 N...
Github user techaddict commented on the pull request: https://github.com/apache/spark/pull/356#issuecomment-40120674 :smile: --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: SPARK-1428: MLlib should convert non-float64 N...
Github user techaddict closed the pull request at: https://github.com/apache/spark/pull/356 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: SPARK-1428: MLlib should convert non-float64 N...
Github user mateiz commented on the pull request: https://github.com/apache/spark/pull/356#issuecomment-40119758 Thanks Sandeep. Merged into master and 1.0. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: SPARK-1428: MLlib should convert non-float64 N...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/356#issuecomment-40116579 Merged build finished. All automated tests passed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: SPARK-1428: MLlib should convert non-float64 N...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/356#issuecomment-40116580 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/14002/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: SPARK-1428: MLlib should convert non-float64 N...
Github user techaddict commented on the pull request: https://github.com/apache/spark/pull/356#issuecomment-40112564 @mateiz done working now :+1: --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: SPARK-1428: MLlib should convert non-float64 N...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/356#issuecomment-40111911 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: SPARK-1428: MLlib should convert non-float64 N...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/356#issuecomment-40111891 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: SPARK-1428: MLlib should convert non-float64 N...
Github user techaddict commented on the pull request: https://github.com/apache/spark/pull/356#issuecomment-40110285 New to spark don't know what's the standard way of adding tests --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: SPARK-1428: MLlib should convert non-float64 N...
Github user mateiz commented on the pull request: https://github.com/apache/spark/pull/356#issuecomment-40107316 This is why it would be good to add a test actually. For instance you can add a test with `_deserialize_double_vector(_serialize_double_vector(array([1,2,3]))` and check that it returns `array([1.0,2.0,3.0])`. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: SPARK-1428: MLlib should convert non-float64 N...
Github user mateiz commented on the pull request: https://github.com/apache/spark/pull/356#issuecomment-40107176 This seems to be failing tests unfortunately -- click through to the Jenkins log. This is what it says: ``` = Running PySpark tests = ** File "pyspark/mllib/_common.py", line 76, in __main__._deserialize_double_vector Failed example: array_equal(x, _deserialize_double_vector(_serialize_double_vector(x))) Exception raised: Traceback (most recent call last): File "/usr/local/lib/python2.7/doctest.py", line 1289, in __run compileflags, 1) in test.globs File "", line 1, in array_equal(x, _deserialize_double_vector(_serialize_double_vector(x))) File "pyspark/mllib/_common.py", line 55, in _serialize_double_vector if numpy.issubdtype(v.dtype, numpy.complex): NameError: global name 'numpy' is not defined ** 1 of 2 in __main__._deserialize_double_vector ***Test Failed*** 1 failures. Had test failures; see logs. ``` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: SPARK-1428: MLlib should convert non-float64 N...
Github user techaddict commented on the pull request: https://github.com/apache/spark/pull/356#issuecomment-40046556 @mateiz is there any other problem ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: SPARK-1428: MLlib should convert non-float64 N...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/356#issuecomment-39933138 Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13932/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: SPARK-1428: MLlib should convert non-float64 N...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/356#issuecomment-39933137 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: SPARK-1428: MLlib should convert non-float64 N...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/356#issuecomment-39931229 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: SPARK-1428: MLlib should convert non-float64 N...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/356#issuecomment-39931222 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: SPARK-1428: MLlib should convert non-float64 N...
Github user techaddict commented on the pull request: https://github.com/apache/spark/pull/356#issuecomment-39930653 You mean `v = v.astype(float64, copy=True)` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: SPARK-1428: MLlib should convert non-float64 N...
Github user mateiz commented on the pull request: https://github.com/apache/spark/pull/356#issuecomment-39930461 Sorry meant true, not false. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: SPARK-1428: MLlib should convert non-float64 N...
Github user mateiz commented on the pull request: https://github.com/apache/spark/pull/356#issuecomment-39930441 Actually one more thing, can you set copy=false in astype? I'm not sure that it preserves the right byte order and such and I know that this works with copy=false. Also we don't want to mess with the user's input data. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: SPARK-1428: MLlib should convert non-float64 N...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/356#issuecomment-39929014 Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13927/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: SPARK-1428: MLlib should convert non-float64 N...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/356#issuecomment-39929012 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: SPARK-1428: MLlib should convert non-float64 N...
Github user techaddict commented on the pull request: https://github.com/apache/spark/pull/356#issuecomment-39927556 @mateiz updated, should i add tests too (IMHO i don't think there is need, because its a trivial patch :smile: ) ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: SPARK-1428: MLlib should convert non-float64 N...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/356#issuecomment-39927502 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: SPARK-1428: MLlib should convert non-float64 N...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/356#issuecomment-39927496 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: SPARK-1428: MLlib should convert non-float64 N...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/356#issuecomment-39888436 Merged build finished. All automated tests passed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: SPARK-1428: MLlib should convert non-float64 N...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/356#issuecomment-39888438 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13897/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: SPARK-1428: MLlib should convert non-float64 N...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/356#issuecomment-39882833 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: SPARK-1428: MLlib should convert non-float64 N...
Github user mateiz commented on the pull request: https://github.com/apache/spark/pull/356#issuecomment-39882724 You should also check that the vector does not contain complex numbers, since that is the one NumPy data type we can't convert to floats. You can do ``` if numpy.issubdtype(v.dtype, numpy.complex): raise TypeError(...) ``` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: SPARK-1428: MLlib should convert non-float64 N...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/356#issuecomment-39882816 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: SPARK-1428: MLlib should convert non-float64 N...
Github user mateiz commented on the pull request: https://github.com/apache/spark/pull/356#issuecomment-39882618 Jenkins, this is ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: SPARK-1428: MLlib should convert non-float64 N...
Github user techaddict commented on the pull request: https://github.com/apache/spark/pull/356#issuecomment-39857157 @mengxr ok i'll try adding some test's --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: SPARK-1428: MLlib should convert non-float64 N...
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/356#issuecomment-39821991 @techaddict Is it easy to add a test to verify that it works? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: SPARK-1428: MLlib should convert non-float64 N...
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/356#issuecomment-39816913 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] spark pull request: SPARK-1428: MLlib should convert non-float64 N...
GitHub user techaddict opened a pull request: https://github.com/apache/spark/pull/356 SPARK-1428: MLlib should convert non-float64 NumPy arrays to float64 instead of complaining You can merge this pull request into a Git repository by running: $ git pull https://github.com/techaddict/spark 1428 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/356.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #356 commit 2b393cbb3df935f6c8cc23e7cecf060e3068b61f Author: Sandeep Date: 2014-04-08T06:44:09Z SPARK-1428: MLlib should convert non-float64 NumPy arrays to float64 instead of complaining --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
