[GitHub] spark pull request: [SPARK-14938][ML] replace some RDD.map with Da...

2016-04-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12718#issuecomment-215400799 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-14938][ML] replace some RDD.map with Da...

2016-04-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12718#issuecomment-215400798 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-14938][ML] replace some RDD.map with Da...

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12718#issuecomment-215400722 **[Test build #57247 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57247/consoleFull)** for PR 12718 at commit [`9ef9a51`](https://g

[GitHub] spark pull request: [SPARK-14938][ML] replace some RDD.map with Da...

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12718#issuecomment-215392196 **[Test build #57247 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57247/consoleFull)** for PR 12718 at commit [`9ef9a51`](https://gi

[GitHub] spark pull request: [SPARK-14938][ML] replace some RDD.map with Da...

2016-04-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12718#issuecomment-215389458 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-14938][ML] replace some RDD.map with Da...

2016-04-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12718#issuecomment-215389461 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-14938][ML] replace some RDD.map with Da...

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12718#issuecomment-215389452 **[Test build #57246 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57246/consoleFull)** for PR 12718 at commit [`8863991`](https://g

[GitHub] spark pull request: [SPARK-14938][ML] replace some RDD.map with Da...

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12718#issuecomment-215388583 **[Test build #57246 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57246/consoleFull)** for PR 12718 at commit [`8863991`](https://gi

[GitHub] spark pull request: [SPARK-14938][ML] replace some RDD.map with Da...

2016-04-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12718#issuecomment-215385743 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-14938][ML] replace some RDD.map with Da...

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12718#issuecomment-215385737 **[Test build #57245 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57245/consoleFull)** for PR 12718 at commit [`df5e917`](https://g

[GitHub] spark pull request: [SPARK-14938][ML] replace some RDD.map with Da...

2016-04-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12718#issuecomment-215385742 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-14938][ML] replace some RDD.map with Da...

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12718#issuecomment-215385539 **[Test build #57245 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57245/consoleFull)** for PR 12718 at commit [`df5e917`](https://gi

[GitHub] spark pull request: [SPARK-14938][ML] replace some RDD.map with Da...

2016-04-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12718#issuecomment-215367423 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-14938][ML] replace some RDD.map with Da...

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12718#issuecomment-215367301 **[Test build #57238 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57238/consoleFull)** for PR 12718 at commit [`9584026`](https://g

[GitHub] spark pull request: [SPARK-14938][ML] replace some RDD.map with Da...

2016-04-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12718#issuecomment-215367425 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-14938][ML] replace some RDD.map with Da...

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12718#issuecomment-215357553 **[Test build #57238 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57238/consoleFull)** for PR 12718 at commit [`9584026`](https://gi

[GitHub] spark pull request: [SPARK-14938][ML] replace some RDD.map with Da...

2016-04-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12718#issuecomment-215355549 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-14938][ML] replace some RDD.map with Da...

2016-04-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12718#issuecomment-215355548 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-14938][ML] replace some RDD.map with Da...

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12718#issuecomment-215355402 **[Test build #57235 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57235/consoleFull)** for PR 12718 at commit [`530f037`](https://g

[GitHub] spark pull request: [SPARK-14938][ML] replace some RDD.map with Da...

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12718#issuecomment-215344251 **[Test build #57235 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57235/consoleFull)** for PR 12718 at commit [`530f037`](https://gi

[GitHub] spark pull request: [SPARK-14938][ML] replace some RDD.map with Da...

2016-04-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12718#issuecomment-215339652 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-14938][ML] replace some RDD.map with Da...

2016-04-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12718#issuecomment-215339654 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-14938][ML] replace some RDD.map with Da...

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12718#issuecomment-215339565 **[Test build #57231 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57231/consoleFull)** for PR 12718 at commit [`2378829`](https://g

[GitHub] spark pull request: [SPARK-14938][ML] replace some RDD.map with Da...

2016-04-28 Thread viirya
Github user viirya commented on the pull request: https://github.com/apache/spark/pull/12718#issuecomment-215332276 e.g., in `BisectingKMeans`, this patch changes val data = dataset.select(col($(featuresCol))).rdd.map { case Row(point: Vector) => point } to

[GitHub] spark pull request: [SPARK-14938][ML] replace some RDD.map with Da...

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12718#issuecomment-215332412 **[Test build #57231 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57231/consoleFull)** for PR 12718 at commit [`2378829`](https://gi

[GitHub] spark pull request: [SPARK-14938][ML] replace some RDD.map with Da...

2016-04-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12718#issuecomment-215329786 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-14938][ML] replace some RDD.map with Da...

2016-04-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12718#issuecomment-215329787 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-14938][ML] replace some RDD.map with Da...

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12718#issuecomment-215329754 **[Test build #57225 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57225/consoleFull)** for PR 12718 at commit [`b2101b2`](https://g

[GitHub] spark pull request: [SPARK-14938][ML] replace some RDD.map with Da...

2016-04-27 Thread MLnick
Github user MLnick commented on the pull request: https://github.com/apache/spark/pull/12718#issuecomment-215327760 @viirya can you provide an example of how this works for use here in this PR? --- If your project is set up for it, you can reply to this email and have your reply appe

[GitHub] spark pull request: [SPARK-14938][ML] replace some RDD.map with Da...

2016-04-27 Thread viirya
Github user viirya commented on the pull request: https://github.com/apache/spark/pull/12718#issuecomment-215327227 encoder now supports UDTs. You just need to declare one before you want to use it. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request: [SPARK-14938][ML] replace some RDD.map with Da...

2016-04-27 Thread MLnick
Github user MLnick commented on the pull request: https://github.com/apache/spark/pull/12718#issuecomment-215326887 @viirya yeah, sorry I meant we should create an encoder that can be used in `ml`... whether an implicit or explicit. --- If your project is set up for it, you can reply

[GitHub] spark pull request: [SPARK-14938][ML] replace some RDD.map with Da...

2016-04-27 Thread zhengruifeng
Github user zhengruifeng commented on a diff in the pull request: https://github.com/apache/spark/pull/12718#discussion_r61379580 --- Diff: mllib/src/main/scala/org/apache/spark/ml/Predictor.scala --- @@ -121,9 +121,9 @@ abstract class Predictor[ * and put it in an RDD with

[GitHub] spark pull request: [SPARK-14938][ML] replace some RDD.map with Da...

2016-04-27 Thread viirya
Github user viirya commented on the pull request: https://github.com/apache/spark/pull/12718#issuecomment-215326639 We have no implicit encoder for vector udt. But we can explicitly create it. --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request: [SPARK-14938][ML] replace some RDD.map with Da...

2016-04-27 Thread MLnick
Github user MLnick commented on a diff in the pull request: https://github.com/apache/spark/pull/12718#discussion_r61379351 --- Diff: mllib/src/main/scala/org/apache/spark/ml/Predictor.scala --- @@ -121,9 +121,9 @@ abstract class Predictor[ * and put it in an RDD with strong

[GitHub] spark pull request: [SPARK-14938][ML] replace some RDD.map with Da...

2016-04-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12718#issuecomment-215325772 **[Test build #57225 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57225/consoleFull)** for PR 12718 at commit [`b2101b2`](https://gi

[GitHub] spark pull request: [SPARK-14938][ML] replace some RDD.map with Da...

2016-04-27 Thread MLnick
Github user MLnick commented on the pull request: https://github.com/apache/spark/pull/12718#issuecomment-215320479 We should definitely have an encoder for vector udts... cc @dbtsai @viirya --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request: [SPARK-14938][ML] replace some RDD.map with Da...

2016-04-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12718#issuecomment-215319643 **[Test build #57222 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57222/consoleFull)** for PR 12718 at commit [`e57332a`](https://g

[GitHub] spark pull request: [SPARK-14938][ML] replace some RDD.map with Da...

2016-04-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12718#issuecomment-215319671 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-14938][ML] replace some RDD.map with Da...

2016-04-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12718#issuecomment-215319667 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-14938][ML] replace some RDD.map with Da...

2016-04-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12718#issuecomment-215316563 **[Test build #57222 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57222/consoleFull)** for PR 12718 at commit [`e57332a`](https://gi

[GitHub] spark pull request: [SPARK-14938][ML] replace some RDD.map with Da...

2016-04-27 Thread zhengruifeng
Github user zhengruifeng commented on the pull request: https://github.com/apache/spark/pull/12718#issuecomment-215074883 Vector, LabeledPoint, Instance and AFTPoint can not be used in `DataSet.as` now --- If your project is set up for it, you can reply to this email and have your re

[GitHub] spark pull request: [SPARK-14938][ML] replace some RDD.map with Da...

2016-04-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12718#issuecomment-215042014 **[Test build #57114 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57114/consoleFull)** for PR 12718 at commit [`ed90f24`](https://g

[GitHub] spark pull request: [SPARK-14938][ML] replace some RDD.map with Da...

2016-04-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12718#issuecomment-215042113 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-14938][ML] replace some RDD.map with Da...

2016-04-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12718#issuecomment-215042112 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-14938][ML] replace some RDD.map with Da...

2016-04-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12718#issuecomment-215032138 **[Test build #57114 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57114/consoleFull)** for PR 12718 at commit [`ed90f24`](https://gi

[GitHub] spark pull request: [SPARK-14938][ML] replace some RDD.map with Da...

2016-04-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12718#issuecomment-215013140 **[Test build #57105 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57105/consoleFull)** for PR 12718 at commit [`a69e1fd`](https://g

[GitHub] spark pull request: [SPARK-14938][ML] replace some RDD.map with Da...

2016-04-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12718#issuecomment-215013201 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-14938][ML] replace some RDD.map with Da...

2016-04-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12718#issuecomment-215013198 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-14938][ML] replace some RDD.map with Da...

2016-04-27 Thread zhengruifeng
Github user zhengruifeng commented on the pull request: https://github.com/apache/spark/pull/12718#issuecomment-215010119 Now, Encoder for Vector is missing... --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your proje