[GitHub] spark pull request: [SPARK-12660] [SPARK-14967] [SQL] Implement Ex...

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12736#issuecomment-215342650 **[Test build #57227 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57227/consoleFull)** for PR 12736 at commit [`4920360`](https://g

[GitHub] spark pull request: [SPARK-14850][ML] convert primitive array from...

2016-04-28 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/12640#discussion_r61387931 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/UnsafeArrayData.java --- @@ -336,4 +336,62 @@ public UnsafeArrayData copy() {

[GitHub] spark pull request: [SPARK-14850][ML] convert primitive array from...

2016-04-28 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/12640#discussion_r61387937 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/UnsafeArrayData.java --- @@ -336,4 +336,62 @@ public UnsafeArrayData copy() {

[GitHub] spark pull request: [SPARK-14850][ML] convert primitive array from...

2016-04-28 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/12640#discussion_r61386974 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/UnsafeArrayData.java --- @@ -336,4 +336,62 @@ public UnsafeArrayData copy() {

[GitHub] spark pull request: [SPARK-14850][ML] convert primitive array from...

2016-04-28 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/12640#discussion_r61386945 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/UnsafeArrayData.java --- @@ -336,4 +336,62 @@ public UnsafeArrayData copy() {

[GitHub] spark pull request: [SPARK-14945][PYTHON] SparkSession Python API

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12746#issuecomment-215340310 **[Test build #57234 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57234/consoleFull)** for PR 12746 at commit [`e61aa8a`](https://gi

[GitHub] spark pull request: [SPARK-14945][PYTHON] SparkSession Python API

2016-04-28 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/12746#issuecomment-215340073 retest this please?? Why did you run all the unit tests?? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as w

[GitHub] spark pull request: [SPARK-14938][ML] replace some RDD.map with Da...

2016-04-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12718#issuecomment-215339652 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-14938][ML] replace some RDD.map with Da...

2016-04-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12718#issuecomment-215339654 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-14938][ML] replace some RDD.map with Da...

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12718#issuecomment-215339565 **[Test build #57231 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57231/consoleFull)** for PR 12718 at commit [`2378829`](https://g

[GitHub] spark pull request: [SPARK-14850][ML] convert primitive array from...

2016-04-28 Thread kiszk
Github user kiszk commented on the pull request: https://github.com/apache/spark/pull/12640#issuecomment-215338341 @cloud-fan , @mengxr, it would be worth to add ```final``` to a declaration of ```UnsafeArrayData ``` for encouraging method inlining by JIT compiler, as follows: ```

[GitHub] spark pull request: [SPARK-13961][ML] spark.ml ChiSqSelector and R...

2016-04-28 Thread BenFradet
Github user BenFradet commented on a diff in the pull request: https://github.com/apache/spark/pull/12467#discussion_r61384769 --- Diff: mllib/src/test/scala/org/apache/spark/ml/feature/RFormulaSuite.scala --- @@ -290,4 +291,18 @@ class RFormulaSuite extends SparkFunSuite with MLl

[GitHub] spark pull request: [SPARK-14571][ML]Log instrumentation in ALS

2016-04-28 Thread MLnick
Github user MLnick commented on a diff in the pull request: https://github.com/apache/spark/pull/12560#discussion_r61384326 --- Diff: mllib/src/main/scala/org/apache/spark/ml/util/Instrumentation.scala --- @@ -85,6 +85,13 @@ private[ml] class Instrumentation[E <: Estimator[_]] pri

[GitHub] spark pull request: [SPARK-13568] [ML] Create feature transformer ...

2016-04-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11601#issuecomment-215335519 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-12660] [SPARK-14967] [SQL] Implement Ex...

2016-04-28 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/12736#discussion_r61383924 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/HiveTypeCoercionSuite.scala --- @@ -488,14 +488,6 @@ class HiveTypeCoercionSu

[GitHub] spark pull request: [SPARK-13568] [ML] Create feature transformer ...

2016-04-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11601#issuecomment-215335518 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-13568] [ML] Create feature transformer ...

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11601#issuecomment-215335486 **[Test build #57229 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57229/consoleFull)** for PR 11601 at commit [`4e1c34a`](https://g

[GitHub] spark pull request: [SPARK-14783] [SPARK-14786] [BRANCH-1.6] Prese...

2016-04-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12724#issuecomment-215335070 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-14783] [SPARK-14786] [BRANCH-1.6] Prese...

2016-04-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12724#issuecomment-215335073 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-14783] [SPARK-14786] [BRANCH-1.6] Prese...

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12724#issuecomment-215334962 **[Test build #57221 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57221/consoleFull)** for PR 12724 at commit [`49b7b52`](https://g

[GitHub] spark pull request: [SPARK-14706][SPARK-14973][ML][PySpark] Python...

2016-04-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12604#issuecomment-215334819 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-14706][SPARK-14973][ML][PySpark] Python...

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12604#issuecomment-215334571 **[Test build #57228 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57228/consoleFull)** for PR 12604 at commit [`cdab34a`](https://g

[GitHub] spark pull request: [SPARK-14706][SPARK-14973][ML][PySpark] Python...

2016-04-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12604#issuecomment-215334815 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-14654][CORE] New accumulator API

2016-04-28 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/12612 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request: [SPARK-14945][PYTHON] SparkSession Python API

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12746#issuecomment-215334370 **[Test build #57233 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57233/consoleFull)** for PR 12746 at commit [`e61aa8a`](https://gi

[GitHub] spark pull request: [SPARK-14858][SQL] Enable subquery pushdown

2016-04-28 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/12720#issuecomment-215334359 @hvanhovell That make sense, I will take another deep look tomorrow. --- If your project is set up for it, you can reply to this email and have your reply appear on GitH

[GitHub] spark pull request: [SPARK-14850][ML] convert primitive array from...

2016-04-28 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/12640#discussion_r61383378 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/UnsafeArrayData.java --- @@ -336,4 +336,62 @@ public UnsafeArrayData copy()

[GitHub] spark pull request: [SPARK-14654][CORE] New accumulator API

2016-04-28 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/12612#issuecomment-215334099 Merging in master! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this featu

[GitHub] spark pull request: [SPARK-14858][SQL] Enable subquery pushdown

2016-04-28 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/12720#discussion_r61383358 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/namedExpressions.scala --- @@ -337,6 +337,16 @@ case class PrettyAttribute(

[GitHub] spark pull request: [SPARK-14654][CORE] New accumulator API

2016-04-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12612#issuecomment-215332700 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-14945][PYTHON] SparkSession Python API

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12746#issuecomment-215333558 **[Test build #57232 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57232/consoleFull)** for PR 12746 at commit [`c4e48dc`](https://g

[GitHub] spark pull request: [SPARK-14945][PYTHON] SparkSession Python API

2016-04-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12746#issuecomment-215333563 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-14945][PYTHON] SparkSession Python API

2016-04-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12746#issuecomment-215333565 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-14938][ML] replace some RDD.map with Da...

2016-04-28 Thread viirya
Github user viirya commented on the pull request: https://github.com/apache/spark/pull/12718#issuecomment-215332276 e.g., in `BisectingKMeans`, this patch changes val data = dataset.select(col($(featuresCol))).rdd.map { case Row(point: Vector) => point } to

[GitHub] spark pull request: [SPARK-14945][PYTHON] SparkSession Python API

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12746#issuecomment-215333433 **[Test build #57232 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57232/consoleFull)** for PR 12746 at commit [`c4e48dc`](https://gi

[GitHub] spark pull request: [SPARK-14858][SQL] Enable subquery pushdown

2016-04-28 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/12720#discussion_r61382939 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/namedExpressions.scala --- @@ -337,6 +337,16 @@ case class PrettyAttribut

[GitHub] spark pull request: [SPARK-14706][SPARK-14973][ML][PySpark] Python...

2016-04-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12604#issuecomment-215332574 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-14706][SPARK-14973][ML][PySpark] Python...

2016-04-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12604#issuecomment-215332573 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-14654][CORE] New accumulator API

2016-04-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12612#issuecomment-215332701 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-14654][CORE] New accumulator API

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12612#issuecomment-215332517 **[Test build #57220 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57220/consoleFull)** for PR 12612 at commit [`124568b`](https://g

[GitHub] spark pull request: [SPARK-14706][SPARK-14973][ML][PySpark] Python...

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12604#issuecomment-215332479 **[Test build #57226 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57226/consoleFull)** for PR 12604 at commit [`ba664f9`](https://g

[GitHub] spark pull request: [SPARK-14938][ML] replace some RDD.map with Da...

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12718#issuecomment-215332412 **[Test build #57231 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57231/consoleFull)** for PR 12718 at commit [`2378829`](https://gi

[GitHub] spark pull request: [SPARK-14850][ML] specialize array data for Ve...

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12640#issuecomment-215331169 **[Test build #57230 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57230/consoleFull)** for PR 12640 at commit [`f4d2cbb`](https://gi

[GitHub] spark pull request: [SPARK-14858][SQL] Enable subquery pushdown

2016-04-28 Thread hvanhovell
Github user hvanhovell commented on the pull request: https://github.com/apache/spark/pull/12720#issuecomment-215330406 This is an example that will fail if we pull out predicates in the optimizer (`min(b.value)` does not resolve): ```sql select b.key, min(b.value) from src

[GitHub] spark pull request: [SPARK-13568] [ML] Create feature transformer ...

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11601#issuecomment-215330151 **[Test build #57229 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57229/consoleFull)** for PR 11601 at commit [`4e1c34a`](https://gi

[GitHub] spark pull request: [SPARK-13568] [ML] Create feature transformer ...

2016-04-28 Thread hhbyyh
Github user hhbyyh commented on the pull request: https://github.com/apache/spark/pull/11601#issuecomment-215330198 Documents updated and remove an unit test. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-14938][ML] replace some RDD.map with Da...

2016-04-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12718#issuecomment-215329786 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-14938][ML] replace some RDD.map with Da...

2016-04-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12718#issuecomment-215329787 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-14938][ML] replace some RDD.map with Da...

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12718#issuecomment-215329754 **[Test build #57225 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57225/consoleFull)** for PR 12718 at commit [`b2101b2`](https://g

[GitHub] spark pull request: [SPARK-13902][SCHEDULER] Make DAGScheduler.get...

2016-04-28 Thread suyanNone
Github user suyanNone commented on the pull request: https://github.com/apache/spark/pull/12655#issuecomment-215329471 as I know, duplicate stage occurs: stage 2 and stage 3 all depends on stage 1 stage 4 depends on stage 2 and stage 3 So, if we get getAncestorShuffl

<    4   5   6   7   8   9