[GitHub] spark pull request #14834: [SPARK-17163][ML] Unified LogisticRegression inte...

2016-09-12 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/14834#discussion_r78464438 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala --- @@ -311,8 +350,28 @@ class LogisticRegression @Since("1.2.0")

[GitHub] spark pull request #15026: [SPARK-17472] [PYSPARK] Better error message for ...

2016-09-12 Thread Stibbons
Github user Stibbons commented on a diff in the pull request: https://github.com/apache/spark/pull/15026#discussion_r78464418 --- Diff: python/pyspark/broadcast.py --- @@ -75,7 +75,13 @@ def __init__(self, sc=None, value=None, pickle_registry=None, path=None): self

[GitHub] spark issue #15048: [SPARK-17409] [SQL] Do Not Optimize Query in CTAS More T...

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15048 **[Test build #65283 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65283/consoleFull)** for PR 15048 at commit [`4c3c955`](https://github.com/apache/spark/commit/4

[GitHub] spark issue #15065: [SPARK-17463][Core]Add necessary memory barrier for accu...

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15065 **[Test build #65282 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65282/consoleFull)** for PR 15065 at commit [`4c6bb0b`](https://github.com/apache/spark/commit/4

[GitHub] spark pull request #15065: [SPARK-17463][Core]Add necessary memory barrier f...

2016-09-12 Thread zsxwing
GitHub user zsxwing opened a pull request: https://github.com/apache/spark/pull/15065 [SPARK-17463][Core]Add necessary memory barrier for accumulators ## What changes were proposed in this pull request? Added `volatile` for fields that will be read in the heartbeat thread.

[GitHub] spark issue #14467: [SPARK-16861][PYSPARK][CORE] Refactor PySpark accumulato...

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14467 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14467: [SPARK-16861][PYSPARK][CORE] Refactor PySpark accumulato...

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14467 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65274/ Test PASSed. ---

[GitHub] spark issue #14467: [SPARK-16861][PYSPARK][CORE] Refactor PySpark accumulato...

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14467 **[Test build #65274 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65274/consoleFull)** for PR 14467 at commit [`6169c3c`](https://github.com/apache/spark/commit/

[GitHub] spark issue #11105: [SPARK-12469][CORE] Data Property accumulators for Spark

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/11105 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65276/ Test PASSed. ---

[GitHub] spark issue #11105: [SPARK-12469][CORE] Data Property accumulators for Spark

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/11105 **[Test build #65276 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65276/consoleFull)** for PR 11105 at commit [`491499d`](https://github.com/apache/spark/commit/

[GitHub] spark issue #11105: [SPARK-12469][CORE] Data Property accumulators for Spark

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/11105 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark pull request #15048: [SPARK-17409] [SQL] Do Not Optimize Query in CTAS...

2016-09-12 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/15048#discussion_r78461431 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/rules.scala --- @@ -68,7 +68,7 @@ class ResolveDataSource(sparkSession: Spar

[GitHub] spark issue #15063: [SPARK-17463][Core]Make CollectionAccumulator and SetAcc...

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15063 **[Test build #65281 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65281/consoleFull)** for PR 15063 at commit [`0da2f9b`](https://github.com/apache/spark/commit/0

[GitHub] spark issue #12601: [SPARK-14525][SQL] Make DataFrameWrite.save work for jdb...

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/12601 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65275/ Test PASSed. ---

[GitHub] spark issue #12601: [SPARK-14525][SQL] Make DataFrameWrite.save work for jdb...

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/12601 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #12601: [SPARK-14525][SQL] Make DataFrameWrite.save work for jdb...

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/12601 **[Test build #65275 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65275/consoleFull)** for PR 12601 at commit [`7ef7a48`](https://github.com/apache/spark/commit/

[GitHub] spark issue #13617: [SPARK-10409] [ML] Add Multilayer Perceptron Regression ...

2016-09-12 Thread avulanov
Github user avulanov commented on the issue: https://github.com/apache/spark/pull/13617 @JeremyNixon Sounds good! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wish

[GitHub] spark issue #15064: [SPARK-17509]]When wrapping catalyst datatype to Hive da...

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15064 **[Test build #65280 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65280/consoleFull)** for PR 15064 at commit [`19a2d96`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15064: [SPARK-17509]]When wrapping catalyst datatype to Hive da...

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15064 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15064: [SPARK-17509]]When wrapping catalyst datatype to Hive da...

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15064 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65280/ Test FAILed. ---

[GitHub] spark issue #15030: [SPARK-17474] [SQL] fix python udf in TakeOrderedAndProj...

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15030 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15030: [SPARK-17474] [SQL] fix python udf in TakeOrderedAndProj...

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15030 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65273/ Test PASSed. ---

[GitHub] spark issue #15064: [SPARK-17509]]When wrapping catalyst datatype to Hive da...

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15064 **[Test build #65280 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65280/consoleFull)** for PR 15064 at commit [`19a2d96`](https://github.com/apache/spark/commit/1

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13758 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15030: [SPARK-17474] [SQL] fix python udf in TakeOrderedAndProj...

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15030 **[Test build #65273 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65273/consoleFull)** for PR 15030 at commit [`1e319d8`](https://github.com/apache/spark/commit/

[GitHub] spark pull request #15064: [SPARK-17509]]When wrapping catalyst datatype to ...

2016-09-12 Thread sitalkedia
GitHub user sitalkedia opened a pull request: https://github.com/apache/spark/pull/15064 [SPARK-17509]]When wrapping catalyst datatype to Hive data type avoid… ## What changes were proposed in this pull request? When wrapping catalyst datatypes to Hive data type, wrap func

[GitHub] spark issue #15064: [SPARK-17509]]When wrapping catalyst datatype to Hive da...

2016-09-12 Thread sitalkedia
Github user sitalkedia commented on the issue: https://github.com/apache/spark/pull/15064 cc - @rxin --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13758 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65277/ Test FAILed. ---

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13758 **[Test build #65277 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65277/consoleFull)** for PR 13758 at commit [`45ae9bf`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15061: [SPARK-14818] Post-2.0 MiMa exclusion and build changes

2016-09-12 Thread yhuai
Github user yhuai commented on the issue: https://github.com/apache/spark/pull/15061 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark issue #14959: [SPARK-17387][PYSPARK] Creating SparkContext() from pyth...

2016-09-12 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/14959 Also someone else more familiar with pyspark (I know Holden has already looked), maybe @davies, should take a look. --- If your project is set up for it, you can reply to this email and have your re

[GitHub] spark issue #14959: [SPARK-17387][PYSPARK] Creating SparkContext() from pyth...

2016-09-12 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/14959 The conf code looks kinda nasty with the checks for whether a JVM has been set or not... I guess part of it is mandatory because otherwise this wouldn't work, but in particular, I'm not so sure the `

[GitHub] spark issue #15063: [SPARK-17463][Core]Make CollectionAccumulator and SetAcc...

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15063 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15063: [SPARK-17463][Core]Make CollectionAccumulator and SetAcc...

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15063 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65272/ Test FAILed. ---

[GitHub] spark issue #15063: [SPARK-17463][Core]Make CollectionAccumulator and SetAcc...

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15063 **[Test build #65272 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65272/consoleFull)** for PR 15063 at commit [`5a7183b`](https://github.com/apache/spark/commit/

[GitHub] spark pull request #14959: [SPARK-17387][PYSPARK] Creating SparkContext() fr...

2016-09-12 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/14959#discussion_r78453870 --- Diff: python/pyspark/java_gateway.py --- @@ -51,13 +51,16 @@ def launch_gateway(): on_windows = platform.system() == "Windows" sc

[GitHub] spark issue #13617: [SPARK-10409] [ML] Add Multilayer Perceptron Regression ...

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13617 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65278/ Test PASSed. ---

[GitHub] spark issue #13617: [SPARK-10409] [ML] Add Multilayer Perceptron Regression ...

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13617 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #13617: [SPARK-10409] [ML] Add Multilayer Perceptron Regression ...

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13617 **[Test build #65278 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65278/consoleFull)** for PR 13617 at commit [`509cb23`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15063: [SPARK-17463][Core]Make CollectionAccumulator and SetAcc...

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15063 **[Test build #3255 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3255/consoleFull)** for PR 15063 at commit [`21b6b4d`](https://github.com/apache/spark/commit

[GitHub] spark pull request #13642: [MINOR] Clean up several build warnings, mostly d...

2016-09-12 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/13642#discussion_r78450919 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/metric/SQLMetricsSuite.scala --- @@ -17,48 +17,18 @@ package org.apache.spark.sq

[GitHub] spark issue #15063: [SPARK-17463][Core]Make CollectionAccumulator and SetAcc...

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15063 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15063: [SPARK-17463][Core]Make CollectionAccumulator and SetAcc...

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15063 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65270/ Test PASSed. ---

[GitHub] spark issue #15063: [SPARK-17463][Core]Make CollectionAccumulator and SetAcc...

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15063 **[Test build #65270 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65270/consoleFull)** for PR 15063 at commit [`4b8c277`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15059: [SPARK-17506][SQL] Improve the check double values equal...

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15059 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65265/ Test PASSed. ---

[GitHub] spark issue #15059: [SPARK-17506][SQL] Improve the check double values equal...

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15059 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15059: [SPARK-17506][SQL] Improve the check double values equal...

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15059 **[Test build #65265 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65265/consoleFull)** for PR 15059 at commit [`78f3733`](https://github.com/apache/spark/commit/

[GitHub] spark issue #13617: [SPARK-10409] [ML] Add Multilayer Perceptron Regression ...

2016-09-12 Thread JeremyNixon
Github user JeremyNixon commented on the issue: https://github.com/apache/spark/pull/13617 @avulanov I am interested - how about I replicate this PR at github.com/avulanov/scalable-deeplearning and we discuss details there? --- If your project is set up for it, you can reply to this

[GitHub] spark issue #15037: [SPARK-17485] Prevent failed remote reads of cached bloc...

2016-09-12 Thread ericl
Github user ericl commented on the issue: https://github.com/apache/spark/pull/15037 This LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the fe

[GitHub] spark issue #15062: SPARK-17424: Fix unsound substitution bug in ScalaReflec...

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15062 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65269/ Test PASSed. ---

[GitHub] spark issue #15062: SPARK-17424: Fix unsound substitution bug in ScalaReflec...

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15062 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15062: SPARK-17424: Fix unsound substitution bug in ScalaReflec...

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15062 **[Test build #65269 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65269/consoleFull)** for PR 15062 at commit [`931f156`](https://github.com/apache/spark/commit/

[GitHub] spark pull request #15036: [SPARK-17483] Refactoring in BlockManager status ...

2016-09-12 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15036 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #15036: [SPARK-17483] Refactoring in BlockManager status reporti...

2016-09-12 Thread JoshRosen
Github user JoshRosen commented on the issue: https://github.com/apache/spark/pull/15036 Thanks for the reviews. I'm going to merge this into master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does no

[GitHub] spark issue #14842: [SPARK-10747][SQL] Support NULLS FIRST|LAST clause in OR...

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14842 **[Test build #65279 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65279/consoleFull)** for PR 14842 at commit [`5153ce5`](https://github.com/apache/spark/commit/5

[GitHub] spark issue #14961: [SPARK-17379] [BUILD] Upgrade netty-all to 4.0.41 final ...

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14961 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14961: [SPARK-17379] [BUILD] Upgrade netty-all to 4.0.41 final ...

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14961 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65264/ Test FAILed. ---

[GitHub] spark issue #14961: [SPARK-17379] [BUILD] Upgrade netty-all to 4.0.41 final ...

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14961 **[Test build #65264 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65264/consoleFull)** for PR 14961 at commit [`502ebf4`](https://github.com/apache/spark/commit/

[GitHub] spark issue #13617: [SPARK-10409] [ML] Add Multilayer Perceptron Regression ...

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13617 **[Test build #65278 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65278/consoleFull)** for PR 13617 at commit [`509cb23`](https://github.com/apache/spark/commit/5

[GitHub] spark issue #13758: [SPARK-16043][SQL] Prepare GenericArrayData implementati...

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13758 **[Test build #65277 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65277/consoleFull)** for PR 13758 at commit [`45ae9bf`](https://github.com/apache/spark/commit/4

[GitHub] spark issue #12601: [SPARK-14525][SQL] Make DataFrameWrite.save work for jdb...

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/12601 **[Test build #65275 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65275/consoleFull)** for PR 12601 at commit [`7ef7a48`](https://github.com/apache/spark/commit/7

[GitHub] spark issue #11105: [SPARK-12469][CORE] Data Property accumulators for Spark

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/11105 **[Test build #65276 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65276/consoleFull)** for PR 11105 at commit [`491499d`](https://github.com/apache/spark/commit/4

[GitHub] spark issue #14467: [SPARK-16861][PYSPARK][CORE] Refactor PySpark accumulato...

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14467 **[Test build #65274 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65274/consoleFull)** for PR 14467 at commit [`6169c3c`](https://github.com/apache/spark/commit/6

[GitHub] spark pull request #14644: [MESOS] Enable GPU support with Mesos

2016-09-12 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/14644#discussion_r78438011 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosCoarseGrainedSchedulerBackend.scala --- @@ -103,6 +103,7 @@ private[spark] class

[GitHub] spark issue #12601: [SPARK-14525][SQL] Make DataFrameWrite.save work for jdb...

2016-09-12 Thread JustinPihony
Github user JustinPihony commented on the issue: https://github.com/apache/spark/pull/12601 @srowen Documentation added. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled a

[GitHub] spark issue #15024: [SPARK-17470][SQL] unify path for data source table and ...

2016-09-12 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15024 [`showDataSourceTableOptions`](https://github.com/apache/spark/blob/c0ae6bc6ea38909730fad36e653d3c7ab0a84b44/sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala#L866-L873)

[GitHub] spark issue #15030: [SPARK-17474] [SQL] fix python udf in TakeOrderedAndProj...

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15030 **[Test build #65273 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65273/consoleFull)** for PR 15030 at commit [`1e319d8`](https://github.com/apache/spark/commit/1

[GitHub] spark issue #15023: Backport [SPARK-5847] Allow for configuring MetricsSyste...

2016-09-12 Thread marmbrus
Github user marmbrus commented on the issue: https://github.com/apache/spark/pull/15023 Thanks for spending the time to backport this, but it does seem a little risky to include changes to the configuration system in a maintenance release. As such, I'd probably error on the side of c

[GitHub] spark pull request #15030: [SPARK-17474] [SQL] fix python udf in TakeOrdered...

2016-09-12 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/15030#discussion_r78435023 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/limit.scala --- @@ -148,8 +148,8 @@ case class TakeOrderedAndProjectExec( localT

[GitHub] spark issue #15060: [SPARK-17507][ML][MLLib] check weight vector size in ANN

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15060 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15060: [SPARK-17507][ML][MLLib] check weight vector size in ANN

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15060 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65266/ Test PASSed. ---

[GitHub] spark issue #15060: [SPARK-17507][ML][MLLib] check weight vector size in ANN

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15060 **[Test build #65266 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65266/consoleFull)** for PR 15060 at commit [`e89aef5`](https://github.com/apache/spark/commit/

[GitHub] spark pull request #15030: [SPARK-17474] [SQL] fix python udf in TakeOrdered...

2016-09-12 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/15030#discussion_r78434025 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/limit.scala --- @@ -148,8 +148,8 @@ case class TakeOrderedAndProjectExec( loc

[GitHub] spark issue #14857: [SPARK-17261][PYSPARK] Using HiveContext after re-creati...

2016-09-12 Thread coderfi
Github user coderfi commented on the issue: https://github.com/apache/spark/pull/14857 Awesome, we ran into this problem as well, and finally had some bandwidth to track down the cause well enough to be able to search for this pull request. Looking forward to test this in our fork.

[GitHub] spark pull request #15063: [SPARK-17463][Core]Make CollectionAccumulator and...

2016-09-12 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/15063#discussion_r78433588 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/debug/package.scala --- @@ -107,18 +109,20 @@ package object debug { case class Debug

[GitHub] spark pull request #15024: [SPARK-17470][SQL] unify path for data source tab...

2016-09-12 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/15024#discussion_r78433410 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala --- @@ -447,17 +461,10 @@ private[spark] class HiveExternalCatalog(c

[GitHub] spark issue #15063: [SPARK-17463][Core]Make CollectionAccumulator and SetAcc...

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15063 **[Test build #65272 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65272/consoleFull)** for PR 15063 at commit [`5a7183b`](https://github.com/apache/spark/commit/5

[GitHub] spark pull request #15063: [SPARK-17463][Core]Make CollectionAccumulator and...

2016-09-12 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/15063#discussion_r78432833 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/debug/package.scala --- @@ -107,18 +109,20 @@ package object debug { case class Deb

[GitHub] spark issue #14467: [SPARK-16861][PYSPARK][CORE] Refactor PySpark accumulato...

2016-09-12 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/14467 Ping @MLnick / @srowen ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #15026: [SPARK-17472] [PYSPARK] Better error message for seriali...

2016-09-12 Thread davies
Github user davies commented on the issue: https://github.com/apache/spark/pull/15026 Once we could log the original stacktrace, this looks good to me. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request #15026: [SPARK-17472] [PYSPARK] Better error message for ...

2016-09-12 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/15026#discussion_r78432383 --- Diff: python/pyspark/cloudpickle.py --- @@ -109,6 +109,15 @@ def dump(self, obj): if 'recursion' in e.args[0]: msg =

[GitHub] spark pull request #15026: [SPARK-17472] [PYSPARK] Better error message for ...

2016-09-12 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/15026#discussion_r78432125 --- Diff: python/pyspark/broadcast.py --- @@ -75,7 +75,13 @@ def __init__(self, sc=None, value=None, pickle_registry=None, path=None): self._

[GitHub] spark pull request #15036: [SPARK-17483] Refactoring in BlockManager status ...

2016-09-12 Thread ericl
Github user ericl commented on a diff in the pull request: https://github.com/apache/spark/pull/15036#discussion_r78430377 --- Diff: core/src/main/scala/org/apache/spark/storage/BlockManager.scala --- @@ -1316,21 +1303,31 @@ private[spark] class BlockManager( // The bl

[GitHub] spark pull request #15056: [SPARK-17503][Core] Fix memory leak in Memory sto...

2016-09-12 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15056 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #15056: [SPARK-17503][Core] Fix memory leak in Memory store when...

2016-09-12 Thread JoshRosen
Github user JoshRosen commented on the issue: https://github.com/apache/spark/pull/15056 LGTM as well, so I'm going to merge this to master and branch-2.0 (2.0.1-SNAPSHOT). Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark pull request #15056: [SPARK-17503][Core] Fix memory leak in Memory sto...

2016-09-12 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/15056#discussion_r78426904 --- Diff: core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala --- @@ -663,31 +663,43 @@ private[spark] class MemoryStore( private[storag

[GitHub] spark issue #10655: [SPARK-12639][SQL] Improve Explain for Datasources with ...

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/10655 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #10655: [SPARK-12639][SQL] Improve Explain for Datasources with ...

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/10655 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65267/ Test FAILed. ---

[GitHub] spark issue #10655: [SPARK-12639][SQL] Improve Explain for Datasources with ...

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/10655 **[Test build #65267 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65267/consoleFull)** for PR 10655 at commit [`5a0daf6`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15063: [SPARK-17463][Core]Make CollectionAccumulator and SetAcc...

2016-09-12 Thread JoshRosen
Github user JoshRosen commented on the issue: https://github.com/apache/spark/pull/15063 I think that we may also want to do this for `BlockStatusesAccumulator` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your proj

[GitHub] spark pull request #15056: [SPARK-17503][Core] Fix memory leak in Memory sto...

2016-09-12 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/15056#discussion_r78425128 --- Diff: core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala --- @@ -663,31 +663,43 @@ private[spark] class MemoryStore( private[st

[GitHub] spark issue #14842: [SPARK-10747][SQL] Support NULLS FIRST|LAST clause in OR...

2016-09-12 Thread xwu0226
Github user xwu0226 commented on the issue: https://github.com/apache/spark/pull/14842 @ericl Thanks so much for the detailed review and suggestions. I will fix the last comments. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark issue #14842: [SPARK-10747][SQL] Support NULLS FIRST|LAST clause in OR...

2016-09-12 Thread ericl
Github user ericl commented on the issue: https://github.com/apache/spark/pull/14842 A few more minor comments but otherwise the prefix parts look good to me! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark issue #10655: [SPARK-12639][SQL] Improve Explain for Datasources with ...

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/10655 **[Test build #65267 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65267/consoleFull)** for PR 10655 at commit [`5a0daf6`](https://github.com/apache/spark/commit/5

[GitHub] spark issue #15062: SPARK-17424: Fix unsound substitution bug in ScalaReflec...

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15062 **[Test build #65269 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65269/consoleFull)** for PR 15062 at commit [`931f156`](https://github.com/apache/spark/commit/9

[GitHub] spark issue #15061: [SPARK-14818] Post-2.0 MiMa exclucsion and build changes

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15061 **[Test build #65268 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65268/consoleFull)** for PR 15061 at commit [`1224e75`](https://github.com/apache/spark/commit/1

[GitHub] spark pull request #14842: [SPARK-10747][SQL] Support NULLS FIRST|LAST claus...

2016-09-12 Thread ericl
Github user ericl commented on a diff in the pull request: https://github.com/apache/spark/pull/14842#discussion_r78424378 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/SortOrder.scala --- @@ -21,26 +21,44 @@ import org.apache.spark.sql.catalyst.I

[GitHub] spark issue #15063: [SPARK-17463][Core]Make CollectionAccumulator and SetAcc...

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15063 **[Test build #3255 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3255/consoleFull)** for PR 15063 at commit [`21b6b4d`](https://github.com/apache/spark/commit/

[GitHub] spark pull request #14842: [SPARK-10747][SQL] Support NULLS FIRST|LAST claus...

2016-09-12 Thread ericl
Github user ericl commented on a diff in the pull request: https://github.com/apache/spark/pull/14842#discussion_r78424255 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/SortOrder.scala --- @@ -71,12 +95,38 @@ case class SortPrefix(child: SortOrder

[GitHub] spark pull request #14842: [SPARK-10747][SQL] Support NULLS FIRST|LAST claus...

2016-09-12 Thread ericl
Github user ericl commented on a diff in the pull request: https://github.com/apache/spark/pull/14842#discussion_r78424193 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/SortOrder.scala --- @@ -71,12 +95,38 @@ case class SortPrefix(child: SortOrder

<    1   2   3   4   5   6   >