[GitHub] spark issue #15043: [SPARK-17491] Close serialization stream to fix wrong an...

2016-09-12 Thread JoshRosen
Github user JoshRosen commented on the issue: https://github.com/apache/spark/pull/15043 #15056 also touches this code and creates a new test suite for this component so I'd prefer to merge that PR first. --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request #15056: [SPARK-17503][Core] Fix memory leak in Memory sto...

2016-09-12 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/15056#discussion_r78424222 --- Diff: core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala --- @@ -663,31 +663,43 @@ private[spark] class MemoryStore( private[storag

[GitHub] spark issue #15063: [SPARK-17463][Core]Make CollectionAccumulator and SetAcc...

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15063 **[Test build #65271 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65271/consoleFull)** for PR 15063 at commit [`21b6b4d`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15063: [SPARK-17463][Core]Make CollectionAccumulator and SetAcc...

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15063 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65271/ Test FAILed. ---

[GitHub] spark issue #15063: [SPARK-17463][Core]Make CollectionAccumulator and SetAcc...

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15063 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15060: [SPARK-17507][ML][MLLib] check weight vector size in ANN

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15060 **[Test build #65266 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65266/consoleFull)** for PR 15060 at commit [`e89aef5`](https://github.com/apache/spark/commit/e

[GitHub] spark issue #15059: [SPARK-17506][SQL] Improve the check double values equal...

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15059 **[Test build #65265 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65265/consoleFull)** for PR 15059 at commit [`78f3733`](https://github.com/apache/spark/commit/7

[GitHub] spark issue #15063: [SPARK-17463][Core]Make CollectionAccumulator and SetAcc...

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15063 **[Test build #65271 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65271/consoleFull)** for PR 15063 at commit [`21b6b4d`](https://github.com/apache/spark/commit/2

[GitHub] spark issue #14961: [SPARK-17379] [BUILD] Upgrade netty-all to 4.0.41 final ...

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14961 **[Test build #65264 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65264/consoleFull)** for PR 14961 at commit [`502ebf4`](https://github.com/apache/spark/commit/5

[GitHub] spark issue #15063: [SPARK-17463][Core]Make CollectionAccumulator and SetAcc...

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15063 **[Test build #65270 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65270/consoleFull)** for PR 15063 at commit [`4b8c277`](https://github.com/apache/spark/commit/4

[GitHub] spark issue #15057: [BUILD] Closing some stale PRs and ones suggested to be ...

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15057 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65254/ Test PASSed. ---

[GitHub] spark issue #15057: [BUILD] Closing some stale PRs and ones suggested to be ...

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15057 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15057: [BUILD] Closing some stale PRs and ones suggested to be ...

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15057 **[Test build #65254 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65254/consoleFull)** for PR 15057 at commit [`50a8a6c`](https://github.com/apache/spark/commit/

[GitHub] spark pull request #15063: [SPARK-17463][Core]Make CollectionAccumulator and...

2016-09-12 Thread zsxwing
GitHub user zsxwing opened a pull request: https://github.com/apache/spark/pull/15063 [SPARK-17463][Core]Make CollectionAccumulator and SetAccumulator's value can be read thread-safely ## What changes were proposed in this pull request? Make CollectionAccumulator and SetAcc

[GitHub] spark pull request #14834: [SPARK-17163][ML] Unified LogisticRegression inte...

2016-09-12 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/14834#discussion_r78420053 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala --- @@ -460,33 +577,74 @@ class LogisticRegression @Since("1.2.0")

[GitHub] spark issue #13762: [SPARK-14926] [ML] OneVsRest labelMetadata uses incorrec...

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13762 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #13762: [SPARK-14926] [ML] OneVsRest labelMetadata uses incorrec...

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13762 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65256/ Test PASSed. ---

[GitHub] spark pull request #14834: [SPARK-17163][ML] Unified LogisticRegression inte...

2016-09-12 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/14834#discussion_r78419383 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala --- @@ -311,8 +350,28 @@ class LogisticRegression @Since("1.2.0")

[GitHub] spark issue #13762: [SPARK-14926] [ML] OneVsRest labelMetadata uses incorrec...

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13762 **[Test build #65256 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65256/consoleFull)** for PR 13762 at commit [`b4badae`](https://github.com/apache/spark/commit/

[GitHub] spark pull request #15061: [SPARK-14818] Post-2.0 MiMa exclucsion and build ...

2016-09-12 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/15061#discussion_r78419280 --- Diff: project/MimaExcludes.scala --- @@ -787,9 +792,10 @@ object MimaExcludes { ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apa

[GitHub] spark issue #14969: [SPARK-17406][WEB UI] limit timeline executor events

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14969 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65261/ Test PASSed. ---

[GitHub] spark issue #14969: [SPARK-17406][WEB UI] limit timeline executor events

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14969 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14992: [SPARK-17378] [HOTFIX] Upgrade snappy-java to 1.1.2.6 --...

2016-09-12 Thread yhuai
Github user yhuai commented on the issue: https://github.com/apache/spark/pull/14992 Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feat

[GitHub] spark issue #14969: [SPARK-17406][WEB UI] limit timeline executor events

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14969 **[Test build #65261 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65261/consoleFull)** for PR 14969 at commit [`ac99524`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14750: [SPARK-17183][SQL] put hive serde table schema to table ...

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14750 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65262/ Test PASSed. ---

[GitHub] spark issue #14750: [SPARK-17183][SQL] put hive serde table schema to table ...

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14750 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14750: [SPARK-17183][SQL] put hive serde table schema to table ...

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14750 **[Test build #65262 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65262/consoleFull)** for PR 14750 at commit [`edafaa6`](https://github.com/apache/spark/commit/

[GitHub] spark pull request #15009: [SPARK-17443][SPARK-11035] Stop Spark Application...

2016-09-12 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/15009#discussion_r78417431 --- Diff: launcher/src/main/java/org/apache/spark/launcher/SparkLauncher.java --- @@ -538,6 +539,71 @@ public SparkAppHandle startApplication(SparkAppHand

[GitHub] spark pull request #15062: SPARK-17424: Fix unsound substitution bug in Scal...

2016-09-12 Thread rdblue
GitHub user rdblue opened a pull request: https://github.com/apache/spark/pull/15062 SPARK-17424: Fix unsound substitution bug in ScalaReflection. ## What changes were proposed in this pull request? This method gets a type's primary constructor and fills in type parameters

[GitHub] spark issue #14452: [SPARK-16849][SQL][WIP] Improve subquery execution by de...

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14452 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14452: [SPARK-16849][SQL][WIP] Improve subquery execution by de...

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14452 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65253/ Test PASSed. ---

[GitHub] spark issue #14452: [SPARK-16849][SQL][WIP] Improve subquery execution by de...

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14452 **[Test build #65253 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65253/consoleFull)** for PR 14452 at commit [`df8b490`](https://github.com/apache/spark/commit/

[GitHub] spark pull request #15061: [SPARK-14818] Post-2.0 MiMa exclucsion and build ...

2016-09-12 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/15061#discussion_r78415480 --- Diff: project/MimaExcludes.scala --- @@ -787,9 +792,10 @@ object MimaExcludes { ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.a

[GitHub] spark pull request #15061: [SPARK-14818] Post-2.0 MiMa exclucsion and build ...

2016-09-12 Thread JoshRosen
GitHub user JoshRosen opened a pull request: https://github.com/apache/spark/pull/15061 [SPARK-14818] Post-2.0 MiMa exclucsion and build changes This patch makes a handful of post-Spark-2.0 MiMa exclusion and build updates. It should be merged to master and a subset of it should be

[GitHub] spark issue #14678: [MINOR][SQL] Add missing functions for some options in S...

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14678 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65263/ Test PASSed. ---

[GitHub] spark issue #14678: [MINOR][SQL] Add missing functions for some options in S...

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14678 **[Test build #65263 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65263/consoleFull)** for PR 14678 at commit [`a26c08e`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14678: [MINOR][SQL] Add missing functions for some options in S...

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14678 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15035: [SPARK-17477]: SparkSQL cannot handle schema evolution f...

2016-09-12 Thread wgtmac
Github user wgtmac commented on the issue: https://github.com/apache/spark/pull/15035 @JoshRosen yes it may have mask overflow risk. This conversion happens when user provided schema or hive metastore schema has Long but the parquet files have Int as the schema. We cannot avoid this r

[GitHub] spark issue #10655: SPARK-12639 SQL Improve Explain for Datasources with Han...

2016-09-12 Thread yhuai
Github user yhuai commented on the issue: https://github.com/apache/spark/pull/10655 test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark issue #15035: [SPARK-17477]: SparkSQL cannot handle schema evolution f...

2016-09-12 Thread wgtmac
Github user wgtmac commented on the issue: https://github.com/apache/spark/pull/15035 @HyukjinKwon This is not parquet specific, it applies to other data sources as well. 1. Change the reading path for parquet: It does not solve the problem. Some queries need to read all parquet f

[GitHub] spark issue #15059: [SPARK-17506][SQL] Improve the check double values equal...

2016-09-12 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15059 Oh I see. Yes if we can move it to Spark's core test module that would be nicer. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If yo

[GitHub] spark issue #15059: [SPARK-17506][SQL] Improve the check double values equal...

2016-09-12 Thread WeichenXu123
Github user WeichenXu123 commented on the issue: https://github.com/apache/spark/pull/15059 but relTol is defined in mllib and sql not reference it, seems better to move it to spark-core project? --- If your project is set up for it, you can reply to this email and have your reply a

[GitHub] spark issue #15024: [SPARK-17470][SQL] unify path for data source table and ...

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15024 **[Test build #65260 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65260/consoleFull)** for PR 15024 at commit [`35def5b`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15024: [SPARK-17470][SQL] unify path for data source table and ...

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15024 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65260/ Test FAILed. ---

[GitHub] spark issue #15024: [SPARK-17470][SQL] unify path for data source table and ...

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15024 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15060: [SPARK-17507][ML][MLLib] check weight vector size in ANN

2016-09-12 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15060 Is there any other arg checking we can tighten up here? There are a couple places were weights need to match something else in size. --- If your project is set up for it, you can reply to this email

[GitHub] spark pull request #15060: [SPARK-17507][ML][MLLib] check weight vector size...

2016-09-12 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/15060#discussion_r78409055 --- Diff: mllib/src/main/scala/org/apache/spark/ml/ann/Layer.scala --- @@ -545,7 +553,9 @@ private[ann] object FeedForwardModel { * @return model

[GitHub] spark pull request #15060: [SPARK-17507][ML][MLLib] check weight vector size...

2016-09-12 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/15060#discussion_r78408999 --- Diff: mllib/src/main/scala/org/apache/spark/ml/ann/Layer.scala --- @@ -395,6 +395,14 @@ private[ann] class FeedForwardTopology private(val layers: Array[L

[GitHub] spark issue #15024: [SPARK-17470][SQL] unify path for data source table and ...

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15024 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65258/ Test FAILed. ---

[GitHub] spark issue #15024: [SPARK-17470][SQL] unify path for data source table and ...

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15024 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15024: [SPARK-17470][SQL] unify path for data source table and ...

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15024 **[Test build #65258 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65258/consoleFull)** for PR 15024 at commit [`d24d6ed`](https://github.com/apache/spark/commit/

[GitHub] spark pull request #15060: [SPARK-17507][ML][MLLib] check weight vector size...

2016-09-12 Thread WeichenXu123
GitHub user WeichenXu123 opened a pull request: https://github.com/apache/spark/pull/15060 [SPARK-17507][ML][MLLib] check weight vector size in ANN ## What changes were proposed in this pull request? as the TODO described, check weight vector size and if wrong throw exce

[GitHub] spark issue #15000: [SPARK-17437] Add uiWebUrl to JavaSparkContext and pyspa...

2016-09-12 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15000 Ah right dumb question. Yeah I think it makes some sense ... maybe not even for Java because there are lots of methods we don't plumb through because you can easily access them directly rom scala. Py

[GitHub] spark issue #15000: [SPARK-17437] Add uiWebUrl to JavaSparkContext and pyspa...

2016-09-12 Thread apetresc
Github user apetresc commented on the issue: https://github.com/apache/spark/pull/15000 PySpark. I don't think anyone runs Java through Jupyter, haha. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does n

[GitHub] spark issue #15000: [SPARK-17437] Add uiWebUrl to JavaSparkContext and pyspa...

2016-09-12 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15000 Is this Java or Pyspark? In Java you can still get this property directly from the underlying SparkContext. --- If your project is set up for it, you can reply to this email and have your reply appe

[GitHub] spark issue #15000: [SPARK-17437] Add uiWebUrl to JavaSparkContext and pyspa...

2016-09-12 Thread apetresc
Github user apetresc commented on the issue: https://github.com/apache/spark/pull/15000 Well, here's the use case I want it for: I'm building some plugins for JupyterHub to make it more Spark-aware, and I want to be able to link the user out to the right WebUI for their kernel. Short

[GitHub] spark pull request #15051: [SPARK-17499][ML][MLLib] make the default params ...

2016-09-12 Thread shivaram
Github user shivaram commented on a diff in the pull request: https://github.com/apache/spark/pull/15051#discussion_r78406032 --- Diff: R/pkg/R/mllib.R --- @@ -694,8 +694,8 @@ setMethod("predict", signature(object = "KMeansModel"), #' } #' @note spark.mlp since 2.1.0

[GitHub] spark issue #15058: [SPARK-17505][MLLIB]Add setBins for BinaryClassification...

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15058 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65255/ Test PASSed. ---

[GitHub] spark issue #15058: [SPARK-17505][MLLIB]Add setBins for BinaryClassification...

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15058 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15058: [SPARK-17505][MLLIB]Add setBins for BinaryClassification...

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15058 **[Test build #65255 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65255/consoleFull)** for PR 15058 at commit [`d06b2b1`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15040: [WIP] [SPARK-17487] [SQL] Configurable bucketing info ex...

2016-09-12 Thread tejasapatil
Github user tejasapatil commented on the issue: https://github.com/apache/spark/pull/15040 @cloud-fan : Would it be ok to add a field in CatalogTable to indicate if a table is from Hive ? For Hive tables, the hashing function also needs to be different while doing bucketing so having

[GitHub] spark issue #15009: [SPARK-17443][SPARK-11035] Stop Spark Application if lau...

2016-09-12 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/15009 It would also be nice to have unit tests for this. I see there are unit tests for launch but not startApplication, but I"m not sure why. --- If your project is set up for it, you can reply to thi

[GitHub] spark issue #12819: [SPARK-14077][ML] Refactor NaiveBayes to support weighte...

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/12819 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65257/ Test FAILed. ---

[GitHub] spark pull request #15009: [SPARK-17443][SPARK-11035] Stop Spark Application...

2016-09-12 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/15009#discussion_r78400990 --- Diff: launcher/src/main/java/org/apache/spark/launcher/SparkLauncher.java --- @@ -538,6 +539,71 @@ public SparkAppHandle startApplication(SparkAppHand

[GitHub] spark issue #12819: [SPARK-14077][ML] Refactor NaiveBayes to support weighte...

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/12819 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #12819: [SPARK-14077][ML] Refactor NaiveBayes to support weighte...

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/12819 **[Test build #65257 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65257/consoleFull)** for PR 12819 at commit [`9d53987`](https://github.com/apache/spark/commit/

[GitHub] spark issue #12819: [SPARK-14077][ML] Refactor NaiveBayes to support weighte...

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/12819 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #12819: [SPARK-14077][ML] Refactor NaiveBayes to support weighte...

2016-09-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/12819 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65259/ Test FAILed. ---

[GitHub] spark issue #12819: [SPARK-14077][ML] Refactor NaiveBayes to support weighte...

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/12819 **[Test build #65259 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65259/consoleFull)** for PR 12819 at commit [`b310fe0`](https://github.com/apache/spark/commit/

[GitHub] spark pull request #15009: [SPARK-17443][SPARK-11035] Stop Spark Application...

2016-09-12 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/15009#discussion_r78400270 --- Diff: launcher/src/main/java/org/apache/spark/launcher/SparkLauncher.java --- @@ -538,6 +539,71 @@ public SparkAppHandle startApplication(SparkAppHand

[GitHub] spark issue #15059: [SPARK-17506][SQL] Improve the check double values equal...

2016-09-12 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15059 We already have a 'relTol' operator for this purpose BTW. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request #15059: [SPARK-17506][SQL] Improve the check double value...

2016-09-12 Thread jiangxb1987
GitHub user jiangxb1987 opened a pull request: https://github.com/apache/spark/pull/15059 [SPARK-17506][SQL] Improve the check double values equality rule. ## What changes were proposed in this pull request? In `ExpressionEvalHelper`, we check the equality between two double

[GitHub] spark issue #15009: [SPARK-17443][SPARK-11035] Stop Spark Application if lau...

2016-09-12 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/15009 do all the options work in thread mode and client mode? For instance there were some things add in 2.x like redirectOutput and redirecterror that do processbuilder stuff and I assume don't work i

[GitHub] spark issue #14917: [SPARK-17142][SQL] Complex query triggers binding error ...

2016-09-12 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/14917 @hvanhovell Could you review this PR when you have some time please? Thank you! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. I

[GitHub] spark issue #12574: [SPARK-13857][ML][WIP] Add "recommend all" functionality...

2016-09-12 Thread MLnick
Github user MLnick commented on the issue: https://github.com/apache/spark/pull/12574 @debasish83 any chance to take a look yet? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14961: [SPARK-17379] [BUILD] Upgrade netty-all to 4.0.41 final ...

2016-09-12 Thread a-roberts
Github user a-roberts commented on the issue: https://github.com/apache/spark/pull/14961 No new test failures with my runs ranging from Hadoop 2.3 to Hadoop 2.7 today so pushed the commit above --- If your project is set up for it, you can reply to this email and have your reply appe

[GitHub] spark issue #14912: [SPARK-17357][SQL] Fix current predicate pushdown

2016-09-12 Thread nsyca
Github user nsyca commented on the issue: https://github.com/apache/spark/pull/14912 Thanks, @gatorsmile, for mentioning me. I will try my best to comment on this thread. Disclaimer: I have not looked at the existing code manipulating predicates/expressions in Spark. Nor have I the co

[GitHub] spark issue #14750: [SPARK-17183][SQL] put hive serde table schema to table ...

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14750 **[Test build #65262 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65262/consoleFull)** for PR 14750 at commit [`edafaa6`](https://github.com/apache/spark/commit/e

[GitHub] spark issue #15057: [BUILD] Closing some stale PRs and ones suggested to be ...

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15057 **[Test build #65254 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65254/consoleFull)** for PR 15057 at commit [`50a8a6c`](https://github.com/apache/spark/commit/5

[GitHub] spark issue #14969: [SPARK-17406][WEB UI] limit timeline executor events

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14969 **[Test build #65261 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65261/consoleFull)** for PR 14969 at commit [`ac99524`](https://github.com/apache/spark/commit/a

[GitHub] spark issue #15058: [SPARK-17505][MLLIB]Add setBins for BinaryClassification...

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15058 **[Test build #65255 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65255/consoleFull)** for PR 15058 at commit [`d06b2b1`](https://github.com/apache/spark/commit/d

[GitHub] spark issue #12819: [SPARK-14077][ML] Refactor NaiveBayes to support weighte...

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/12819 **[Test build #65259 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65259/consoleFull)** for PR 12819 at commit [`b310fe0`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #14678: [MINOR][SQL] Add missing functions for some options in S...

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14678 **[Test build #65263 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65263/consoleFull)** for PR 14678 at commit [`a26c08e`](https://github.com/apache/spark/commit/a

[GitHub] spark issue #15024: [SPARK-17470][SQL] unify path for data source table and ...

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15024 **[Test build #65260 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65260/consoleFull)** for PR 15024 at commit [`35def5b`](https://github.com/apache/spark/commit/3

[GitHub] spark issue #14452: [SPARK-16849][SQL][WIP] Improve subquery execution by de...

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14452 **[Test build #65253 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65253/consoleFull)** for PR 14452 at commit [`df8b490`](https://github.com/apache/spark/commit/d

[GitHub] spark issue #15024: [SPARK-17470][SQL] unify path for data source table and ...

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15024 **[Test build #65258 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65258/consoleFull)** for PR 15024 at commit [`d24d6ed`](https://github.com/apache/spark/commit/d

[GitHub] spark issue #13762: [SPARK-14926] [ML] OneVsRest labelMetadata uses incorrec...

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13762 **[Test build #65256 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65256/consoleFull)** for PR 13762 at commit [`b4badae`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #12819: [SPARK-14077][ML] Refactor NaiveBayes to support weighte...

2016-09-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/12819 **[Test build #65257 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65257/consoleFull)** for PR 12819 at commit [`9d53987`](https://github.com/apache/spark/commit/9

[GitHub] spark pull request #14834: [SPARK-17163][ML] Unified LogisticRegression inte...

2016-09-12 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/14834#discussion_r78389388 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala --- @@ -311,8 +350,28 @@ class LogisticRegression @Since("1.2.0")

[GitHub] spark pull request #14834: [SPARK-17163][ML] Unified LogisticRegression inte...

2016-09-12 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/14834#discussion_r7839 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/LogisticRegression.scala --- @@ -460,33 +577,74 @@ class LogisticRegression @Since("1.2.0")

[GitHub] spark pull request #15009: [SPARK-17443][SPARK-11035] Stop Spark Application...

2016-09-12 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/15009#discussion_r78388796 --- Diff: launcher/src/main/java/org/apache/spark/launcher/SparkSubmitRunner.java --- @@ -0,0 +1,59 @@ +/* + * Licensed to the Apache Software Foun

[GitHub] spark pull request #15009: [SPARK-17443][SPARK-11035] Stop Spark Application...

2016-09-12 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/15009#discussion_r78387851 --- Diff: launcher/src/main/java/org/apache/spark/launcher/SparkLauncher.java --- @@ -73,6 +74,21 @@ /** Logger name to use when launching a child p

[GitHub] spark pull request #15009: [SPARK-17443][SPARK-11035] Stop Spark Application...

2016-09-12 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/15009#discussion_r78387176 --- Diff: launcher/src/main/java/org/apache/spark/launcher/SparkLauncher.java --- @@ -73,6 +74,21 @@ /** Logger name to use when launching a child p

[GitHub] spark pull request #15009: [SPARK-17443][SPARK-11035] Stop Spark Application...

2016-09-12 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/15009#discussion_r78387149 --- Diff: launcher/src/main/java/org/apache/spark/launcher/SparkLauncher.java --- @@ -73,6 +74,21 @@ /** Logger name to use when launching a child p

[GitHub] spark pull request #15009: [SPARK-17443][SPARK-11035] Stop Spark Application...

2016-09-12 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/15009#discussion_r78386962 --- Diff: launcher/src/main/java/org/apache/spark/launcher/SparkLauncher.java --- @@ -73,6 +74,21 @@ /** Logger name to use when launching a child p

[GitHub] spark pull request #15009: [SPARK-17443][SPARK-11035] Stop Spark Application...

2016-09-12 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/15009#discussion_r78385859 --- Diff: launcher/src/main/java/org/apache/spark/launcher/SparkLauncher.java --- @@ -73,6 +74,21 @@ /** Logger name to use when launching a child p

[GitHub] spark pull request #15009: [SPARK-17443][SPARK-11035] Stop Spark Application...

2016-09-12 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/15009#discussion_r78385134 --- Diff: launcher/src/main/java/org/apache/spark/launcher/SparkLauncher.java --- @@ -73,6 +74,21 @@ /** Logger name to use when launching a child p

[GitHub] spark pull request #15009: [SPARK-17443][SPARK-11035] Stop Spark Application...

2016-09-12 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/15009#discussion_r78384231 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala --- @@ -73,6 +73,11 @@ private[spark] class Client( private val isClust

[GitHub] spark pull request #15009: [SPARK-17443][SPARK-11035] Stop Spark Application...

2016-09-12 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/15009#discussion_r78384350 --- Diff: launcher/src/main/java/org/apache/spark/launcher/SparkLauncher.java --- @@ -528,6 +528,7 @@ public SparkAppHandle startApplication(SparkAppHandl

[GitHub] spark issue #15009: [SPARK-17443][SPARK-11035] Stop Spark Application if lau...

2016-09-12 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/15009 we should update javadoc for package in launcher/src/main/java/org/apache/spark/launcher/package-info.java --- If your project is set up for it, you can reply to this email and have your reply ap

<    1   2   3   4   5   6   >