[GitHub] spark issue #16785: [SPARK-19443][SQL] The function to generate constraints ...

2017-02-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16785 **[Test build #72303 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72303/testReport)** for PR 16785 at commit [`b4e514a`](https://github.com/apache/spark/commit/b4

[GitHub] spark issue #16775: [SPARK-19433][ML] Periodic checkout datasets for long ml...

2017-02-02 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/16775 For the issue reported on mailing list, I found the root cause makes significant difference between 1.6 and current branch. The fix is at #16785. However, I think this patch is still useful.

[GitHub] spark pull request #16785: [SPARK-19443][SQL] The function to generate const...

2017-02-02 Thread viirya
GitHub user viirya opened a pull request: https://github.com/apache/spark/pull/16785 [SPARK-19443][SQL] The function to generate constraints takes too long when the query plan grows continuously ## What changes were proposed in this pull request? This issue is originally re

[GitHub] spark pull request #16766: [SPARK-19426][SQL] Custom coalesce for Dataset

2017-02-02 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16766#discussion_r99286995 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DatasetSuite.scala --- @@ -117,6 +134,34 @@ class DatasetSuite extends QueryTest with SharedSQLCon

[GitHub] spark pull request #16766: [SPARK-19426][SQL] Custom coalesce for Dataset

2017-02-02 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16766#discussion_r99286957 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DatasetSuite.scala --- @@ -17,24 +17,41 @@ package org.apache.spark.sql -impor

[GitHub] spark pull request #16766: [SPARK-19426][SQL] Custom coalesce for Dataset

2017-02-02 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16766#discussion_r99286805 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DatasetSuite.scala --- @@ -117,6 +134,34 @@ class DatasetSuite extends QueryTest with SharedSQLCon

[GitHub] spark pull request #16779: [SPARK-19437] Rectify spark executor id in Heartb...

2017-02-02 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16779 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request #16766: [SPARK-19426][SQL] Custom coalesce for Dataset

2017-02-02 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16766#discussion_r99286475 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DatasetSuite.scala --- @@ -117,6 +134,34 @@ class DatasetSuite extends QueryTest with SharedSQLCon

[GitHub] spark issue #16779: [SPARK-19437] Rectify spark executor id in HeartbeatRece...

2017-02-02 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/16779 Thanks! Merging to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wish

[GitHub] spark pull request #16766: [SPARK-19426][SQL] Custom coalesce for Dataset

2017-02-02 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16766#discussion_r99286218 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/basicPhysicalOperators.scala --- @@ -497,7 +496,9 @@ case class UnionExec(children: Seq[

[GitHub] spark pull request #16766: [SPARK-19426][SQL] Custom coalesce for Dataset

2017-02-02 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16766#discussion_r99286066 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/basicPhysicalOperators.scala --- @@ -19,9 +19,8 @@ package org.apache.spark.sql.executio

[GitHub] spark issue #16784: [SPARK-19382][ML]:Test sparse vectors in LinearSVCSuite

2017-02-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16784 **[Test build #72302 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72302/testReport)** for PR 16784 at commit [`fc1f7d1`](https://github.com/apache/spark/commit/fc

[GitHub] spark pull request #16766: [SPARK-19426][SQL] Custom coalesce for Dataset

2017-02-02 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16766#discussion_r99285902 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala --- @@ -823,6 +825,17 @@ case class Reparti

[GitHub] spark pull request #16766: [SPARK-19426][SQL] Custom coalesce for Dataset

2017-02-02 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16766#discussion_r99285925 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala --- @@ -823,6 +825,17 @@ case class Reparti

[GitHub] spark pull request #16766: [SPARK-19426][SQL] Custom coalesce for Dataset

2017-02-02 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16766#discussion_r99285876 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala --- @@ -823,6 +825,17 @@ case class Reparti

[GitHub] spark pull request #16784: [SPARK-19382][ML]:Test sparse vectors in LinearSV...

2017-02-02 Thread wangmiao1981
GitHub user wangmiao1981 opened a pull request: https://github.com/apache/spark/pull/16784 [SPARK-19382][ML]:Test sparse vectors in LinearSVCSuite ## What changes were proposed in this pull request? Add unit tests for testing SparseVector. We can't add mixed DenseVe

[GitHub] spark pull request #16766: [SPARK-19426][SQL] Custom coalesce for Dataset

2017-02-02 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16766#discussion_r99285447 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala --- @@ -17,7 +17,9 @@ package or

[GitHub] spark pull request #16766: [SPARK-19426][SQL] Custom coalesce for Dataset

2017-02-02 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16766#discussion_r99284849 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/basicPhysicalOperators.scala --- @@ -497,7 +496,9 @@ case class UnionExec(children: Seq[

[GitHub] spark pull request #16766: [SPARK-19426][SQL] Custom coalesce for Dataset

2017-02-02 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16766#discussion_r99284809 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -2437,9 +2435,12 @@ class Dataset[T] private[sql]( * @group typedrel

[GitHub] spark issue #16765: [SPARK-19425][SQL] Make ExtractEquiJoinKeys support UDT ...

2017-02-02 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16765 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the fe

[GitHub] spark pull request #16777: [SPARK-19435][SQL] Type coercion between ArrayTyp...

2017-02-02 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16777#discussion_r99283834 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala --- @@ -101,24 +101,13 @@ object TypeCoercion {

[GitHub] spark issue #16138: [SPARK-16609] Add to_date/to_timestamp with format funct...

2017-02-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16138 **[Test build #72301 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72301/testReport)** for PR 16138 at commit [`a2d0221`](https://github.com/apache/spark/commit/a2

[GitHub] spark pull request #16138: [SPARK-16609] Add to_date/to_timestamp with forma...

2017-02-02 Thread anabranch
Github user anabranch commented on a diff in the pull request: https://github.com/apache/spark/pull/16138#discussion_r99283708 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala --- @@ -1047,6 +1048,64 @@ case class ToDate(chi

[GitHub] spark issue #16779: [SPARK-19437] Rectify spark executor id in HeartbeatRece...

2017-02-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16779 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16779: [SPARK-19437] Rectify spark executor id in HeartbeatRece...

2017-02-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16779 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72297/ Test PASSed. ---

[GitHub] spark issue #16779: [SPARK-19437] Rectify spark executor id in HeartbeatRece...

2017-02-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16779 **[Test build #72297 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72297/testReport)** for PR 16779 at commit [`a9bc3f4`](https://github.com/apache/spark/commit/a

[GitHub] spark issue #16740: [SPARK-19400][ML] Allow GLM to handle intercept only mod...

2017-02-02 Thread actuaryzhang
Github user actuaryzhang commented on the issue: https://github.com/apache/spark/pull/16740 @seth @imatiach-msft Let me know if there is any other changes needed. Thanks much for your review! --- If your project is set up for it, you can reply to this email and have your reply appea

[GitHub] spark issue #16765: [SPARK-19425][SQL] Make ExtractEquiJoinKeys support UDT ...

2017-02-02 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/16765 @gatorsmile Updated. Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wis

[GitHub] spark issue #16740: [SPARK-19400][ML] Allow GLM to handle intercept only mod...

2017-02-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16740 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16740: [SPARK-19400][ML] Allow GLM to handle intercept only mod...

2017-02-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16740 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72299/ Test PASSed. ---

[GitHub] spark issue #16740: [SPARK-19400][ML] Allow GLM to handle intercept only mod...

2017-02-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16740 **[Test build #72299 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72299/testReport)** for PR 16740 at commit [`b57af08`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #16783: [SPARK-19441] [SQL] Remove IN type coercion from Promote...

2017-02-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16783 **[Test build #72300 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72300/testReport)** for PR 16783 at commit [`127a114`](https://github.com/apache/spark/commit/12

[GitHub] spark pull request #16783: [SPARK-19441] [SQL] Remove IN type coercion from ...

2017-02-02 Thread gatorsmile
GitHub user gatorsmile opened a pull request: https://github.com/apache/spark/pull/16783 [SPARK-19441] [SQL] Remove IN type coercion from PromoteStrings ### What changes were proposed in this pull request? The removed codes are not reachable, because `InConversion` already resolv

[GitHub] spark issue #16740: [SPARK-19400][ML] Allow GLM to handle intercept only mod...

2017-02-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16740 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72298/ Test PASSed. ---

[GitHub] spark issue #16740: [SPARK-19400][ML] Allow GLM to handle intercept only mod...

2017-02-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16740 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16740: [SPARK-19400][ML] Allow GLM to handle intercept only mod...

2017-02-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16740 **[Test build #72298 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72298/testReport)** for PR 16740 at commit [`931f7ec`](https://github.com/apache/spark/commit/9

[GitHub] spark issue #14702: [SPARK-15694] Implement ScriptTransformation in sql/core...

2017-02-02 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/14702 I will try to review it in the next few days. Thanks for working on it! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pr

[GitHub] spark issue #16765: [SPARK-19425][SQL] Make df.except work for UDT

2017-02-02 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16765 Could you update the PR description and title? This PR fixes three scenarios: - `except`on two Datasets with UDT - `intersect` on two Datasets with UDT - `Join` with the join conditi

[GitHub] spark pull request #16722: [SPARK-9478][ML][MLlib] Add sample weights to dec...

2017-02-02 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/16722#discussion_r99279786 --- Diff: mllib/src/test/scala/org/apache/spark/ml/classification/DecisionTreeClassifierSuite.scala --- @@ -351,6 +370,36 @@ class DecisionTreeClassif

[GitHub] spark pull request #16722: [SPARK-9478][ML][MLlib] Add sample weights to dec...

2017-02-02 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/16722#discussion_r99279066 --- Diff: mllib/src/test/scala/org/apache/spark/mllib/tree/ImpuritySuite.scala --- @@ -18,23 +18,62 @@ package org.apache.spark.mllib.tree

[GitHub] spark pull request #16722: [SPARK-9478][ML][MLlib] Add sample weights to dec...

2017-02-02 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/16722#discussion_r99278975 --- Diff: mllib/src/test/scala/org/apache/spark/ml/util/MLTestingUtils.scala --- @@ -281,10 +283,26 @@ object MLTestingUtils extends SparkFunSuite {

[GitHub] spark pull request #16722: [SPARK-9478][ML][MLlib] Add sample weights to dec...

2017-02-02 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/16722#discussion_r99278910 --- Diff: mllib/src/test/scala/org/apache/spark/ml/tree/impl/TreeTests.scala --- @@ -124,8 +129,8 @@ private[ml] object TreeTests extends SparkFunSuite

[GitHub] spark pull request #16138: [SPARK-16609] Add to_date/to_timestamp with forma...

2017-02-02 Thread anabranch
Github user anabranch commented on a diff in the pull request: https://github.com/apache/spark/pull/16138#discussion_r99278789 --- Diff: R/pkg/inst/tests/testthat/test_sparkSQL.R --- @@ -1177,6 +1177,9 @@ test_that("column functions", { c17 <- cov(c, c1) + cov("c", "c1") + co

[GitHub] spark pull request #16138: [SPARK-16609] Add to_date/to_timestamp with forma...

2017-02-02 Thread anabranch
Github user anabranch commented on a diff in the pull request: https://github.com/apache/spark/pull/16138#discussion_r99278746 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala --- @@ -1047,6 +1048,64 @@ case class ToDate(chi

[GitHub] spark pull request #16138: [SPARK-16609] Add to_date/to_timestamp with forma...

2017-02-02 Thread anabranch
Github user anabranch commented on a diff in the pull request: https://github.com/apache/spark/pull/16138#discussion_r99278738 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala --- @@ -1047,6 +1048,64 @@ case class ToDate(chi

[GitHub] spark pull request #16138: [SPARK-16609] Add to_date/to_timestamp with forma...

2017-02-02 Thread anabranch
Github user anabranch commented on a diff in the pull request: https://github.com/apache/spark/pull/16138#discussion_r99278624 --- Diff: R/pkg/R/functions.R --- @@ -1746,7 +1750,7 @@ setMethod("toRadians", #' to_date(df$c) #' to_date(df$c, '-MM-dd') #' } -#' @

[GitHub] spark pull request #16722: [SPARK-9478][ML][MLlib] Add sample weights to dec...

2017-02-02 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/16722#discussion_r99278201 --- Diff: mllib/src/test/scala/org/apache/spark/ml/classification/DecisionTreeClassifierSuite.scala --- @@ -58,6 +62,20 @@ class DecisionTreeClassifie

[GitHub] spark pull request #16722: [SPARK-9478][ML][MLlib] Add sample weights to dec...

2017-02-02 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/16722#discussion_r99278110 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/tree/impurity/Variance.scala --- @@ -70,17 +70,24 @@ object Variance extends Impurity { * N

[GitHub] spark pull request #16740: [SPARK-19400][ML] Allow GLM to handle intercept o...

2017-02-02 Thread actuaryzhang
Github user actuaryzhang commented on a diff in the pull request: https://github.com/apache/spark/pull/16740#discussion_r99277921 --- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/GeneralizedLinearRegressionSuite.scala --- @@ -743,6 +744,48 @@ class GeneralizedLinearRe

[GitHub] spark pull request #16722: [SPARK-9478][ML][MLlib] Add sample weights to dec...

2017-02-02 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/16722#discussion_r99277799 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/tree/impurity/Gini.scala --- @@ -80,23 +80,29 @@ object Gini extends Impurity { * @param nu

[GitHub] spark issue #16740: [SPARK-19400][ML] Allow GLM to handle intercept only mod...

2017-02-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16740 **[Test build #72299 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72299/testReport)** for PR 16740 at commit [`b57af08`](https://github.com/apache/spark/commit/b5

[GitHub] spark pull request #16740: [SPARK-19400][ML] Allow GLM to handle intercept o...

2017-02-02 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/16740#discussion_r99276472 --- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/GeneralizedLinearRegressionSuite.scala --- @@ -743,6 +744,48 @@ class GeneralizedLinearR

[GitHub] spark issue #16740: [SPARK-19400][ML] Allow GLM to handle intercept only mod...

2017-02-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16740 **[Test build #72298 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72298/testReport)** for PR 16740 at commit [`931f7ec`](https://github.com/apache/spark/commit/93

[GitHub] spark pull request #16740: [SPARK-19400][ML] Allow GLM to handle intercept o...

2017-02-02 Thread actuaryzhang
Github user actuaryzhang commented on a diff in the pull request: https://github.com/apache/spark/pull/16740#discussion_r99276315 --- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/GeneralizedLinearRegression.scala --- @@ -335,6 +335,11 @@ class GeneralizedLinearRegress

[GitHub] spark pull request #16722: [SPARK-9478][ML][MLlib] Add sample weights to dec...

2017-02-02 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/16722#discussion_r99275923 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/tree/impurity/Entropy.scala --- @@ -83,23 +83,29 @@ object Entropy extends Impurity { * @pa

[GitHub] spark pull request #16722: [SPARK-9478][ML][MLlib] Add sample weights to dec...

2017-02-02 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/16722#discussion_r99275700 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/tree/impurity/Impurity.scala --- @@ -79,7 +79,12 @@ private[spark] abstract class ImpurityAggrega

[GitHub] spark pull request #16722: [SPARK-9478][ML][MLlib] Add sample weights to dec...

2017-02-02 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/16722#discussion_r99274459 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/tree/impurity/Entropy.scala --- @@ -83,23 +83,29 @@ object Entropy extends Impurity { * @pa

[GitHub] spark pull request #16733: [SPARK-19392][SQL] Fix the bug that throws an exc...

2017-02-02 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/16733#discussion_r99273860 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/jdbc/OracleDialect.scala --- @@ -29,7 +29,12 @@ private case object OracleDialect extends JdbcDialect {

[GitHub] spark pull request #16733: [SPARK-19392][SQL] Fix the bug that throws an exc...

2017-02-02 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16733#discussion_r99273809 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/jdbc/OracleDialect.scala --- @@ -29,7 +29,12 @@ private case object OracleDialect extends JdbcDiale

[GitHub] spark pull request #16740: [SPARK-19400][ML] Allow GLM to handle intercept o...

2017-02-02 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/16740#discussion_r99273642 --- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/GeneralizedLinearRegression.scala --- @@ -335,6 +335,11 @@ class GeneralizedLinearRegres

[GitHub] spark issue #16765: [SPARK-19425][SQL] Make df.except work for UDT

2017-02-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16765 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72295/ Test PASSed. ---

[GitHub] spark issue #16765: [SPARK-19425][SQL] Make df.except work for UDT

2017-02-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16765 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16765: [SPARK-19425][SQL] Make df.except work for UDT

2017-02-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16765 **[Test build #72295 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72295/testReport)** for PR 16765 at commit [`ac3c3bf`](https://github.com/apache/spark/commit/a

[GitHub] spark issue #16779: [SPARK-19437] Rectify spark executor id in HeartbeatRece...

2017-02-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16779 **[Test build #72297 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72297/testReport)** for PR 16779 at commit [`a9bc3f4`](https://github.com/apache/spark/commit/a9

[GitHub] spark issue #16740: [SPARK-19400][ML] Allow GLM to handle intercept only mod...

2017-02-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16740 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72296/ Test PASSed. ---

[GitHub] spark issue #16740: [SPARK-19400][ML] Allow GLM to handle intercept only mod...

2017-02-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16740 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16779: [SPARK-19437] Rectify spark executor id in HeartbeatRece...

2017-02-02 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/16779 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #16740: [SPARK-19400][ML] Allow GLM to handle intercept only mod...

2017-02-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16740 **[Test build #72296 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72296/testReport)** for PR 16740 at commit [`3a0a2af`](https://github.com/apache/spark/commit/3

[GitHub] spark issue #16739: [SPARK-19399][SPARKR] Add R coalesce API for DataFrame a...

2017-02-02 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16739 : ) This might be caused by the optimizer rule `CollapseRepartition`. Can you output the plan by `explain(true)`? --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark issue #15435: [SPARK-17139][ML] Add model summary for MultinomialLogis...

2017-02-02 Thread WeichenXu123
Github user WeichenXu123 commented on the issue: https://github.com/apache/spark/pull/15435 @sethah If I merge the MulticlassLogisticRegressionSummary into LogisticRegressionSummary, then, according to the hierarchy currently designed, it became: class LogisticRe

[GitHub] spark pull request #16740: [SPARK-19400][ML] Allow GLM to handle intercept o...

2017-02-02 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/16740#discussion_r99269075 --- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/GeneralizedLinearRegressionSuite.scala --- @@ -743,6 +743,55 @@ class GeneralizedLinearRegressi

[GitHub] spark issue #16740: [SPARK-19400][ML] Allow GLM to handle intercept only mod...

2017-02-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16740 **[Test build #72296 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72296/testReport)** for PR 16740 at commit [`3a0a2af`](https://github.com/apache/spark/commit/3a

[GitHub] spark pull request #16740: [SPARK-19400][ML] Allow GLM to handle intercept o...

2017-02-02 Thread actuaryzhang
Github user actuaryzhang commented on a diff in the pull request: https://github.com/apache/spark/pull/16740#discussion_r99269006 --- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/GeneralizedLinearRegressionSuite.scala --- @@ -743,6 +743,55 @@ class GeneralizedLinearRe

[GitHub] spark issue #16779: [SPARK-19437] Rectify spark executor id in HeartbeatRece...

2017-02-02 Thread jinxing64
Github user jinxing64 commented on the issue: https://github.com/apache/spark/pull/16779 @zsxwing Thanks a lot for reviewing this. Not sure why the test doesn't start automatically. --- If your project is set up for it, you can reply to this email and have your reply appear on G

[GitHub] spark pull request #16740: [SPARK-19400][ML] Allow GLM to handle intercept o...

2017-02-02 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/16740#discussion_r99268773 --- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/GeneralizedLinearRegressionSuite.scala --- @@ -743,6 +743,55 @@ class GeneralizedLinearRegressi

[GitHub] spark issue #15435: [SPARK-17139][ML] Add model summary for MultinomialLogis...

2017-02-02 Thread WeichenXu123
Github user WeichenXu123 commented on the issue: https://github.com/apache/spark/pull/15435 sethah About this issue: Why is there a one-to-one overlap between MulticlassClassificationSummary and LogisticRegressionSummary, and MulticlassLogisticRegressionSummary inherits

[GitHub] spark issue #16740: [SPARK-19400][ML] Allow GLM to handle intercept only mod...

2017-02-02 Thread actuaryzhang
Github user actuaryzhang commented on the issue: https://github.com/apache/spark/pull/16740 @srowen would you please take a look and merge this if all is good? Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If yo

[GitHub] spark pull request #16780: [SPARK-19438] Both reading and updating executorD...

2017-02-02 Thread jinxing64
Github user jinxing64 closed the pull request at: https://github.com/apache/spark/pull/16780 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #16780: [SPARK-19438] Both reading and updating executorDataMap ...

2017-02-02 Thread jinxing64
Github user jinxing64 commented on the issue: https://github.com/apache/spark/pull/16780 Thanks a lot for looking into this~ @zsxwing You are right. My understanding about this is incorrect. `CoarseGrainedSchedulerBackend: DriverEndpoint` is a `ThreadSafeRpcEndpoint`, thus concur

[GitHub] spark issue #15264: [SPARK-17477][SQL] SparkSQL cannot handle schema evoluti...

2017-02-02 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15264 Yeap, I will try to get that back after finishing up few issues I am currently working on. I just realised that it'd take a bit of time for me to proceed (as I noticed we need a more careful tou

[GitHub] spark pull request #16607: [SPARK-19247][ML] Save large word2vec models

2017-02-02 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/16607#discussion_r99263532 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/Word2Vec.scala --- @@ -302,16 +302,36 @@ class Word2VecModel private[ml] ( @Since("1.6.0")

[GitHub] spark pull request #16607: [SPARK-19247][ML] Save large word2vec models

2017-02-02 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/16607#discussion_r99263525 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/Word2Vec.scala --- @@ -320,14 +340,29 @@ object Word2VecModel extends MLReadable[Word2VecModel]

[GitHub] spark pull request #16607: [SPARK-19247][ML] Save large word2vec models

2017-02-02 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/16607#discussion_r99259617 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/Word2Vec.scala --- @@ -18,10 +18,9 @@ package org.apache.spark.ml.feature import

[GitHub] spark issue #16686: [SPARK-18682][SS] Batch Source for Kafka

2017-02-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16686 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72294/ Test PASSed. ---

[GitHub] spark issue #16686: [SPARK-18682][SS] Batch Source for Kafka

2017-02-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16686 **[Test build #72294 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72294/testReport)** for PR 16686 at commit [`5b48fc6`](https://github.com/apache/spark/commit/5

[GitHub] spark issue #16686: [SPARK-18682][SS] Batch Source for Kafka

2017-02-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16686 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark pull request #16699: [SPARK-18710][ML] Add offset in GLM

2017-02-02 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/16699#discussion_r99263111 --- Diff: mllib/src/test/scala/org/apache/spark/ml/regression/GeneralizedLinearRegressionSuite.scala --- @@ -743,6 +743,84 @@ class GeneralizedLinearRegressi

[GitHub] spark issue #14702: [SPARK-15694] Implement ScriptTransformation in sql/core...

2017-02-02 Thread tejasapatil
Github user tejasapatil commented on the issue: https://github.com/apache/spark/pull/14702 can anyone please review this PR ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enab

[GitHub] spark issue #16740: [SPARK-19400][ML] Allow GLM to handle intercept only mod...

2017-02-02 Thread sethah
Github user sethah commented on the issue: https://github.com/apache/spark/pull/16740 Ok, yeah, let's go with this fix now then - seems both R and statsmodels fit to compute the null model. Thanks for following up on that! --- If your project is set up for it, you can reply to this e

[GitHub] spark issue #16765: [SPARK-19425][SQL] Make df.except work for UDT

2017-02-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16765 **[Test build #72295 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72295/testReport)** for PR 16765 at commit [`ac3c3bf`](https://github.com/apache/spark/commit/ac

[GitHub] spark pull request #16733: [SPARK-19392][SQL] Fix the bug that throws an exc...

2017-02-02 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/16733#discussion_r99261637 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/jdbc/OracleDialect.scala --- @@ -29,7 +29,12 @@ private case object OracleDialect extends JdbcDialect {

[GitHub] spark issue #16686: [SPARK-18682][SS] Batch Source for Kafka

2017-02-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16686 **[Test build #72294 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72294/testReport)** for PR 16686 at commit [`5b48fc6`](https://github.com/apache/spark/commit/5b

[GitHub] spark issue #16607: [SPARK-19247][ML] Save large word2vec models

2017-02-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16607 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16607: [SPARK-19247][ML] Save large word2vec models

2017-02-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16607 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72293/ Test FAILed. ---

[GitHub] spark issue #16607: [SPARK-19247][ML] Save large word2vec models

2017-02-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16607 **[Test build #72293 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72293/testReport)** for PR 16607 at commit [`9b5e928`](https://github.com/apache/spark/commit/9

[GitHub] spark issue #16607: [SPARK-19247][ML] Save large word2vec models

2017-02-02 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16607 **[Test build #72293 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/72293/testReport)** for PR 16607 at commit [`9b5e928`](https://github.com/apache/spark/commit/9b

[GitHub] spark issue #16607: [SPARK-19247][ML] Save large word2vec models

2017-02-02 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/16607 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if t

[GitHub] spark issue #16607: [SPARK-19247][ML] Save large word2vec models

2017-02-02 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/16607 Sorry for the delay; will take a look now! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this featur

[GitHub] spark issue #14412: [SPARK-15355] [CORE] Proactive block replication

2017-02-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14412 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14412: [SPARK-15355] [CORE] Proactive block replication

2017-02-02 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14412 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72291/ Test PASSed. ---

  1   2   3   4   >