[GitHub] spark issue #15441: [SPARK-4411] [Web UI] Add "kill" link for jobs in the UI

2016-10-21 Thread ajbozarth
Github user ajbozarth commented on the issue: https://github.com/apache/spark/pull/15441 @srowen @tgravescs any chance you'd be able to take a look at this before spark summit? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as w

[GitHub] spark issue #15590: [SPARK-17949][SQL] A Java object based aggregate operato...

2016-10-21 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15590 Can you change the tile to "JVM object" rather than "Java object"? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request #15590: [SPARK-17949][SQL] A Java object based aggregate ...

2016-10-21 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/15590#discussion_r84553296 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/ObjectHashAggregateExec.scala --- @@ -0,0 +1,125 @@ +/* + * Licensed to the

[GitHub] spark pull request #15590: [SPARK-17949][SQL] A Java object based aggregate ...

2016-10-21 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/15590#discussion_r84553255 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/ObjectHashAggregateExec.scala --- @@ -0,0 +1,125 @@ +/* + * Licensed to the

[GitHub] spark pull request #15590: [SPARK-17949][SQL] A Java object based aggregate ...

2016-10-21 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/15590#discussion_r84553559 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/ObjectAggregationIterator.scala --- @@ -0,0 +1,323 @@ +/* + * Licensed to th

[GitHub] spark issue #15515: [SPARK-17970][SQL][WIP] store partition spec in metastor...

2016-10-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15515 **[Test build #67357 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67357/consoleFull)** for PR 15515 at commit [`1b73b7b`](https://github.com/apache/spark/commit/1

[GitHub] spark pull request #15591: [SPARK-17922] [SQL] ClassCastException java.lang....

2016-10-21 Thread AbhiMadav
GitHub user AbhiMadav opened a pull request: https://github.com/apache/spark/pull/15591 [SPARK-17922] [SQL] ClassCastException java.lang.ClassCastException: org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIterator cannot be cast to org.apache.spark.sql.catalyst.express

[GitHub] spark issue #15591: [SPARK-17922] [SQL] ClassCastException java.lang.ClassCa...

2016-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15591 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feat

[GitHub] spark issue #15581: [SPARK-18044][STREAMING] FileStreamSource should not inf...

2016-10-21 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/15581 LGTM. Merging to master and 2.0. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature en

[GitHub] spark issue #15581: [SPARK-18044][STREAMING] FileStreamSource should not inf...

2016-10-21 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/15581 @cloud-fan there are conflicts with 2.0. Could you submit another PR for that? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If you

[GitHub] spark pull request #15581: [SPARK-18044][STREAMING] FileStreamSource should ...

2016-10-21 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15581 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #15515: [SPARK-17970][SQL][WIP] store partition spec in metastor...

2016-10-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15515 **[Test build #67358 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67358/consoleFull)** for PR 15515 at commit [`4da7724`](https://github.com/apache/spark/commit/4

[GitHub] spark issue #14553: [SPARK-16963] [STREAMING] [SQL] Changes to Source trait ...

2016-10-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14553 **[Test build #67350 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67350/consoleFull)** for PR 14553 at commit [`47eee52`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14553: [SPARK-16963] [STREAMING] [SQL] Changes to Source trait ...

2016-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14553 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14553: [SPARK-16963] [STREAMING] [SQL] Changes to Source trait ...

2016-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14553 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67350/ Test FAILed. ---

[GitHub] spark issue #15515: [SPARK-17970][SQL][WIP] store partition spec in metastor...

2016-10-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15515 **[Test build #67359 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67359/consoleFull)** for PR 15515 at commit [`34b1ae6`](https://github.com/apache/spark/commit/3

[GitHub] spark issue #15504: [SPARK-17812][SQL][KAFKA] Assign and specific startingOf...

2016-10-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15504 **[Test build #3369 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3369/consoleFull)** for PR 15504 at commit [`c188461`](https://github.com/apache/spark/commit

[GitHub] spark issue #15504: [SPARK-17812][SQL][KAFKA] Assign and specific startingOf...

2016-10-21 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/15504 LGTM. Merging to master and 2.0. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature en

[GitHub] spark issue #15580: [SPARK-18042][SQL] OutputWriter should expose file path ...

2016-10-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15580 **[Test build #67344 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67344/consoleFull)** for PR 15580 at commit [`d3ddaf7`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15580: [SPARK-18042][SQL] OutputWriter should expose file path ...

2016-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15580 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15580: [SPARK-18042][SQL] OutputWriter should expose file path ...

2016-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15580 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67344/ Test PASSed. ---

[GitHub] spark pull request #15504: [SPARK-17812][SQL][KAFKA] Assign and specific sta...

2016-10-21 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15504 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #14050: [MINOR][EXAMPLES] Window function examples

2016-10-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14050 **[Test build #67360 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67360/consoleFull)** for PR 14050 at commit [`fed1761`](https://github.com/apache/spark/commit/f

[GitHub] spark issue #15539: [SPARK-17994] [SQL] Add back a file status cache for cat...

2016-10-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15539 **[Test build #67346 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67346/consoleFull)** for PR 15539 at commit [`2a96537`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15539: [SPARK-17994] [SQL] Add back a file status cache for cat...

2016-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15539 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15539: [SPARK-17994] [SQL] Add back a file status cache for cat...

2016-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15539 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67346/ Test PASSed. ---

[GitHub] spark pull request #14957: [SPARK-4502][SQL]Support parquet nested struct pr...

2016-10-21 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/14957#discussion_r84422762 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetQuerySuite.scala --- @@ -571,6 +571,37 @@ class ParquetQuerySu

[GitHub] spark pull request #14957: [SPARK-4502][SQL]Support parquet nested struct pr...

2016-10-21 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/14957#discussion_r84422376 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileSourceStrategy.scala --- @@ -126,4 +136,52 @@ object FileSourceStrategy e

[GitHub] spark pull request #14957: [SPARK-4502][SQL]Support parquet nested struct pr...

2016-10-21 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/14957#discussion_r84436528 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileSourceStrategy.scala --- @@ -97,7 +99,15 @@ object FileSourceStrategy ext

[GitHub] spark pull request #14957: [SPARK-4502][SQL]Support parquet nested struct pr...

2016-10-21 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/14957#discussion_r84422353 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileSourceStrategy.scala --- @@ -126,4 +136,52 @@ object FileSourceStrategy e

[GitHub] spark pull request #14957: [SPARK-4502][SQL]Support parquet nested struct pr...

2016-10-21 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/14957#discussion_r84406104 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileSourceStrategy.scala --- @@ -97,7 +99,15 @@ object FileSourceStrategy ext

[GitHub] spark pull request #14957: [SPARK-4502][SQL]Support parquet nested struct pr...

2016-10-21 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/14957#discussion_r84559521 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileSourceStrategy.scala --- @@ -126,4 +136,52 @@ object FileSourceStrategy e

[GitHub] spark pull request #14957: [SPARK-4502][SQL]Support parquet nested struct pr...

2016-10-21 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/14957#discussion_r84558190 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileSourceStrategy.scala --- @@ -126,4 +136,52 @@ object FileSourceStrategy e

[GitHub] spark pull request #14957: [SPARK-4502][SQL]Support parquet nested struct pr...

2016-10-21 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/14957#discussion_r84422606 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -212,6 +212,11 @@ object SQLConf { .booleanConf .cre

[GitHub] spark pull request #14957: [SPARK-4502][SQL]Support parquet nested struct pr...

2016-10-21 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/14957#discussion_r84422346 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileSourceStrategy.scala --- @@ -126,4 +136,52 @@ object FileSourceStrategy e

[GitHub] spark pull request #14957: [SPARK-4502][SQL]Support parquet nested struct pr...

2016-10-21 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/14957#discussion_r84422485 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -212,6 +212,11 @@ object SQLConf { .booleanConf .cre

[GitHub] spark pull request #14957: [SPARK-4502][SQL]Support parquet nested struct pr...

2016-10-21 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/14957#discussion_r84422636 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -661,6 +666,8 @@ private[sql] class SQLConf extends Serializable with

[GitHub] spark pull request #15592: [SPARK-17624][SQL][STREAMING][TEST] Fixed flaky S...

2016-10-21 Thread tdas
GitHub user tdas opened a pull request: https://github.com/apache/spark/pull/15592 [SPARK-17624][SQL][STREAMING][TEST] Fixed flaky StateStoreSuite.maintenance ## What changes were proposed in this pull request? The reason for the flakiness was follows. The test starts the ma

[GitHub] spark issue #15592: [SPARK-17624][SQL][STREAMING][TEST] Fixed flaky StateSto...

2016-10-21 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/15592 @zsxwing Can you take a look? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wish

[GitHub] spark issue #15570: [STREAMING][KAFKA][DOC] clarify kafka settings needed fo...

2016-10-21 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/15570 LGTM. Thanks! Merging to master and 2.0. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature en

[GitHub] spark issue #15592: [SPARK-17624][SQL][STREAMING][TEST] Fixed flaky StateSto...

2016-10-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15592 **[Test build #67361 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67361/consoleFull)** for PR 15592 at commit [`96079fe`](https://github.com/apache/spark/commit/9

[GitHub] spark pull request #15570: [STREAMING][KAFKA][DOC] clarify kafka settings ne...

2016-10-21 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15570 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request #15593: [SPARK-18060][ML] Avoid unnecessary computation f...

2016-10-21 Thread sethah
GitHub user sethah opened a pull request: https://github.com/apache/spark/pull/15593 [SPARK-18060][ML] Avoid unnecessary computation for MLOR ## What changes were proposed in this pull request? Before this patch, the gradient updates for multinomial logistic regression were

[GitHub] spark issue #15593: [SPARK-18060][ML] Avoid unnecessary computation for MLOR

2016-10-21 Thread sethah
Github user sethah commented on the issue: https://github.com/apache/spark/pull/15593 cc @dbtsai This may improve the problems you mentioned you were having on [SPARK-17134](https://issues.apache.org/jira/browse/SPARK-17134) :) --- If your project is set up for it, you can

[GitHub] spark issue #15517: [SPARK-17972][SQL] Build Datasets upon `withCachedData` ...

2016-10-21 Thread liancheng
Github user liancheng commented on the issue: https://github.com/apache/spark/pull/15517 I'm closing this since caching is not the ultimate solution for this problem anyway. Caching is too memory consuming when you, say, computing connected components in an iterative way over a graph

[GitHub] spark pull request #15517: [SPARK-17972][SQL] Build Datasets upon `withCache...

2016-10-21 Thread liancheng
Github user liancheng closed the pull request at: https://github.com/apache/spark/pull/15517 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #14050: [MINOR][EXAMPLES] Window function examples

2016-10-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14050 **[Test build #67360 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67360/consoleFull)** for PR 14050 at commit [`fed1761`](https://github.com/apache/spark/commit/

[GitHub] spark issue #14050: [MINOR][EXAMPLES] Window function examples

2016-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14050 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #14050: [MINOR][EXAMPLES] Window function examples

2016-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14050 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67360/ Test PASSed. ---

[GitHub] spark issue #15593: [SPARK-18060][ML] Avoid unnecessary computation for MLOR

2016-10-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15593 **[Test build #67362 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67362/consoleFull)** for PR 15593 at commit [`07fd150`](https://github.com/apache/spark/commit/0

[GitHub] spark issue #14957: [SPARK-4502][SQL]Support parquet nested struct pruning a...

2016-10-21 Thread davies
Github user davies commented on the issue: https://github.com/apache/spark/pull/14957 Together with this one, we should have a optimizer rule that could 1) extract GetStructField (and others) and push that down closer to the data source, or 2) flatten all the nested field in data sou

[GitHub] spark issue #15515: [SPARK-17970][SQL][WIP] store partition spec in metastor...

2016-10-21 Thread ericl
Github user ericl commented on the issue: https://github.com/apache/spark/pull/15515 @cloud-fan I think this is pretty close once https://github.com/apache/spark/pull/15539 is merged. The only remaining issues I know about are (1) supporting mixed case columns, and (2) optimi

[GitHub] spark pull request #15428: [SPARK-17219][ML] enhanced NaN value handling in ...

2016-10-21 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/15428#discussion_r84561834 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/Bucketizer.scala --- @@ -73,15 +74,51 @@ final class Bucketizer @Since("1.4.0") (@Since("1.4.0")

[GitHub] spark pull request #15428: [SPARK-17219][ML] enhanced NaN value handling in ...

2016-10-21 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/15428#discussion_r84562692 --- Diff: docs/ml-features.md --- @@ -1104,9 +1104,11 @@ for more details on the API. `QuantileDiscretizer` takes a column with continuous features and

[GitHub] spark issue #15515: [SPARK-17970][SQL][WIP] store partition spec in metastor...

2016-10-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15515 **[Test build #67355 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67355/consoleFull)** for PR 15515 at commit [`de6c00d`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15515: [SPARK-17970][SQL][WIP] store partition spec in metastor...

2016-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15515 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15515: [SPARK-17970][SQL][WIP] store partition spec in metastor...

2016-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15515 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67355/ Test FAILed. ---

[GitHub] spark issue #15580: [SPARK-18042][SQL] OutputWriter should expose file path ...

2016-10-21 Thread ericl
Github user ericl commented on the issue: https://github.com/apache/spark/pull/15580 lgtm --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark pull request #15520: [SPARK-13747][SQL]Fix concurrent executions in Fo...

2016-10-21 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/15520#discussion_r84563401 --- Diff: scalastyle-config.xml --- @@ -200,6 +200,7 @@ This file is divided into 3 sections: // scalastyle:off awaitresult Await.result(.

[GitHub] spark pull request #15428: [SPARK-17219][ML] enhanced NaN value handling in ...

2016-10-21 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/15428#discussion_r84563374 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/QuantileDiscretizer.scala --- @@ -100,6 +102,24 @@ final class QuantileDiscretizer @Since("1.6.0

[GitHub] spark pull request #15428: [SPARK-17219][ML] enhanced NaN value handling in ...

2016-10-21 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/15428#discussion_r84563479 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/Bucketizer.scala --- @@ -73,15 +74,51 @@ final class Bucketizer @Since("1.4.0") (@Since("1.4.0")

[GitHub] spark pull request #15428: [SPARK-17219][ML] enhanced NaN value handling in ...

2016-10-21 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/15428#discussion_r84563480 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/Bucketizer.scala --- @@ -73,15 +74,51 @@ final class Bucketizer @Since("1.4.0") (@Since("1.4.0")

[GitHub] spark pull request #15428: [SPARK-17219][ML] enhanced NaN value handling in ...

2016-10-21 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/15428#discussion_r84563237 --- Diff: python/pyspark/ml/feature.py --- @@ -1157,9 +1157,11 @@ class QuantileDiscretizer(JavaEstimator, HasInputCol, HasOutputCol, JavaMLReadab

[GitHub] spark pull request #15428: [SPARK-17219][ML] enhanced NaN value handling in ...

2016-10-21 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/15428#discussion_r84563388 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/QuantileDiscretizer.scala --- @@ -100,6 +102,24 @@ final class QuantileDiscretizer @Since("1.6.0

[GitHub] spark pull request #15428: [SPARK-17219][ML] enhanced NaN value handling in ...

2016-10-21 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/15428#discussion_r84563124 --- Diff: mllib/src/test/scala/org/apache/spark/ml/feature/QuantileDiscretizerSuite.scala --- @@ -82,14 +82,23 @@ class QuantileDiscretizerSuite

[GitHub] spark issue #15515: [SPARK-17970][SQL][WIP] store partition spec in metastor...

2016-10-21 Thread ericl
Github user ericl commented on the issue: https://github.com/apache/spark/pull/15515 Looks like two failing tests in FileStreamSourceSuite --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #15394: [SPARK-17748][ML] One pass solver for Weighted Le...

2016-10-21 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/15394#discussion_r84564333 --- Diff: mllib/src/main/scala/org/apache/spark/ml/optim/NormalEquationSolver.scala --- @@ -0,0 +1,153 @@ +/* + * Licensed to the Apache Software Foun

[GitHub] spark pull request #15394: [SPARK-17748][ML] One pass solver for Weighted Le...

2016-10-21 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/15394#discussion_r84564368 --- Diff: mllib/src/main/scala/org/apache/spark/ml/optim/NormalEquationSolver.scala --- @@ -0,0 +1,153 @@ +/* + * Licensed to the Apache Software Foun

[GitHub] spark pull request #15394: [SPARK-17748][ML] One pass solver for Weighted Le...

2016-10-21 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/15394#discussion_r84564391 --- Diff: mllib/src/main/scala/org/apache/spark/ml/optim/WeightedLeastSquares.scala --- @@ -44,35 +46,52 @@ private[ml] class WeightedLeastSquaresModel(

[GitHub] spark pull request #15394: [SPARK-17748][ML] One pass solver for Weighted Le...

2016-10-21 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/15394#discussion_r84564447 --- Diff: mllib/src/main/scala/org/apache/spark/ml/optim/WeightedLeastSquares.scala --- @@ -85,73 +104,198 @@ private[ml] class WeightedLeastSquares(

[GitHub] spark pull request #15394: [SPARK-17748][ML] One pass solver for Weighted Le...

2016-10-21 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/15394#discussion_r84564414 --- Diff: mllib/src/main/scala/org/apache/spark/ml/optim/WeightedLeastSquares.scala --- @@ -44,35 +46,52 @@ private[ml] class WeightedLeastSquaresModel(

[GitHub] spark issue #15394: [SPARK-17748][ML] One pass solver for Weighted Least Squ...

2016-10-21 Thread sethah
Github user sethah commented on the issue: https://github.com/apache/spark/pull/15394 cc @dbtsai Would be great if you get a chance to look at this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark issue #15580: [SPARK-18042][SQL] OutputWriter should expose file path ...

2016-10-21 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15580 Thanks - I'm going to merge this. I will address the doc and naming comment in the next pr in this series. --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark issue #15394: [SPARK-17748][ML] One pass solver for Weighted Least Squ...

2016-10-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15394 **[Test build #67363 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67363/consoleFull)** for PR 15394 at commit [`28f8c2f`](https://github.com/apache/spark/commit/2

[GitHub] spark pull request #15580: [SPARK-18042][SQL] OutputWriter should expose fil...

2016-10-21 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15580 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #15515: [SPARK-17970][SQL][WIP] store partition spec in metastor...

2016-10-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15515 **[Test build #67358 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67358/consoleFull)** for PR 15515 at commit [`4da7724`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15515: [SPARK-17970][SQL][WIP] store partition spec in metastor...

2016-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15515 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67358/ Test FAILed. ---

[GitHub] spark issue #15515: [SPARK-17970][SQL][WIP] store partition spec in metastor...

2016-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15515 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15515: [SPARK-17970][SQL][WIP] store partition spec in metastor...

2016-10-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15515 **[Test build #67357 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67357/consoleFull)** for PR 15515 at commit [`1b73b7b`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15590: [SPARK-17949][SQL] A Java object based aggregate operato...

2016-10-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15590 **[Test build #67356 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67356/consoleFull)** for PR 15590 at commit [`c8ec18b`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15515: [SPARK-17970][SQL][WIP] store partition spec in metastor...

2016-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15515 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67357/ Test FAILed. ---

[GitHub] spark issue #15515: [SPARK-17970][SQL][WIP] store partition spec in metastor...

2016-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15515 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15590: [SPARK-17949][SQL] A Java object based aggregate operato...

2016-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15590 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67356/ Test PASSed. ---

[GitHub] spark issue #15590: [SPARK-17949][SQL] A Java object based aggregate operato...

2016-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15590 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15593: [SPARK-18060][ML] Avoid unnecessary computation for MLOR

2016-10-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15593 **[Test build #67362 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67362/consoleFull)** for PR 15593 at commit [`07fd150`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15593: [SPARK-18060][ML] Avoid unnecessary computation for MLOR

2016-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15593 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15593: [SPARK-18060][ML] Avoid unnecessary computation for MLOR

2016-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15593 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67362/ Test PASSed. ---

[GitHub] spark issue #15553: [SPARK-18008] [build] Add support for -Dmaven.test.skip=...

2016-10-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15553 **[Test build #67345 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67345/consoleFull)** for PR 15553 at commit [`ec49e8b`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15553: [SPARK-18008] [build] Add support for -Dmaven.test.skip=...

2016-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15553 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67345/ Test FAILed. ---

[GitHub] spark issue #15553: [SPARK-18008] [build] Add support for -Dmaven.test.skip=...

2016-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15553 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15515: [SPARK-17970][SQL][WIP] store partition spec in metastor...

2016-10-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15515 **[Test build #67359 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67359/consoleFull)** for PR 15515 at commit [`34b1ae6`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15515: [SPARK-17970][SQL][WIP] store partition spec in metastor...

2016-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15515 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15515: [SPARK-17970][SQL][WIP] store partition spec in metastor...

2016-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15515 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67359/ Test FAILed. ---

[GitHub] spark issue #15463: [SPARK-17894] [CORE] Ensure uniqueness of TaskSetManager...

2016-10-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15463 **[Test build #67349 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67349/consoleFull)** for PR 15463 at commit [`cd6d240`](https://github.com/apache/spark/commit/

[GitHub] spark issue #15463: [SPARK-17894] [CORE] Ensure uniqueness of TaskSetManager...

2016-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15463 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #15463: [SPARK-17894] [CORE] Ensure uniqueness of TaskSetManager...

2016-10-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15463 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/67349/ Test FAILed. ---

[GitHub] spark pull request #15594: [SPARK-18061][SQL][Security] Spark Thriftserver n...

2016-10-21 Thread cmirash
GitHub user cmirash opened a pull request: https://github.com/apache/spark/pull/15594 [SPARK-18061][SQL][Security] Spark Thriftserver needs to create SPNego principal ## What changes were proposed in this pull request? Spark Thriftserver when running in HTTP mode with Kerberos e

[GitHub] spark pull request #15595: [SPARK-18058][SQL] Comparing column types ignorin...

2016-10-21 Thread CodingCat
GitHub user CodingCat opened a pull request: https://github.com/apache/spark/pull/15595 [SPARK-18058][SQL] Comparing column types ignoring Nullability in BinaryOperator ## What changes were proposed in this pull request? The PR tries to fix [SPARK-18058](https://issues.apa

[GitHub] spark issue #15515: [SPARK-17970][SQL][WIP] store partition spec in metastor...

2016-10-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15515 **[Test build #67365 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67365/consoleFull)** for PR 15515 at commit [`f903243`](https://github.com/apache/spark/commit/f

[GitHub] spark issue #15595: [SPARK-18058][SQL] Comparing column types ignoring Nulla...

2016-10-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15595 **[Test build #67364 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/67364/consoleFull)** for PR 15595 at commit [`c62f84f`](https://github.com/apache/spark/commit/c

<    1   2   3   4   5   >