[GitHub] spark issue #15249: [SPARK-17675] [CORE] Expand Blacklist for TaskSets

2016-10-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15249 **[Test build #66394 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66394/consoleFull)** for PR 15249 at commit

[GitHub] spark issue #15351: [SPARK-17612][SQL][branch-2.0] Support `DESCRIBE table P...

2016-10-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/15351 Hi, @hvanhovell . Could you give some optinion about this backport when you have sometime? --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark issue #15285: [SPARK-17711] Compress rolled executor log

2016-10-05 Thread loneknightpy
Github user loneknightpy commented on the issue: https://github.com/apache/spark/pull/15285 ping @tdas --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #15318: [SPARK-17750][SQL] Fix CREATE VIEW with INTERVAL arithme...

2016-10-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15318 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66393/ Test PASSed. ---

[GitHub] spark issue #15318: [SPARK-17750][SQL] Fix CREATE VIEW with INTERVAL arithme...

2016-10-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15318 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15357: [SPARK-17328][SQL] Fix NPE with EXPLAIN DESCRIBE TABLE

2016-10-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/15357 Thank you, @hvanhovell ! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #15318: [SPARK-17750][SQL] Fix CREATE VIEW with INTERVAL arithme...

2016-10-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15318 **[Test build #66393 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66393/consoleFull)** for PR 15318 at commit

[GitHub] spark issue #15102: [SPARK-17346][SQL] Add Kafka source for Structured Strea...

2016-10-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15102 **[Test build #66397 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66397/consoleFull)** for PR 15102 at commit

[GitHub] spark issue #15357: [SPARK-17328][SQL] Fix NPE with EXPLAIN DESCRIBE TABLE

2016-10-05 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15357 LGTM - merging to master --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #15102: [SPARK-17346][SQL] Add Kafka source for Structure...

2016-10-05 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/15102#discussion_r82030620 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaSource.scala --- @@ -0,0 +1,396 @@ +/* + * Licensed to the

[GitHub] spark pull request #15102: [SPARK-17346][SQL] Add Kafka source for Structure...

2016-10-05 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/15102#discussion_r82030589 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaSource.scala --- @@ -0,0 +1,396 @@ +/* + * Licensed to the

[GitHub] spark pull request #15102: [SPARK-17346][SQL] Add Kafka source for Structure...

2016-10-05 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/15102#discussion_r82030575 --- Diff: external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaSourceSuite.scala --- @@ -0,0 +1,422 @@ +/* + * Licensed to the

[GitHub] spark pull request #15102: [SPARK-17346][SQL] Add Kafka source for Structure...

2016-10-05 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/15102#discussion_r82030607 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaSource.scala --- @@ -0,0 +1,396 @@ +/* + * Licensed to the

[GitHub] spark pull request #15102: [SPARK-17346][SQL] Add Kafka source for Structure...

2016-10-05 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/15102#discussion_r82030626 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaSource.scala --- @@ -0,0 +1,396 @@ +/* + * Licensed to the

[GitHub] spark pull request #15102: [SPARK-17346][SQL] Add Kafka source for Structure...

2016-10-05 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/15102#discussion_r82030551 --- Diff: docs/structured-streaming-kafka-integration.md --- @@ -0,0 +1,239 @@ +--- +layout: global +title: Structured Streaming + Kafka

[GitHub] spark pull request #15102: [SPARK-17346][SQL] Add Kafka source for Structure...

2016-10-05 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/15102#discussion_r82030561 --- Diff: docs/structured-streaming-kafka-integration.md --- @@ -0,0 +1,239 @@ +--- +layout: global +title: Structured Streaming + Kafka

[GitHub] spark pull request #15102: [SPARK-17346][SQL] Add Kafka source for Structure...

2016-10-05 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/15102#discussion_r82030538 --- Diff: docs/structured-streaming-kafka-integration.md --- @@ -0,0 +1,239 @@ +--- +layout: global +title: Structured Streaming + Kafka

[GitHub] spark issue #15102: [SPARK-17346][SQL] Add Kafka source for Structured Strea...

2016-10-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15102 **[Test build #66396 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66396/consoleFull)** for PR 15102 at commit

[GitHub] spark issue #15148: [SPARK-5992][ML] Locality Sensitive Hashing

2016-10-05 Thread Yunni
Github user Yunni commented on the issue: https://github.com/apache/spark/pull/15148 @jkbradley Take you time for the code review. :) I will be working on the open dataset testing at the same time. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request #15148: [SPARK-5992][ML] Locality Sensitive Hashing

2016-10-05 Thread Yunni
Github user Yunni commented on a diff in the pull request: https://github.com/apache/spark/pull/15148#discussion_r82027195 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/LSH.scala --- @@ -0,0 +1,334 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] spark issue #15148: [SPARK-5992][ML] Locality Sensitive Hashing

2016-10-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15148 **[Test build #66395 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66395/consoleFull)** for PR 15148 at commit

[GitHub] spark pull request #15148: [SPARK-5992][ML] Locality Sensitive Hashing

2016-10-05 Thread Yunni
Github user Yunni commented on a diff in the pull request: https://github.com/apache/spark/pull/15148#discussion_r82027114 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/LSH.scala --- @@ -0,0 +1,334 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] spark pull request #15148: [SPARK-5992][ML] Locality Sensitive Hashing

2016-10-05 Thread Yunni
Github user Yunni commented on a diff in the pull request: https://github.com/apache/spark/pull/15148#discussion_r82027003 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/LSH.scala --- @@ -0,0 +1,334 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] spark pull request #15148: [SPARK-5992][ML] Locality Sensitive Hashing

2016-10-05 Thread Yunni
Github user Yunni commented on a diff in the pull request: https://github.com/apache/spark/pull/15148#discussion_r82027065 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/LSH.scala --- @@ -0,0 +1,334 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] spark pull request #15148: [SPARK-5992][ML] Locality Sensitive Hashing

2016-10-05 Thread Yunni
Github user Yunni commented on a diff in the pull request: https://github.com/apache/spark/pull/15148#discussion_r82027088 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/LSH.scala --- @@ -0,0 +1,334 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] spark pull request #15148: [SPARK-5992][ML] Locality Sensitive Hashing

2016-10-05 Thread Yunni
Github user Yunni commented on a diff in the pull request: https://github.com/apache/spark/pull/15148#discussion_r82026834 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/LSH.scala --- @@ -0,0 +1,334 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] spark pull request #15335: [SPARK-17769][Core][Scheduler]Some FetchFailure r...

2016-10-05 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/15335#discussion_r82010161 --- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala --- @@ -1255,27 +1255,46 @@ class DAGScheduler( s"longer

[GitHub] spark pull request #15335: [SPARK-17769][Core][Scheduler]Some FetchFailure r...

2016-10-05 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/15335#discussion_r82021050 --- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala --- @@ -1255,27 +1255,46 @@ class DAGScheduler( s"longer

[GitHub] spark pull request #15148: [SPARK-5992][ML] Locality Sensitive Hashing

2016-10-05 Thread Yunni
Github user Yunni commented on a diff in the pull request: https://github.com/apache/spark/pull/15148#discussion_r82018962 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/LSH.scala --- @@ -0,0 +1,338 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] spark issue #15249: [SPARK-17675] [CORE] Expand Blacklist for TaskSets

2016-10-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15249 **[Test build #66394 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66394/consoleFull)** for PR 15249 at commit

[GitHub] spark issue #15249: [SPARK-17675] [CORE] Expand Blacklist for TaskSets

2016-10-05 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/15249 I forgot to add that I had turned off blacklisting by default, I agree with your suggestion Kay. I pushed another commit which updated the docs as well. There are some other small style things and

[GitHub] spark pull request #15249: [SPARK-17675] [CORE] Expand Blacklist for TaskSet...

2016-10-05 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/15249#discussion_r82007301 --- Diff: core/src/test/scala/org/apache/spark/scheduler/TaskSetManagerSuite.scala --- @@ -809,32 +816,65 @@ class TaskSetManagerSuite extends SparkFunSuite

[GitHub] spark issue #15009: [SPARK-17443][SPARK-11035] Stop Spark Application if lau...

2016-10-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15009 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15009: [SPARK-17443][SPARK-11035] Stop Spark Application if lau...

2016-10-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15009 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66392/ Test FAILed. ---

[GitHub] spark issue #15009: [SPARK-17443][SPARK-11035] Stop Spark Application if lau...

2016-10-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15009 **[Test build #66392 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66392/consoleFull)** for PR 15009 at commit

[GitHub] spark issue #15318: [SPARK-17750][SQL] Fix CREATE VIEW with INTERVAL arithme...

2016-10-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/15318 Thank you, @gatorsmile . --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #15318: [SPARK-17750][SQL] Fix CREATE VIEW with INTERVAL arithme...

2016-10-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15318 **[Test build #66393 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66393/consoleFull)** for PR 15318 at commit

[GitHub] spark issue #15318: [SPARK-17750][SQL] Fix CREATE VIEW with INTERVAL arithme...

2016-10-05 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15318 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #15318: [SPARK-17750][SQL] Fix CREATE VIEW with INTERVAL arithme...

2016-10-05 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15318 Will make a try. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #14531: [SPARK-17353] [SPARK-16943] [SPARK-16942] [SQL] Fix mult...

2016-10-05 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/14531 @cloud-fan Hive does not copy the table properties in CREATE TABLE LIKE --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request #15358: [SPARK-17783] [SQL] Hide Credentials in CREATE an...

2016-10-05 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/15358#discussion_r82001717 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/interface.scala --- @@ -52,9 +52,15 @@ case class CatalogStorageFormat(

[GitHub] spark issue #15358: [SPARK-17783] [SQL] Hide Credentials in CREATE and DESC ...

2016-10-05 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15358 @rxin Sorry, I did not finish the PR description last night. The connection is broken in the train. Will fix it soon. --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request #15249: [SPARK-17675] [CORE] Expand Blacklist for TaskSet...

2016-10-05 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/15249#discussion_r81999277 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala --- @@ -592,34 +589,54 @@ private[spark] class TaskSetManager( *

[GitHub] spark issue #15044: [SQL][SPARK-17490] Optimize SerializeFromObject() for a ...

2016-10-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15044 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66391/ Test PASSed. ---

[GitHub] spark issue #15044: [SQL][SPARK-17490] Optimize SerializeFromObject() for a ...

2016-10-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15044 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15044: [SQL][SPARK-17490] Optimize SerializeFromObject() for a ...

2016-10-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15044 **[Test build #66391 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66391/consoleFull)** for PR 15044 at commit

[GitHub] spark issue #15246: [MINOR][SQL] Use resource path for test_script.sh

2016-10-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15246 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66390/ Test FAILed. ---

[GitHub] spark issue #15246: [MINOR][SQL] Use resource path for test_script.sh

2016-10-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15246 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15246: [MINOR][SQL] Use resource path for test_script.sh

2016-10-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15246 **[Test build #66390 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66390/consoleFull)** for PR 15246 at commit

[GitHub] spark issue #15359: [Minor][ML] Avoid 2D array flatten in NB training.

2016-10-05 Thread sethah
Github user sethah commented on the issue: https://github.com/apache/spark/pull/15359 I like @srowen's suggestion. Otherwise LGTM. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #15359: [Minor][ML] Avoid 2D array flatten in NB training...

2016-10-05 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/15359#discussion_r81990605 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/NaiveBayes.scala --- @@ -177,7 +177,7 @@ class NaiveBayes @Since("1.5.0") ( val

[GitHub] spark issue #14567: [SPARK-16992][PYSPARK] Python Pep8 formatting and import...

2016-10-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14567 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66389/ Test PASSed. ---

[GitHub] spark issue #14567: [SPARK-16992][PYSPARK] Python Pep8 formatting and import...

2016-10-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14567 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14963: [SPARK-16992][PYSPARK] Virtualenv for Pylint and pep8 in...

2016-10-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14963 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66387/ Test PASSed. ---

[GitHub] spark issue #14567: [SPARK-16992][PYSPARK] Python Pep8 formatting and import...

2016-10-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14567 **[Test build #66389 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66389/consoleFull)** for PR 14567 at commit

[GitHub] spark issue #14963: [SPARK-16992][PYSPARK] Virtualenv for Pylint and pep8 in...

2016-10-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14963 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14963: [SPARK-16992][PYSPARK] Virtualenv for Pylint and pep8 in...

2016-10-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14963 **[Test build #66387 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66387/consoleFull)** for PR 14963 at commit

[GitHub] spark issue #15342: [SPARK-11560] [SPARK-3261] [MLLIB] Optimize KMeans imple...

2016-10-05 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15342 This is what SPARK-3261 is about. It's a corner case to be sure. To me it seems like having duplicate centroids is worse because the model loses some of its meaning. Points may arbitrarily assign to

[GitHub] spark issue #15342: [SPARK-11560] [SPARK-3261] [MLLIB] Optimize KMeans imple...

2016-10-05 Thread yanboliang
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/15342 I'm more prefer to maintain the original logic that keep ```model.clusterCenters.length``` equal to ```k```. Was there some discussion as to make this change? I checked popular Python

[GitHub] spark issue #15009: [SPARK-17443][SPARK-11035] Stop Spark Application if lau...

2016-10-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15009 **[Test build #66392 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66392/consoleFull)** for PR 15009 at commit

[GitHub] spark issue #15343: [SPARK-8480] [CORE] [PYSPARK] [SPARKR] Add setName for D...

2016-10-05 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15343 @davies OK I think I'm 'fixing' this in the wrong place. RDD.setName() will change the name displayed in the storage tab, but, this new Dataset.setName() won't. You're right that it's always the

[GitHub] spark issue #15009: [SPARK-17443][SPARK-11035] Stop Spark Application if lau...

2016-10-05 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/15009 test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #15009: [SPARK-17443][SPARK-11035] Stop Spark Application if lau...

2016-10-05 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/15009 add to whitelist --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #15314: [SPARK-17747][ML] WeightCol support non-double datatypes

2016-10-05 Thread zhengruifeng
Github user zhengruifeng commented on the issue: https://github.com/apache/spark/pull/15314 @sethah I have add weighCol datatype checking into `checkNumericTypes`. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #15246: [MINOR][SQL] Use resource path for test_script.sh

2016-10-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15246 **[Test build #66390 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66390/consoleFull)** for PR 15246 at commit

[GitHub] spark issue #15044: [WIP][SQL][SPARK-17490] Optimize SerializeFromObject() f...

2016-10-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15044 **[Test build #66391 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66391/consoleFull)** for PR 15044 at commit

[GitHub] spark issue #15246: [MINOR][SQL] Use resource path for test_script.sh

2016-10-05 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15246 Jenkins retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #15246: [MINOR][SQL] Use resource path for test_script.sh

2016-10-05 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15246 Jenkins add to whitelist --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #14912: [SPARK-17357][SQL] Fix current predicate pushdown

2016-10-05 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/14912 ping @cloud-fan @hvanhovell @srinathshankar Can you take a look? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark issue #14918: [SPARK-17360][PYSPARK] Support generator in createDataFr...

2016-10-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14918 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66388/ Test PASSed. ---

[GitHub] spark issue #14918: [SPARK-17360][PYSPARK] Support generator in createDataFr...

2016-10-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14918 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14918: [SPARK-17360][PYSPARK] Support generator in createDataFr...

2016-10-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14918 **[Test build #66388 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66388/consoleFull)** for PR 14918 at commit

[GitHub] spark issue #14567: [SPARK-16992][PYSPARK] Python Pep8 formatting and import...

2016-10-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14567 **[Test build #66389 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66389/consoleFull)** for PR 14567 at commit

[GitHub] spark issue #14963: [SPARK-16992][PYSPARK] Virtualenv for Pylint and pep8 in...

2016-10-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14963 **[Test build #66387 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66387/consoleFull)** for PR 14963 at commit

[GitHub] spark issue #14918: [SPARK-17360][PYSPARK] Support generator in createDataFr...

2016-10-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14918 **[Test build #66388 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66388/consoleFull)** for PR 14918 at commit

[GitHub] spark issue #15361: [SPARK-17765][SQL] Support for writing out user-defined ...

2016-10-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15361 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15361: [SPARK-17765][SQL] Support for writing out user-defined ...

2016-10-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15361 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66386/ Test PASSed. ---

[GitHub] spark issue #15361: [SPARK-17765][SQL] Support for writing out user-defined ...

2016-10-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15361 **[Test build #66386 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66386/consoleFull)** for PR 15361 at commit

[GitHub] spark issue #15346: [SPARK-17741][SQL] Grammar to parse top level and nested...

2016-10-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15346 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66385/ Test PASSed. ---

[GitHub] spark issue #15346: [SPARK-17741][SQL] Grammar to parse top level and nested...

2016-10-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15346 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15346: [SPARK-17741][SQL] Grammar to parse top level and nested...

2016-10-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15346 **[Test build #66385 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66385/consoleFull)** for PR 15346 at commit

[GitHub] spark issue #15361: [SPARK-17765][SQL] Support for writing out user-defined ...

2016-10-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15361 **[Test build #66386 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66386/consoleFull)** for PR 15361 at commit

[GitHub] spark issue #15361: [SPARK-17765][SQL] Support for writing out user-defined ...

2016-10-05 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15361 @yhuai and @liancheng Do you mind if I ask to review this please? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request #15361: [SPARK-17765][SQL] Support for writing out user-d...

2016-10-05 Thread HyukjinKwon
GitHub user HyukjinKwon opened a pull request: https://github.com/apache/spark/pull/15361 [SPARK-17765][SQL] Support for writing out user-defined type in ORC datasource ## What changes were proposed in this pull request? `OrcStruct` is being created based on string

[GitHub] spark issue #15258: [SPARK-17689][SQL][STREAMING] added excludeFiles option ...

2016-10-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15258 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15258: [SPARK-17689][SQL][STREAMING] added excludeFiles option ...

2016-10-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15258 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66384/ Test PASSed. ---

[GitHub] spark issue #15258: [SPARK-17689][SQL][STREAMING] added excludeFiles option ...

2016-10-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15258 **[Test build #66384 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66384/consoleFull)** for PR 15258 at commit

[GitHub] spark issue #15354: [SPARK-17764][SQL] Add `to_json` supporting to convert n...

2016-10-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15354 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15354: [SPARK-17764][SQL] Add `to_json` supporting to convert n...

2016-10-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15354 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66383/ Test PASSed. ---

[GitHub] spark issue #15354: [SPARK-17764][SQL] Add `to_json` supporting to convert n...

2016-10-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15354 **[Test build #66383 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66383/consoleFull)** for PR 15354 at commit

[GitHub] spark issue #15072: [SPARK-17123][SQL] Use type-widened encoder for DataFram...

2016-10-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15072 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/66382/ Test PASSed. ---

[GitHub] spark issue #15072: [SPARK-17123][SQL] Use type-widened encoder for DataFram...

2016-10-05 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15072 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15072: [SPARK-17123][SQL] Use type-widened encoder for DataFram...

2016-10-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15072 **[Test build #66382 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66382/consoleFull)** for PR 15072 at commit

[GitHub] spark pull request #15346: [SPARK-17741][SQL] Grammar to parse top level and...

2016-10-05 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/15346#discussion_r81937555 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/SparkSqlParserSuite.scala --- @@ -67,9 +86,133 @@ class SparkSqlParserSuite extends

[GitHub] spark issue #15346: [SPARK-17741][SQL] Grammar to parse top level and nested...

2016-10-05 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/15346 @hvanhovell Thank you for your suggestion! I've addressed your comment and amended the testcases. Thanks! --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request #15346: [SPARK-17741][SQL] Grammar to parse top level and...

2016-10-05 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/15346#discussion_r81936312 --- Diff: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 --- @@ -593,6 +593,14 @@ colTypeList ;

[GitHub] spark issue #15341: [SPARK-17768] [CORE] Small (Sum,Count,Mean)Evaluator pro...

2016-10-05 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/15341 Possibly @MLnick would have some thoughts on this one. I know there's a lot going on here but mostly it's deletion and comments. The math changes are probably best proven by the unit tests. I think

[GitHub] spark issue #15346: [SPARK-17741][SQL] Grammar to parse top level and nested...

2016-10-05 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15346 **[Test build #66385 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/66385/consoleFull)** for PR 15346 at commit

[GitHub] spark issue #15324: [SPARK-16872][ML] Gaussian Naive Bayes Classifier

2016-10-05 Thread zhengruifeng
Github user zhengruifeng commented on the issue: https://github.com/apache/spark/pull/15324 I tend to make GaussianNB as a special `modelType` option in current NB. However, there are significant differences: 1, 'theta' matrix is used to store means 2, extra `sigma` matrix

[GitHub] spark pull request #15102: [SPARK-17346][SQL] Add Kafka source for Structure...

2016-10-05 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/15102#discussion_r81930531 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaSource.scala --- @@ -0,0 +1,396 @@ +/* + * Licensed to the Apache

<    1   2   3   4   5   6   >