[GitHub] spark pull request: [SPARK-12010][SQL] Spark JDBC requires support...

2015-12-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10380#issuecomment-166637187 **[Test build #48199 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48199/consoleFull)** for PR 10380 at commit

[GitHub] spark pull request: we should have training and test sets

2015-12-22 Thread XD-DENG
GitHub user XD-DENG opened a pull request: https://github.com/apache/spark/pull/10434 we should have training and test sets I split the parsed data into two sets for training and testing separately. I guess the original author didn't consider this so that everyone can re-produce

[GitHub] spark pull request: [SPARK-12440][Core] - Avoid setCheckpoint warn...

2015-12-22 Thread pierre-borckmans
Github user pierre-borckmans commented on the pull request: https://github.com/apache/spark/pull/10392#issuecomment-166637885 @srowen Thanks. Just for my curiosity, the failing tests were completely unrelated, right? Does that mean master was in an unstable state? Or are the unit

[GitHub] spark pull request: we should have training and test sets

2015-12-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10434#issuecomment-166638083 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-12440][Core] - Avoid setCheckpoint warn...

2015-12-22 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/10392#issuecomment-166640491 Yes, flaky tests. There are so many, some really complex integration tests, that unfortunately something fails half the time --- If your project is set up for it, you

[GitHub] spark pull request: we should have training and test sets

2015-12-22 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/10434#issuecomment-166642164 This isn't quite how we report issues or suggest changes. Please read https://cwiki.apache.org/confluence/display/SPARK/Contributing+to+Spark first. This is

[GitHub] spark pull request: [SPARK-12476][SQL] Implement JdbcRelation#unha...

2015-12-22 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/10427#discussion_r48261569 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JDBCRelation.scala --- @@ -90,6 +90,21 @@ private[sql] case class

[GitHub] spark pull request: [SPARK-7727] [SQL] Avoid inner classes in Rule...

2015-12-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10174#issuecomment-166644466 **[Test build #48200 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48200/consoleFull)** for PR 10174 at commit

[GitHub] spark pull request: [SPARK-12440][Core] - Avoid setCheckpoint warn...

2015-12-22 Thread pierre-borckmans
Github user pierre-borckmans commented on the pull request: https://github.com/apache/spark/pull/10392#issuecomment-166645424 Thanks, good to know. Is there work planned on that matter? Or do you guys just specialise in "seeing that it was a flaky test, just restart them" ? :) ---

[GitHub] spark pull request: [SPARK-12476][SQL] Implement JdbcRelation#unha...

2015-12-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10427#issuecomment-166645536 **[Test build #48201 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48201/consoleFull)** for PR 10427 at commit

[GitHub] spark pull request: [SPARK-12480][SQL] add Hash expression that ca...

2015-12-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10435#issuecomment-166650019 **[Test build #48203 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48203/consoleFull)** for PR 10435 at commit

[GitHub] spark pull request: [SPARK-12480][SQL] add Hash expression that ca...

2015-12-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10435#issuecomment-166650025 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-12480][SQL] add Hash expression that ca...

2015-12-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10435#issuecomment-166650029 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: Minor corrections, i.e. typo fixes and follow ...

2015-12-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10432#issuecomment-166640300 **[Test build #2247 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2247/consoleFull)** for PR 10432 at commit

[GitHub] spark pull request: [SPARK-12311][CORE] Restore previous value of ...

2015-12-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10289#issuecomment-166644123 **[Test build #48202 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48202/consoleFull)** for PR 10289 at commit

[GitHub] spark pull request: we should have training and test sets

2015-12-22 Thread XD-DENG
Github user XD-DENG commented on the pull request: https://github.com/apache/spark/pull/10434#issuecomment-166644803 Hi Owen, thanks for clarification. Sure, understood. I proposed this PR as I was a little bit confused myself given train/test split should be a

[GitHub] spark pull request: [SPARK-12440][Core] - Avoid setCheckpoint warn...

2015-12-22 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/10392#issuecomment-166646614 Eh, I don't think anyone wants flaky tests of course. Search JIRA. They get improved by people improving them -- please have a go if you'd like to help. --- If your

[GitHub] spark pull request: [SPARK-12480][SQL] add Hash expression that ca...

2015-12-22 Thread cloud-fan
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/10435#issuecomment-166646764 cc @yhuai @rxin --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-12480][SQL] add Hash expression that ca...

2015-12-22 Thread cloud-fan
GitHub user cloud-fan opened a pull request: https://github.com/apache/spark/pull/10435 [SPARK-12480][SQL] add Hash expression that can calculate hash value for a group of expressions The hash algorithm is based on hive's bucketing hash function, see

[GitHub] spark pull request: [SPARK-12439][SQL] Fix toCatalystArray and Map...

2015-12-22 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/10391#discussion_r48263973 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/encoders/ExpressionEncoderSuite.scala --- @@ -239,6 +240,30 @@ class

[GitHub] spark pull request: [SPARK-12480][SQL] add Hash expression that ca...

2015-12-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10435#issuecomment-166649675 **[Test build #48203 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48203/consoleFull)** for PR 10435 at commit

[GitHub] spark pull request: [SPARK-12439][SQL] Fix toCatalystArray and Map...

2015-12-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10391#issuecomment-166652431 **[Test build #48198 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48198/consoleFull)** for PR 10391 at commit

[GitHub] spark pull request: Minor corrections, i.e. typo fixes and follow ...

2015-12-22 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/10432#issuecomment-166701181 I've merged this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-12477][SQL] - Tungsten projection fails...

2015-12-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10429#issuecomment-166708237 **[Test build #2249 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2249/consoleFull)** for PR 10429 at commit

[GitHub] spark pull request: [SPARK-12477][SQL] - Tungsten projection fails...

2015-12-22 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/10429#discussion_r48297749 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameComplexTypeSuite.scala --- @@ -43,4 +43,12 @@ class DataFrameComplexTypeSuite extends QueryTest

[GitHub] spark pull request: [SPARK-12471] [Core] Spark daemons will log th...

2015-12-22 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/10422#issuecomment-166734034 LGTM. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request: [SPARK-12471] [Core] Spark daemons will log th...

2015-12-22 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/10422#issuecomment-166734086 Merging in master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-12475][BUILD][test-maven] Upgrade Zinc ...

2015-12-22 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/10426 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-12478][SQL] Bugfix: Dataset fields of p...

2015-12-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10431#issuecomment-166706327 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-12478][SQL] Bugfix: Dataset fields of p...

2015-12-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10431#issuecomment-166706328 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-12462][SQL] Add ExpressionDescription t...

2015-12-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10437#issuecomment-166707341 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-12396][Core]Once driver connect to a ma...

2015-12-22 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/10407#issuecomment-166727624 That is really confusing... @srowen do you know if we're missing something obvious? --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request: [SPARK-2827][GraphX] Add collectDegreeDist to ...

2015-12-22 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/4399#issuecomment-166727716 I'm not sure. retest this please? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-12425][STREAMING] DStream union optimis...

2015-12-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10382#issuecomment-166731783 **[Test build #48214 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48214/consoleFull)** for PR 10382 at commit

[GitHub] spark pull request: [SPARK-12487][Streaming][Document]Add docs for...

2015-12-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10439#issuecomment-166735358 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-12487][Streaming][Document]Add docs for...

2015-12-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10439#issuecomment-166735260 **[Test build #48213 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48213/consoleFull)** for PR 10439 at commit

[GitHub] spark pull request: [SPARK-12487][Streaming][Document]Add docs for...

2015-12-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10439#issuecomment-166735357 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-12457] [SQL] Add ExpressionDescription ...

2015-12-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10418#issuecomment-166697266 **[Test build #48205 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48205/consoleFull)** for PR 10418 at commit

[GitHub] spark pull request: [SPARK-12421][SQL] Fixed copy() method of Gene...

2015-12-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10374#issuecomment-166700379 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-12477][SQL] - Tungsten projection fails...

2015-12-22 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/10429#issuecomment-166700171 cc @nongli can you review this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-12421][SQL] Fixed copy() method of Gene...

2015-12-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10374#issuecomment-166700377 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: Minor corrections, i.e. typo fixes and follow ...

2015-12-22 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/10432 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-12456][SQL] Add ExpressionDescription t...

2015-12-22 Thread xguo27
Github user xguo27 commented on the pull request: https://github.com/apache/spark/pull/10423#issuecomment-166701737 @rxin Great, thanks Reynold! My JIRA id is xguo27. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: [SPARK-12477][SQL] - Tungsten projection fails...

2015-12-22 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/10429#discussion_r48287694 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameComplexTypeSuite.scala --- @@ -43,4 +43,12 @@ class DataFrameComplexTypeSuite extends QueryTest

[GitHub] spark pull request: [SPARK-12396][Core]Once driver connect to a ma...

2015-12-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10407#issuecomment-166707858 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-12396][Core]Once driver connect to a ma...

2015-12-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10407#issuecomment-166707852 **[Test build #48208 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48208/consoleFull)** for PR 10407 at commit

[GitHub] spark pull request: [SPARK-12311][CORE] Restore previous value of ...

2015-12-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10289#issuecomment-166709517 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-12311][CORE] Restore previous value of ...

2015-12-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10289#issuecomment-166709431 **[Test build #48202 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48202/consoleFull)** for PR 10289 at commit

[GitHub] spark pull request: [SPARK-12311][CORE] Restore previous value of ...

2015-12-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10289#issuecomment-166709515 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [CORE] Refactoring: Use pattern matching and d...

2015-12-22 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/10433#issuecomment-166701682 Your debate aside, I am not sure whether it makes sense to change the current one. It has virtually zero impact on anything, and as a result it is more or less changing

[GitHub] spark pull request: [SPARK-12321][SQL] JSON format for TreeNode (u...

2015-12-22 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/10311#issuecomment-166713612 @scwf the goal here is to be able to do later analysis of queries that users are running with Spark / Spark SQL. --- If your project is set up for it, you can reply

[GitHub] spark pull request: [SPARK-12102][SQL] Cast a non-nullable struct ...

2015-12-22 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/10156#discussion_r48292189 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/conditionalExpressions.scala --- @@ -91,7 +91,9 @@ trait CaseWhenLike

[GitHub] spark pull request: [SPARK-12486] Worker should kill the executors...

2015-12-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10438#issuecomment-166731620 **[Test build #48212 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48212/consoleFull)** for PR 10438 at commit

[GitHub] spark pull request: [SPARK-12487][Streaming][Document]Add docs for...

2015-12-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10439#issuecomment-166732822 **[Test build #48213 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48213/consoleFull)** for PR 10439 at commit

[GitHub] spark pull request: [SPARK-12475][BUILD][test-maven] Upgrade Zinc ...

2015-12-22 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/10426#issuecomment-166696024 Alright, merging this now. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-12478][SQL] Bugfix: Dataset fields of p...

2015-12-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10431#issuecomment-166706134 **[Test build #48207 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48207/consoleFull)** for PR 10431 at commit

[GitHub] spark pull request: [SPARK-12477][SQL] - Tungsten projection fails...

2015-12-22 Thread nongli
Github user nongli commented on the pull request: https://github.com/apache/spark/pull/10429#issuecomment-166706210 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request: [SPARK-12396][Core]Once driver connect to a ma...

2015-12-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10407#issuecomment-166707399 **[Test build #48208 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48208/consoleFull)** for PR 10407 at commit

[GitHub] spark pull request: [SPARK-12331][ML] R^2 for regression through t...

2015-12-22 Thread iyounus
Github user iyounus commented on a diff in the pull request: https://github.com/apache/spark/pull/10384#discussion_r48287387 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/evaluation/RegressionMetrics.scala --- @@ -23,15 +23,23 @@ import org.apache.spark.Logging

[GitHub] spark pull request: [SPARK-12477][SQL] - Tungsten projection fails...

2015-12-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10429#issuecomment-166709057 **[Test build #2249 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2249/consoleFull)** for PR 10429 at commit

[GitHub] spark pull request: [SPARK-12425][STREAMING] DStream union optimis...

2015-12-22 Thread gpoulin
Github user gpoulin commented on the pull request: https://github.com/apache/spark/pull/10382#issuecomment-166722515 @zsxwing based on your comment, I took the liberty to deduplicate the logic to determine if `UnionRDD` or a `PartitionerAwareUnionRDD` --- If your project is set up

[GitHub] spark pull request: [SPARK-12421][SQL] Fixed copy() method of Gene...

2015-12-22 Thread nongli
Github user nongli commented on the pull request: https://github.com/apache/spark/pull/10374#issuecomment-166731474 The comment in GenericRow explicitly says not to do this. "A row implementation that uses an array of objects as the underlying storage. Note that, while

[GitHub] spark pull request: [SPARK-12456][SQL] Add ExpressionDescription t...

2015-12-22 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/10423 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-12456][SQL] Add ExpressionDescription t...

2015-12-22 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/10423#issuecomment-166700763 What's your JIRA id? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [CORE] Refactoring: Use pattern matching and d...

2015-12-22 Thread markhamstra
Github user markhamstra commented on the pull request: https://github.com/apache/spark/pull/10433#issuecomment-166713847 More substantively, `if...else` generally performs better than pattern matching, and this is a fairly hot code path. --- If your project is set up for it, you can

[GitHub] spark pull request: [SPARK-12477][SQL] - Tungsten projection fails...

2015-12-22 Thread pierre-borckmans
Github user pierre-borckmans commented on a diff in the pull request: https://github.com/apache/spark/pull/10429#discussion_r48297441 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameComplexTypeSuite.scala --- @@ -43,4 +43,12 @@ class DataFrameComplexTypeSuite

[GitHub] spark pull request: [SPARK-12477][SQL] - Tungsten projection fails...

2015-12-22 Thread pierre-borckmans
Github user pierre-borckmans commented on a diff in the pull request: https://github.com/apache/spark/pull/10429#discussion_r48297482 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameComplexTypeSuite.scala --- @@ -43,4 +43,12 @@ class DataFrameComplexTypeSuite

[GitHub] spark pull request: [SPARK-12486] Worker should kill the executors...

2015-12-22 Thread nongli
GitHub user nongli opened a pull request: https://github.com/apache/spark/pull/10438 [SPARK-12486] Worker should kill the executors more forcefully if possible. This patch updates the ExecutorRunner's terminate path to use the new java 8 API to terminate processes more

[GitHub] spark pull request: [SPARK-12487][Streaming][Document]Add docs for...

2015-12-22 Thread zsxwing
GitHub user zsxwing opened a pull request: https://github.com/apache/spark/pull/10439 [SPARK-12487][Streaming][Document]Add docs for Kafka message handler You can merge this pull request into a Git repository by running: $ git pull https://github.com/zsxwing/spark

[GitHub] spark pull request: [SPARK-12396][Core]Once driver connect to a ma...

2015-12-22 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/10407#issuecomment-166729697 Actually the error says that the var is an `AtomicReference` and this does not have a `cancel` method. Indeed, it does in master, after

[GitHub] spark pull request: [SPARK-12471] [Core] Spark daemons will log th...

2015-12-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10422#issuecomment-166733143 **[Test build #48209 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48209/consoleFull)** for PR 10422 at commit

[GitHub] spark pull request: [SPARK-12471] [Core] Spark daemons will log th...

2015-12-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10422#issuecomment-166733369 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-12471] [Core] Spark daemons will log th...

2015-12-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10422#issuecomment-166733367 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-12471] [Core] Spark daemons will log th...

2015-12-22 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/10422 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-12421][SQL] Fixed copy() method of Gene...

2015-12-22 Thread hvanhovell
Github user hvanhovell commented on the pull request: https://github.com/apache/spark/pull/10374#issuecomment-166738733 @nongli I think the comment that comment is mainly meant for developers passing an array into the Row constructor; they should not reuse the array. We currently

[GitHub] spark pull request: [SPARK-12489][Core][SQL][MLib]Fix minor issues...

2015-12-22 Thread zsxwing
GitHub user zsxwing opened a pull request: https://github.com/apache/spark/pull/10440 [SPARK-12489][Core][SQL][MLib]Fix minor issues found by FindBugs Include the following changes: 1. Close `java.sql.Statement` 2. Fix incorrect `asInstanceOf`. 3. Remove unnecessary

[GitHub] spark pull request: [SPARK-12489][Core][SQL][MLib]Fix minor issues...

2015-12-22 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/10440#discussion_r48302657 --- Diff: launcher/src/main/java/org/apache/spark/launcher/LauncherServer.java --- @@ -293,9 +293,7 @@ private String createSecret() { protected

[GitHub] spark pull request: [SPARK-12490] Don't use Javascript for web UI'...

2015-12-22 Thread JoshRosen
GitHub user JoshRosen opened a pull request: https://github.com/apache/spark/pull/10441 [SPARK-12490] Don't use Javascript for web UI's paginated table controls The web UI's paginated table uses Javascript to implement certain navigation controls, such as table sorting and the "go

[GitHub] spark pull request: [SPARK-12490] Don't use Javascript for web UI'...

2015-12-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10441#issuecomment-166746163 **[Test build #48216 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48216/consoleFull)** for PR 10441 at commit

[GitHub] spark pull request: [SPARK-12490] Don't use Javascript for web UI'...

2015-12-22 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/10441#issuecomment-166756541 Note that my exception changes were actually motivated by a case where this error-handling logic swallowed other internal exceptions and only printed "1" as output.

[GitHub] spark pull request: [SPARK-12486] Worker should kill the executors...

2015-12-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10438#issuecomment-166756420 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-12396][Core]Once driver connect to a ma...

2015-12-22 Thread zsxwing
Github user zsxwing commented on the pull request: https://github.com/apache/spark/pull/10407#issuecomment-166758871 I think a better approach is changing `registrationRetryThread.scheduleAtFixedRate` to `registrationRetryThread.schedule`. --- If your project is set up for it, you

[GitHub] spark pull request: [SPARK-12102][SQL] Cast a non-nullable struct ...

2015-12-22 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/10156 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-12311][CORE] Restore previous value of ...

2015-12-22 Thread kiszk
Github user kiszk commented on the pull request: https://github.com/apache/spark/pull/10289#issuecomment-166765714 Can I retest this? Timeout may occur in pyspark. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-12490] Don't use Javascript for web UI'...

2015-12-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10441#issuecomment-166766454 **[Test build #48221 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48221/consoleFull)** for PR 10441 at commit

[GitHub] spark pull request: [SPARK-12476][SQL] Implement JdbcRelation#unha...

2015-12-22 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/10427#discussion_r48313071 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/jdbc/JDBCSuite.scala --- @@ -180,14 +181,23 @@ class JDBCSuite extends SparkFunSuite }

[GitHub] spark pull request: [SPARK-12439][SQL] Fix toCatalystArray and Map...

2015-12-22 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/10391#discussion_r48313326 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/encoders/ExpressionEncoderSuite.scala --- @@ -239,6 +240,30 @@ class

[GitHub] spark pull request: [SPARK-12439][SQL] Fix toCatalystArray and Map...

2015-12-22 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/10391#discussion_r48314187 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/encoders/ExpressionEncoderSuite.scala --- @@ -239,6 +240,30 @@ class

[GitHub] spark pull request: [SPARK-12425][STREAMING] DStream union optimis...

2015-12-22 Thread tdas
Github user tdas commented on the pull request: https://github.com/apache/spark/pull/10382#issuecomment-166769944 Would be nice to add a unit test in the WindowOperationsSuite for this. To make sure that windowed RDDs always have the partitioner attached. --- If your project is set

[GitHub] spark pull request: [SPARK-12477][SQL] - Tungsten projection fails...

2015-12-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10429#issuecomment-166778676 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-12477][SQL] - Tungsten projection fails...

2015-12-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10429#issuecomment-166778680 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-12477][SQL] - Tungsten projection fails...

2015-12-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10429#issuecomment-166778197 **[Test build #48223 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48223/consoleFull)** for PR 10429 at commit

[GitHub] spark pull request: [SPARK-12478][SQL] Bugfix: Dataset fields of p...

2015-12-22 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/10431#issuecomment-166788637 Thanks for the review! I'm merging this to master. We should also backport this to branch-1.6 after the release. --- If your project is set up for it, you can reply

[GitHub] spark pull request: [SPARK-12478][SQL] Bugfix: Dataset fields of p...

2015-12-22 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/10431 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-12439][SQL] Fix toCatalystArray and Map...

2015-12-22 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/10391#discussion_r48317931 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala --- @@ -406,11 +406,14 @@ object ScalaReflection extends

[GitHub] spark pull request: [SPARK-12439][SQL] Fix toCatalystArray and Map...

2015-12-22 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/10391#issuecomment-166790415 **[Test build #48226 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48226/consoleFull)** for PR 10391 at commit

[GitHub] spark pull request: [SPARK-12439][SQL] Fix toCatalystArray and Map...

2015-12-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10391#issuecomment-166790390 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-12445][SQL] Fix NullPointerException wh...

2015-12-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10401#issuecomment-166790393 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-12445][SQL] Fix NullPointerException wh...

2015-12-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10401#issuecomment-166790395 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-12439][SQL] Fix toCatalystArray and Map...

2015-12-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/10391#issuecomment-166790392 Test FAILed. Refer to this link for build results (access rights to CI server needed):

  1   2   3   4   5   6   >