[GitHub] spark issue #16028: [SPARK-18518][ML] HasSolver supports override

2017-06-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16028 **[Test build #78960 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78960/testReport)** for PR 16028 at commit [`6bb1daf`](https://github.com/apache/spark/commit/6b

[GitHub] spark issue #18424: [SPARK-17091] Add rule to convert IN predicate to equiva...

2017-06-29 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/18424 Have you done actual benchmarks to validate that this is a perf improvement? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pro

[GitHub] spark issue #18479: WIP - logical plan stat propagation using mixin and visi...

2017-06-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18479 **[Test build #78959 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78959/testReport)** for PR 18479 at commit [`9c32d25`](https://github.com/apache/spark/commit/9c

[GitHub] spark issue #18469: [SPARK-21256] [SQL] Add withSQLConf to Catalyst Test

2017-06-29 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/18469 Can we minimize the change by just adding this method to PlanTest? It's not that many lines of code. --- If your project is set up for it, you can reply to this email and have your reply appear on Gi

[GitHub] spark issue #16028: [SPARK-18518][ML] HasSolver supports override

2017-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16028 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16028: [SPARK-18518][ML] HasSolver supports override

2017-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16028 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78956/ Test FAILed. ---

[GitHub] spark issue #16028: [SPARK-18518][ML] HasSolver supports override

2017-06-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16028 **[Test build #78956 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78956/testReport)** for PR 16028 at commit [`39ea8e1`](https://github.com/apache/spark/commit/3

[GitHub] spark issue #18481: [SPARK-20889][SparkR] Grouped documentation for WINDOW c...

2017-06-29 Thread actuaryzhang
Github user actuaryzhang commented on the issue: https://github.com/apache/spark/pull/18481 Ahh, forgot about the window functions. This is actually the last set... @felixcheung @HyukjinKwon ![image](https://user-images.githubusercontent.com/11082368/27724147-55154b52-5d25-

[GitHub] spark issue #18481: [SPARK-20889][SparkR] Grouped documentation for WINDOW c...

2017-06-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18481 **[Test build #78957 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78957/testReport)** for PR 18481 at commit [`e7d19a3`](https://github.com/apache/spark/commit/e7

[GitHub] spark issue #18480: [SPARK-21052][SQL][Follow-up] Add hash map metrics to jo...

2017-06-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18480 **[Test build #78958 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78958/testReport)** for PR 18480 at commit [`d8df5e0`](https://github.com/apache/spark/commit/d8

[GitHub] spark issue #18470: [SPARK-21258][SQL] Fix WindowExec complex object aggrega...

2017-06-29 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18470 ah the spilling logic is not in 2.1, let me revert it, sorry for the trouble. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If yo

[GitHub] spark pull request #18481: [SPARK-20889][SparkR] Grouped documentation for W...

2017-06-29 Thread actuaryzhang
GitHub user actuaryzhang opened a pull request: https://github.com/apache/spark/pull/18481 [SPARK-20889][SparkR] Grouped documentation for WINDOW column methods ## What changes were proposed in this pull request? Grouped documentation for column window methods. You can mer

[GitHub] spark issue #16028: [SPARK-18518][ML] HasSolver supports override

2017-06-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16028 **[Test build #78953 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78953/testReport)** for PR 16028 at commit [`b907314`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #18480: [SPARK-21052][SQL][Follow-up] Add hash map metrics to jo...

2017-06-29 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/18480 cc @gatorsmile --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark issue #17758: [SPARK-20460][SPARK-21144][SQL] Make it more consistent ...

2017-06-29 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/17758 It's a little messy about where we need to add this check, @maropu can you give a summary about it? thanks! --- If your project is set up for it, you can reply to this email and have your reply a

[GitHub] spark pull request #18480: [SPARK-21052][SQL][Follow-up] Add hash map metric...

2017-06-29 Thread viirya
GitHub user viirya opened a pull request: https://github.com/apache/spark/pull/18480 [SPARK-21052][SQL][Follow-up] Add hash map metrics to join ## What changes were proposed in this pull request? Remove `numHashCollisions` in `BytesToBytesMap`. And change `getAverageProbesP

[GitHub] spark issue #18470: [SPARK-21258][SQL] Fix WindowExec complex object aggrega...

2017-06-29 Thread zzcclp
Github user zzcclp commented on the issue: https://github.com/apache/spark/pull/18470 Hi, @cloud-fan , @hvanhovell , after merging this pr into **branch-2.1**, there are some errors: 1. `value WINDOW_EXEC_BUFFER_SPILL_THRESHOLD is not a member of object org.apache.spark.sql.in

[GitHub] spark issue #16028: [SPARK-18518][ML] HasSolver supports override

2017-06-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16028 **[Test build #78956 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78956/testReport)** for PR 16028 at commit [`39ea8e1`](https://github.com/apache/spark/commit/39

[GitHub] spark pull request #17758: [SPARK-20460][SPARK-21144][SQL] Make it more cons...

2017-06-29 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/17758#discussion_r124973610 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala --- @@ -353,7 +341,10 @@ class SessionCatalog( val

[GitHub] spark issue #18023: [SPARK-12139] [SQL] REGEX Column Specification

2017-06-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18023 **[Test build #78955 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78955/testReport)** for PR 18023 at commit [`4e36ed9`](https://github.com/apache/spark/commit/4e

[GitHub] spark issue #16028: [SPARK-18518][ML] HasSolver supports override

2017-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16028 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #16028: [SPARK-18518][ML] HasSolver supports override

2017-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16028 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78953/ Test FAILed. ---

[GitHub] spark issue #18479: WIP - stat propagation code using mixin and visitor patt...

2017-06-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18479 **[Test build #78954 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78954/testReport)** for PR 18479 at commit [`94fb669`](https://github.com/apache/spark/commit/94

[GitHub] spark pull request #18023: [SPARK-12139] [SQL] REGEX Column Specification

2017-06-29 Thread janewangfb
Github user janewangfb commented on a diff in the pull request: https://github.com/apache/spark/pull/18023#discussion_r124973097 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/unresolved.scala --- @@ -307,6 +311,28 @@ case class UnresolvedStar(target:

[GitHub] spark pull request #17758: [SPARK-20460][SPARK-21144][SQL] Make it more cons...

2017-06-29 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/17758#discussion_r124972759 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala --- @@ -353,7 +341,10 @@ class SessionCatalog(

[GitHub] spark pull request #18479: WIP - stat propagation code using mixin

2017-06-29 Thread rxin
GitHub user rxin opened a pull request: https://github.com/apache/spark/pull/18479 WIP - stat propagation code using mixin ## What changes were proposed in this pull request? TBD ## How was this patch tested? Should be covered by existing test cases. You can merge t

[GitHub] spark issue #16028: [SPARK-18518][ML] HasSolver supports override

2017-06-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16028 **[Test build #78953 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78953/testReport)** for PR 16028 at commit [`b907314`](https://github.com/apache/spark/commit/b9

[GitHub] spark pull request #18334: [SPARK-21127] [SQL] Update statistics after data ...

2017-06-29 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18334#discussion_r124971615 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -774,6 +774,12 @@ object SQLConf { .doubleConf

[GitHub] spark pull request #18334: [SPARK-21127] [SQL] Update statistics after data ...

2017-06-29 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18334#discussion_r124971431 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala --- @@ -437,7 +437,20 @@ case class AlterTableAddPartitionCommand(

[GitHub] spark issue #14431: [SPARK-16258][SparkR] Automatically append the grouping ...

2017-06-29 Thread falaki
Github user falaki commented on the issue: https://github.com/apache/spark/pull/14431 @NarineK how about adding this as a new API e.g., `gapplyWithKeys()`. I am extremely worried about the semantic change. It can break existing SparkR applications and will be confusing for users. --

[GitHub] spark pull request #18445: [Spark-19726][SQL] Faild to insert null timestamp...

2017-06-29 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18445#discussion_r124970124 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JdbcUtils.scala --- @@ -266,10 +266,14 @@ object JdbcUtils extends Log

[GitHub] spark pull request #18445: [Spark-19726][SQL] Faild to insert null timestamp...

2017-06-29 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18445#discussion_r124970067 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JdbcUtils.scala --- @@ -266,10 +266,14 @@ object JdbcUtils extends Log

[GitHub] spark pull request #18445: [Spark-19726][SQL] Faild to insert null timestamp...

2017-06-29 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18445#discussion_r124970022 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JdbcUtils.scala --- @@ -290,7 +294,11 @@ object JdbcUtils extends Logg

[GitHub] spark issue #18465: [SPARK-21093][R] Terminate R's worker processes in the p...

2017-06-29 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18465 > isn't being "terminated" enough? that we have to pskill it again? Indeed, this is the point. I really tested so many times but `exit` does not terminate (I think I might have to change

[GitHub] spark pull request #18458: [SPARK-20889][SparkR] Grouped documentation for C...

2017-06-29 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/18458 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request #18464: [SPARK-21250][WEB-UI]Add a url in the table of 'R...

2017-06-29 Thread guoxiaolongzte
Github user guoxiaolongzte commented on a diff in the pull request: https://github.com/apache/spark/pull/18464#discussion_r124969031 --- Diff: core/src/main/scala/org/apache/spark/deploy/worker/ui/WorkerPage.scala --- @@ -112,7 +112,15 @@ private[ui] class WorkerPage(parent: Worker

[GitHub] spark issue #18458: [SPARK-20889][SparkR] Grouped documentation for COLLECTI...

2017-06-29 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/18458 merged to master. thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and w

[GitHub] spark issue #18458: [SPARK-20889][SparkR] Grouped documentation for COLLECTI...

2017-06-29 Thread actuaryzhang
Github user actuaryzhang commented on the issue: https://github.com/apache/spark/pull/18458 @felixcheung This is the last set of this doc update. Once it gets in, I will do another pass to fix any styles or consistency issue. --- If your project is set up for it, you can reply to th

[GitHub] spark issue #18478: [SPARK-21253][Core][HOTFIX]Fix Scala 2.10 build

2017-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18478 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18478: [SPARK-21253][Core][HOTFIX]Fix Scala 2.10 build

2017-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18478 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78947/ Test FAILed. ---

[GitHub] spark issue #18430: [SPARK-21223]:Thread-safety issue in FsHistoryProvider

2017-06-29 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/18430 let's do this in another PR, seems a different threading issue. Also would you please change the PR title like: Change fileToAppInfo in FsHistoryProvider to fix concurrent issue. --- If

[GitHub] spark issue #18478: [SPARK-21253][Core][HOTFIX]Fix Scala 2.10 build

2017-06-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18478 **[Test build #78947 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78947/testReport)** for PR 18478 at commit [`8295b3c`](https://github.com/apache/spark/commit/8

[GitHub] spark issue #18436: [SPARK-20073][SQL] Prints an explicit warning message in...

2017-06-29 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18436 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the fe

[GitHub] spark issue #18430: [SPARK-21223]:Thread-safety issue in FsHistoryProvider

2017-06-29 Thread zenglinxi0615
Github user zenglinxi0615 commented on the issue: https://github.com/apache/spark/pull/18430 sorry, it's a typing error, i mean the related JIRA: SPARK-21078. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your proje

[GitHub] spark pull request #17899: [SPARK-20636] Add new optimization rule to transp...

2017-06-29 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/17899#discussion_r124966795 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -610,6 +611,25 @@ object CollapseWindow extends Rul

[GitHub] spark issue #17935: [SPARK-20690][SQL] Subqueries in FROM should have alias ...

2017-06-29 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/17935 The reason I found out about this is because the one of the widely circulated TPC-DS benchmark harness online uses this. --- If your project is set up for it, you can reply to this email and have you

[GitHub] spark pull request #17899: [SPARK-20636] Add new optimization rule to transp...

2017-06-29 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/17899#discussion_r124966455 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -610,6 +611,25 @@ object CollapseWindow extends Rul

[GitHub] spark issue #17935: [SPARK-20690][SQL] Subqueries in FROM should have alias ...

2017-06-29 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/17935 Sorry, to be accurate, for the syntax of derived table in SQL, the databases I listed above are commonly seen in the market, and they don't support it without alias name. SQL 2003 grammar also doesn'

[GitHub] spark issue #18458: [SPARK-20889][SparkR] Grouped documentation for COLLECTI...

2017-06-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18458 **[Test build #78949 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78949/testReport)** for PR 18458 at commit [`8be3e49`](https://github.com/apache/spark/commit/8

[GitHub] spark issue #18458: [SPARK-20889][SparkR] Grouped documentation for COLLECTI...

2017-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18458 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18458: [SPARK-20889][SparkR] Grouped documentation for COLLECTI...

2017-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18458 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78949/ Test PASSed. ---

[GitHub] spark issue #17681: [SPARK-20383][SQL] Supporting Create [temporary] Functio...

2017-06-29 Thread ouyangxiaochen
Github user ouyangxiaochen commented on the issue: https://github.com/apache/spark/pull/17681 @felixcheung Thank you very much! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature en

[GitHub] spark issue #18436: [SPARK-20073][SQL] Prints an explicit warning message in...

2017-06-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18436 **[Test build #78952 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78952/testReport)** for PR 18436 at commit [`12347e6`](https://github.com/apache/spark/commit/12

[GitHub] spark issue #18430: [SPARK-21223]:Thread-safety issue in FsHistoryProvider

2017-06-29 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/18430 What's the issue of SPARK-13988? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #18436: [SPARK-20073][SQL] Prints an explicit warning mes...

2017-06-29 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/18436#discussion_r124965300 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Column.scala --- @@ -464,7 +464,15 @@ class Column(val expr: Expression) extends Logging { * @g

[GitHub] spark issue #17681: [SPARK-20383][SQL] Supporting Create [temporary] Functio...

2017-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17681 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #17681: [SPARK-20383][SQL] Supporting Create [temporary] Functio...

2017-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17681 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78946/ Test PASSed. ---

[GitHub] spark issue #18471: [SPARK-21259] More rules for scalastyle

2017-06-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18471 **[Test build #78951 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78951/testReport)** for PR 18471 at commit [`fd5bde5`](https://github.com/apache/spark/commit/fd

[GitHub] spark pull request #18471: [SPARK-21259] More rules for scalastyle

2017-06-29 Thread gengliangwang
Github user gengliangwang commented on a diff in the pull request: https://github.com/apache/spark/pull/18471#discussion_r124964773 --- Diff: scalastyle-config.xml --- @@ -245,7 +245,13 @@ This file is divided into 3 sections: - COMMA + COLO

[GitHub] spark issue #17681: [SPARK-20383][SQL] Supporting Create [temporary] Functio...

2017-06-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17681 **[Test build #78946 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78946/testReport)** for PR 17681 at commit [`b49280a`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #17935: [SPARK-20690][SQL] Subqueries in FROM should have alias ...

2017-06-29 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/17935 I don't think that argument is useful at all. For example, none of the other databases support the DataFrame API. Does that mean few users will write DataFrame code? --- If your project is set up fo

[GitHub] spark pull request #18023: [SPARK-12139] [SQL] REGEX Column Specification

2017-06-29 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18023#discussion_r124964462 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/unresolved.scala --- @@ -307,6 +311,28 @@ case class UnresolvedStar(target:

[GitHub] spark pull request #18436: [SPARK-20073][SQL] Prints an explicit warning mes...

2017-06-29 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18436#discussion_r124963887 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Column.scala --- @@ -464,7 +464,15 @@ class Column(val expr: Expression) extends Logging {

[GitHub] spark issue #18436: [SPARK-20073][SQL] Prints an explicit warning message in...

2017-06-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18436 **[Test build #78950 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78950/testReport)** for PR 18436 at commit [`be56898`](https://github.com/apache/spark/commit/be

[GitHub] spark issue #18436: [SPARK-20073][SQL] Prints an explicit warning message in...

2017-06-29 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18436 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark pull request #18464: [SPARK-21250][WEB-UI]Add a url in the table of 'R...

2017-06-29 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/18464#discussion_r124963695 --- Diff: core/src/main/scala/org/apache/spark/deploy/worker/ui/WorkerPage.scala --- @@ -112,7 +112,15 @@ private[ui] class WorkerPage(parent: WorkerWeb

[GitHub] spark pull request #18388: [SPARK-21175] Reject OpenBlocks when memory short...

2017-06-29 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18388#discussion_r124963673 --- Diff: docs/configuration.md --- @@ -614,6 +614,34 @@ Apart from these, the following properties are also available, and may be useful

[GitHub] spark pull request #16056: [SPARK-18623][SQL] Add `returnNullable` to `Stati...

2017-06-29 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16056#discussion_r124963500 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/encoders/RowEncoder.scala --- @@ -96,28 +96,32 @@ object RowEncoder { Da

[GitHub] spark issue #18266: [SPARK-20427][SQL] Read JDBC table use custom schema

2017-06-29 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/18266 I am fine to support customized schema for read path of JDBC relation. However, we need to check whether the user-specified schema matches the underlying the table schema. If not matched, we need

[GitHub] spark issue #18458: [SPARK-20889][SparkR] Grouped documentation for COLLECTI...

2017-06-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18458 **[Test build #78949 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78949/testReport)** for PR 18458 at commit [`8be3e49`](https://github.com/apache/spark/commit/8b

[GitHub] spark pull request #16056: [SPARK-18623][SQL] Add `returnNullable` to `Stati...

2017-06-29 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16056#discussion_r124963200 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/JavaTypeInference.scala --- @@ -216,23 +216,26 @@ object JavaTypeInference {

[GitHub] spark pull request #18266: [SPARK-20427][SQL] Read JDBC table use custom sch...

2017-06-29 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18266#discussion_r124963184 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JDBCOptions.scala --- @@ -120,6 +120,7 @@ class JDBCOptions( //

[GitHub] spark issue #18474: [SPARK-21235][SPARKR] UTest should clear temp results wh...

2017-06-29 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/18474 We should not need this, because in `afterEach()` we will stop each store, which should clear both the memoryStore and the disk blocks. --- If your project is set up for it, you can reply to th

[GitHub] spark pull request #18266: [SPARK-20427][SQL] Read JDBC table use custom sch...

2017-06-29 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/18266#discussion_r124962985 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/jdbc/JDBCSuite.scala --- @@ -907,7 +907,7 @@ class JDBCSuite extends SparkFunSuite assert

[GitHub] spark pull request #18448: [SPARK-20889][SparkR] Grouped documentation for M...

2017-06-29 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/18448 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request #18470: [SPARK-21258][SQL] Fix WindowExec complex object ...

2017-06-29 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/18470 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #18470: [SPARK-21258][SQL] Fix WindowExec complex object aggrega...

2017-06-29 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18470 thanks, merging to master/2.2/2.1! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enable

[GitHub] spark issue #18448: [SPARK-20889][SparkR] Grouped documentation for MISC col...

2017-06-29 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/18448 merged to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark pull request #18464: [SPARK-21250][WEB-UI]Add a url in the table of 'R...

2017-06-29 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/18464#discussion_r124962728 --- Diff: core/src/main/scala/org/apache/spark/deploy/worker/ui/WorkerPage.scala --- @@ -112,7 +112,15 @@ private[ui] class WorkerPage(parent: WorkerWebUI

[GitHub] spark issue #18468: [SPARK-20873][SQL] Enhance ColumnVector to support compr...

2017-06-29 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/18468 @cloud-fan Could you review this? As we discussed at Spark Summit, I prepared a new ColumnVector for compressed column using the current schemes. Any comments are appreciated. --- If your project

[GitHub] spark issue #18468: [SPARK-20873][SQL] Enhance ColumnVector to support compr...

2017-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18468 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78944/ Test PASSed. ---

[GitHub] spark issue #18468: [SPARK-20873][SQL] Enhance ColumnVector to support compr...

2017-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18468 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18465: [SPARK-21093][R] Terminate R's worker processes in the p...

2017-06-29 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/18465 hmm, this is a fairly small but crucial change. One point we were discussing, earlier, we are saying that we don't know if the child is going away, or merely slow starting, so we explicitly s

[GitHub] spark issue #18468: [SPARK-20873][SQL] Enhance ColumnVector to support compr...

2017-06-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18468 **[Test build #78944 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78944/testReport)** for PR 18468 at commit [`514400c`](https://github.com/apache/spark/commit/5

[GitHub] spark issue #18474: [SPARK-21235][SPARKR] UTest should clear temp results wh...

2017-06-29 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/18474 We should not need this, because in `afterEach()` we will stop each store. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request #18307: [SPARK-21100][SQL] describe should give quartiles...

2017-06-29 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/18307#discussion_r124961333 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -2205,37 +2205,170 @@ class Dataset[T] private[sql]( * // max 9

[GitHub] spark issue #14431: [SPARK-16258][SparkR] Automatically append the grouping ...

2017-06-29 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/14431 https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/RelationalGroupedDataset.scala#L51 You just need to change the above line ``` val groupin

[GitHub] spark issue #18473: [SPARK-21260][SQL][MINOR] Remove the unused OutputFakerE...

2017-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18473 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78945/ Test PASSed. ---

[GitHub] spark issue #18473: [SPARK-21260][SQL][MINOR] Remove the unused OutputFakerE...

2017-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18473 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #17681: [SPARK-20383][SQL] Supporting Create [temporary] Functio...

2017-06-29 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/17681 there's an issue causing `fails due to an unknown error code` - if you haven't rebase, rebase to master should eliminate it --- If your project is set up for it, you can reply to this email and

[GitHub] spark issue #18473: [SPARK-21260][SQL][MINOR] Remove the unused OutputFakerE...

2017-06-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18473 **[Test build #78945 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78945/testReport)** for PR 18473 at commit [`c5084a3`](https://github.com/apache/spark/commit/c

[GitHub] spark issue #18474: [SPARK-21235][SPARKR] UTest should clear temp results wh...

2017-06-29 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/18474 hi - I don't think `[SPARKR]` in the title is right. Sounds like this should be `[core]` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub a

[GitHub] spark issue #18477: [SPARK-21261][DOCS]SQL Regex document fix

2017-06-29 Thread gf53520
Github user gf53520 commented on the issue: https://github.com/apache/spark/pull/18477 test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #18471: [SPARK-21259] More rules for scalastyle

2017-06-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18471 **[Test build #78948 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78948/testReport)** for PR 18471 at commit [`05084a9`](https://github.com/apache/spark/commit/05

[GitHub] spark issue #18430: [SPARK-21223]:Thread-safety issue in FsHistoryProvider

2017-06-29 Thread zenglinxi0615
Github user zenglinxi0615 commented on the issue: https://github.com/apache/spark/pull/18430 @srowen thanks for your suggestions again! and should I address the problem of SPARK-13988 in this PR? --- If your project is set up for it, you can reply to this email and have your reply ap

[GitHub] spark pull request #18478: [SPARK-21253][Core][HOTFIX]Fix Scala 2.10 build

2017-06-29 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/18478 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark issue #18478: [SPARK-21253][Core][HOTFIX]Fix Scala 2.10 build

2017-06-29 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/18478 Verified both Scala 2.10 and 2.11 build locally. Since Jenkins PR build doesn't use Scala 2.10, I'm going to merge directly. --- If your project is set up for it, you can reply to this email and h

[GitHub] spark issue #18467: [SPARK-19659][Core]Disable spark.reducer.maxReqSizeShuff...

2017-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18467 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78943/ Test PASSed. ---

[GitHub] spark issue #18467: [SPARK-19659][Core]Disable spark.reducer.maxReqSizeShuff...

2017-06-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18467 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark issue #18467: [SPARK-19659][Core]Disable spark.reducer.maxReqSizeShuff...

2017-06-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18467 **[Test build #78943 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/78943/testReport)** for PR 18467 at commit [`b8f022e`](https://github.com/apache/spark/commit/b

  1   2   3   4   5   6   7   >