[GitHub] spark issue #18760: [SPARK-21560][Core] Add hold mode for the LiveListenerBu...

2017-09-28 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18760 yea it will be great to see how SPARK-18838 works in real env! --- - To unsubscribe, e-mail:

[GitHub] spark issue #19389: [SPARK-22165][SQL] Resolve type conflicts between decima...

2017-09-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19389 **[Test build #82307 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82307/testReport)** for PR 19389 at commit

[GitHub] spark issue #19389: [SPARK-22165][SQL] Resolve type conflicts between decima...

2017-09-28 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19389 cc @cloud-fan (I believe my similar PR was reviewed by you before), @ueshin and @squito. --- - To unsubscribe, e-mail:

[GitHub] spark pull request #19389: [SPARK-22165][SQL] Resolve type conflicts between...

2017-09-28 Thread HyukjinKwon
GitHub user HyukjinKwon opened a pull request: https://github.com/apache/spark/pull/19389 [SPARK-22165][SQL] Resolve type conflicts between decimals, dates and timestamps in partition column ## What changes were proposed in this pull request? This PR proposes to re-use

[GitHub] spark issue #19290: [SPARK-22063][R] Upgrades lintr to latest commit sha1 ID

2017-09-28 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19290 Oh! but if we upgrade it first, many builds in PRs should be broken due to this check failure. Probably, let me revert the script first to not upgrade it in order to unblock this. ---

[GitHub] spark issue #19290: [SPARK-22063][R] Upgrades lintr to latest commit sha1 ID

2017-09-28 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19290 Let me revert "... || packageVersion("lintr") != "1.0.1.9000" this condition back so that it does not try to upgrade it. ---

[GitHub] spark issue #19290: [SPARK-22063][R] Upgrades lintr to latest commit sha1 ID

2017-09-28 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/19290 hmm, perhaps open a ticket to ask shane to update Jenkins to this commit checksum manually? --- - To unsubscribe, e-mail:

[GitHub] spark issue #19287: [SPARK-22074][Core] Task killed by other attempt task sh...

2017-09-28 Thread xuanyuanking
Github user xuanyuanking commented on the issue: https://github.com/apache/spark/pull/19287 @squito Hi Rashid, thanks for you review and advise. In the last commit I moved `killedByOtherAttempt` into `TaskSetManager ` as you say and added more asserts in UT. ---

[GitHub] spark issue #18760: [SPARK-21560][Core] Add hold mode for the LiveListenerBu...

2017-09-28 Thread xuanyuanking
Github user xuanyuanking commented on the issue: https://github.com/apache/spark/pull/18760 @vanzin Hi Vanzin, thanks a lot for your comments. Firstly answer your question about `Why isn't hold mode just calling queue.put (blocking) instead of queue.offer (non-blocking)?` In

[GitHub] spark issue #19388: [SPARK-22162] Executors and the driver should use consis...

2017-09-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19388 **[Test build #82304 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82304/testReport)** for PR 19388 at commit

[GitHub] spark issue #19388: [SPARK-22162] Executors and the driver should use consis...

2017-09-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19388 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19388: [SPARK-22162] Executors and the driver should use consis...

2017-09-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19388 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82304/ Test FAILed. ---

[GitHub] spark issue #19287: [SPARK-22074][Core] Task killed by other attempt task sh...

2017-09-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19287 **[Test build #82306 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82306/testReport)** for PR 19287 at commit

[GitHub] spark pull request #19387: [SPARK-22160][SQL] Make sample points per partiti...

2017-09-28 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19387 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #19387: [SPARK-22160][SQL] Make sample points per partiti...

2017-09-28 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/19387#discussion_r141786663 --- Diff: core/src/main/scala/org/apache/spark/Partitioner.scala --- @@ -108,11 +108,21 @@ class HashPartitioner(partitions: Int) extends Partitioner {

[GitHub] spark issue #19387: [SPARK-22160][SQL] Make sample points per partition (in ...

2017-09-28 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/19387 Merging in master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19364: [SPARK-22144][SQL] ExchangeCoordinator combine th...

2017-09-28 Thread liutang123
Github user liutang123 commented on a diff in the pull request: https://github.com/apache/spark/pull/19364#discussion_r141785140 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/exchange/ExchangeCoordinator.scala --- @@ -232,7 +232,7 @@ class ExchangeCoordinator(

[GitHub] spark pull request #19287: [SPARK-22074][Core] Task killed by other attempt ...

2017-09-28 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/19287#discussion_r141784872 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskInfo.scala --- @@ -66,6 +66,13 @@ class TaskInfo( */ var finishTime: Long =

[GitHub] spark pull request #19287: [SPARK-22074][Core] Task killed by other attempt ...

2017-09-28 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/19287#discussion_r141784812 --- Diff: core/src/test/scala/org/apache/spark/scheduler/TaskSetManagerSuite.scala --- @@ -744,6 +744,100 @@ class TaskSetManagerSuite extends

[GitHub] spark issue #19330: [SPARK-18134][SQL] Orderable MapType

2017-09-28 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/19330 @jinxing64 I would like to hear opinions from other reviewers whether `ordered` should be included in these methods like `ToString`, `buildFormattedString`, and `jsonValue`. IMO, I think we

[GitHub] spark pull request #19287: [SPARK-22074][Core] Task killed by other attempt ...

2017-09-28 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/19287#discussion_r141784747 --- Diff: core/src/test/scala/org/apache/spark/scheduler/TaskSetManagerSuite.scala --- @@ -744,6 +744,100 @@ class TaskSetManagerSuite extends

[GitHub] spark pull request #19364: [SPARK-22144][SQL] ExchangeCoordinator combine th...

2017-09-28 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/19364#discussion_r141784391 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/exchange/ExchangeCoordinator.scala --- @@ -232,7 +232,7 @@ class ExchangeCoordinator(

[GitHub] spark issue #19327: [SPARK-22136][SS] Implement stream-stream outer joins.

2017-09-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19327 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19327: [SPARK-22136][SS] Implement stream-stream outer joins.

2017-09-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19327 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82305/ Test PASSed. ---

[GitHub] spark issue #18805: [SPARK-19112][CORE] Support for ZStandard codec

2017-09-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18805 **[Test build #3938 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3938/testReport)** for PR 18805 at commit

[GitHub] spark issue #19327: [SPARK-22136][SS] Implement stream-stream outer joins.

2017-09-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19327 **[Test build #82305 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82305/testReport)** for PR 19327 at commit

[GitHub] spark issue #19344: [SPARK-22122][SQL] Use analyzed logical plans to count i...

2017-09-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19344 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82303/ Test PASSed. ---

[GitHub] spark issue #19387: [SPARK-22160][SQL] Make sample points per partition (in ...

2017-09-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19387 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82302/ Test PASSed. ---

[GitHub] spark issue #19344: [SPARK-22122][SQL] Use analyzed logical plans to count i...

2017-09-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19344 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19387: [SPARK-22160][SQL] Make sample points per partition (in ...

2017-09-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19387 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19387: [SPARK-22160][SQL] Make sample points per partition (in ...

2017-09-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19387 **[Test build #82302 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82302/testReport)** for PR 19387 at commit

[GitHub] spark issue #19344: [SPARK-22122][SQL] Use analyzed logical plans to count i...

2017-09-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19344 **[Test build #82303 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82303/testReport)** for PR 19344 at commit

[GitHub] spark pull request #19364: [SPARK-22144][SQL] ExchangeCoordinator combine th...

2017-09-28 Thread jinxing64
Github user jinxing64 commented on a diff in the pull request: https://github.com/apache/spark/pull/19364#discussion_r141781986 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/exchange/ExchangeCoordinator.scala --- @@ -232,7 +232,7 @@ class ExchangeCoordinator(

[GitHub] spark issue #19330: [SPARK-18134][SQL] Orderable MapType

2017-09-28 Thread jinxing64
Github user jinxing64 commented on the issue: https://github.com/apache/spark/pull/19330 @kiszk Thanks a lot for comments. Tests passed now. In current change `ordered` is included in `jsonValue`. But I'm not sure it is appropriate. Thanks again for taking time looking

[GitHub] spark pull request #18732: [SPARK-20396][SQL][PySpark] groupby().apply() wit...

2017-09-28 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18732#discussion_r141780866 --- Diff: python/pyspark/sql/functions.py --- @@ -2206,6 +2207,10 @@ def pandas_udf(f=None, returnType=StringType()): | 8| JOHN DOE|

[GitHub] spark issue #19344: [SPARK-22122][SQL] Use analyzed logical plans to count i...

2017-09-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19344 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19344: [SPARK-22122][SQL] Use analyzed logical plans to count i...

2017-09-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19344 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82301/ Test PASSed. ---

[GitHub] spark issue #19344: [SPARK-22122][SQL] Use analyzed logical plans to count i...

2017-09-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19344 **[Test build #82301 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82301/testReport)** for PR 19344 at commit

[GitHub] spark pull request #19364: [SPARK-22144][SQL] ExchangeCoordinator combine th...

2017-09-28 Thread liutang123
Github user liutang123 commented on a diff in the pull request: https://github.com/apache/spark/pull/19364#discussion_r141780697 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/exchange/ExchangeCoordinator.scala --- @@ -232,7 +232,7 @@ class ExchangeCoordinator(

[GitHub] spark issue #19380: [SPARK-22157] [SQL] The uniux_timestamp method handles t...

2017-09-28 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19380 We also have `FromUnixTime` and seems the data type of unix time is defined as `LongType` across those unix time expressions. We shouldn't change just one expression so there is inconsistency.

[GitHub] spark issue #19380: [SPARK-22157] [SQL] The uniux_timestamp method handles t...

2017-09-28 Thread ouyangxiaochen
Github user ouyangxiaochen commented on the issue: https://github.com/apache/spark/pull/19380 Since the RDMS keep the milliseconds, we should follow it. This proposal LGTM. @gatorsmile CC --- - To unsubscribe,

[GitHub] spark issue #19387: [SPARK-22160][SQL] Make sample points per partition (in ...

2017-09-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19387 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19380: [SPARK-22157] [SQL] The uniux_timestamp method ha...

2017-09-28 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19380#discussion_r141777351 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala --- @@ -620,7 +621,7 @@ abstract class

[GitHub] spark issue #19387: [SPARK-22160][SQL] Make sample points per partition (in ...

2017-09-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19387 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82295/ Test PASSed. ---

[GitHub] spark issue #19387: [SPARK-22160][SQL] Make sample points per partition (in ...

2017-09-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19387 **[Test build #82295 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82295/testReport)** for PR 19387 at commit

[GitHub] spark issue #19387: [SPARK-22160][SQL] Make sample points per partition (in ...

2017-09-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19387 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82296/ Test PASSed. ---

[GitHub] spark issue #19387: [SPARK-22160][SQL] Make sample points per partition (in ...

2017-09-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19387 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19360: [SPARK-22139][CORE]Remove the variable which is never us...

2017-09-28 Thread guoxiaolongzte
Github user guoxiaolongzte commented on the issue: https://github.com/apache/spark/pull/19360 I was trying to find out more not used --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19387: [SPARK-22160][SQL] Make sample points per partition (in ...

2017-09-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19387 **[Test build #82296 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82296/testReport)** for PR 19387 at commit

[GitHub] spark pull request #19388: [SPARK-22162] Executors and the driver should use...

2017-09-28 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19388#discussion_r141775799 --- Diff: core/src/test/scala/org/apache/spark/rdd/PairRDDFunctionsSuite.scala --- @@ -524,6 +525,13 @@ class PairRDDFunctionsSuite extends SparkFunSuite

[GitHub] spark pull request #19388: [SPARK-22162] Executors and the driver should use...

2017-09-28 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19388#discussion_r141774733 --- Diff: core/src/test/scala/org/apache/spark/rdd/PairRDDFunctionsSuite.scala --- @@ -864,6 +872,47 @@ class NewFakeFormatWithCallback() extends

[GitHub] spark pull request #19388: [SPARK-22162] Executors and the driver should use...

2017-09-28 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19388#discussion_r141774796 --- Diff: core/src/test/scala/org/apache/spark/rdd/PairRDDFunctionsSuite.scala --- @@ -864,6 +872,47 @@ class NewFakeFormatWithCallback() extends

[GitHub] spark pull request #19388: [SPARK-22162] Executors and the driver should use...

2017-09-28 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19388#discussion_r141774743 --- Diff: core/src/test/scala/org/apache/spark/rdd/PairRDDFunctionsSuite.scala --- @@ -864,6 +872,47 @@ class NewFakeFormatWithCallback() extends

[GitHub] spark pull request #19387: [SPARK-22160][SQL] Make sample points per partiti...

2017-09-28 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19387#discussion_r141776977 --- Diff: core/src/main/scala/org/apache/spark/Partitioner.scala --- @@ -108,11 +108,21 @@ class HashPartitioner(partitions: Int) extends Partitioner {

[GitHub] spark pull request #18732: [SPARK-20396][SQL][PySpark] groupby().apply() wit...

2017-09-28 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/18732#discussion_r141775540 --- Diff: python/pyspark/sql/functions.py --- @@ -2129,7 +2129,8 @@ def _create_udf(f, returnType, vectorized): def _udf(f,

[GitHub] spark pull request #15769: [SPARK-18191][CORE] Port RDD API to use commit pr...

2017-09-28 Thread rezasafi
Github user rezasafi commented on a diff in the pull request: https://github.com/apache/spark/pull/15769#discussion_r141774719 --- Diff: core/src/main/scala/org/apache/spark/internal/io/SparkHadoopMapReduceWriter.scala --- @@ -0,0 +1,249 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #19380: [SPARK-22157] [SQL] The uniux_timestamp method handles t...

2017-09-28 Thread httfighter
Github user httfighter commented on the issue: https://github.com/apache/spark/pull/19380 In RDMS , unix_timestamp method can keep the milliseconds. For example, execute the command as follows select unix_timestamp("2017-10-10 10:10:20.111") from test; you can get the

[GitHub] spark issue #19363: [Minor]Override toString of KeyValueGroupedDataset

2017-09-28 Thread yaooqinn
Github user yaooqinn commented on the issue: https://github.com/apache/spark/pull/19363 cc again @cloud-fan @gatorsmile --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #19384: [SPARK-22159][SQL] Make config names consistently end wi...

2017-09-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19384 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19384: [SPARK-22159][SQL] Make config names consistently end wi...

2017-09-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19384 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82297/ Test PASSed. ---

[GitHub] spark issue #19384: [SPARK-22159][SQL] Make config names consistently end wi...

2017-09-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19384 **[Test build #82297 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82297/testReport)** for PR 19384 at commit

[GitHub] spark issue #19360: [SPARK-22139][CORE]Remove the variable which is never us...

2017-09-28 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19360 Shall we close this one for now then? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19386: [SPARK-22161] [SQL] Add Impala-modified TPC-DS queries

2017-09-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19386 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82298/ Test PASSed. ---

[GitHub] spark issue #19386: [SPARK-22161] [SQL] Add Impala-modified TPC-DS queries

2017-09-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19386 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19386: [SPARK-22161] [SQL] Add Impala-modified TPC-DS queries

2017-09-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19386 **[Test build #82298 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82298/testReport)** for PR 19386 at commit

[GitHub] spark pull request #19364: [SPARK-22144] ExchangeCoordinator combine the par...

2017-09-28 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/19364#discussion_r141772352 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/exchange/ExchangeCoordinator.scala --- @@ -232,7 +232,7 @@ class ExchangeCoordinator(

[GitHub] spark pull request #19364: [SPARK-22144] ExchangeCoordinator combine the par...

2017-09-28 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/19364#discussion_r141772266 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/exchange/ExchangeCoordinator.scala --- @@ -232,7 +232,7 @@ class ExchangeCoordinator(

[GitHub] spark issue #19364: [SPARK-22144] ExchangeCoordinator combine the partitions...

2017-09-28 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/19364 please add `[SQL]` in the title like "[SPARK-22144][SQL]..." --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #19318: [SPARK-22096][ML] use aggregateByKeyLocally in feature f...

2017-09-28 Thread chenghao-intel
Github user chenghao-intel commented on the issue: https://github.com/apache/spark/pull/19318 @VinceShieh can you please mark this PR's title as "[WIP]"? --- - To unsubscribe, e-mail:

[GitHub] spark issue #19360: [SPARK-22139][CORE]Remove the variable which is never us...

2017-09-28 Thread guoxiaolongzte
Github user guoxiaolongzte commented on the issue: https://github.com/apache/spark/pull/19360 Okay, I accept your opinion --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #19294: [SPARK-21549][CORE] Respect OutputFormats with no output...

2017-09-28 Thread mridulm
Github user mridulm commented on the issue: https://github.com/apache/spark/pull/19294 Adding folks who have context about spark sql's `FileFormatWriter` @ericchiang @rxin @cloud-fan @steveloughran @jiangxb1987 ---

[GitHub] spark issue #19360: [SPARK-22139][CORE]Remove the variable which is never us...

2017-09-28 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19360 We could find like you did now. Otherwise, we could suggest to fix these instances in another PR from other guys fixing them. I personally don't support to open each PR for these if it is not

[GitHub] spark issue #19386: [SPARK-22161] [SQL] Add Impala-modified TPC-DS queries

2017-09-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19386 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19386: [SPARK-22161] [SQL] Add Impala-modified TPC-DS queries

2017-09-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19386 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82292/ Test PASSed. ---

[GitHub] spark issue #19386: [SPARK-22161] [SQL] Add Impala-modified TPC-DS queries

2017-09-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19386 **[Test build #82292 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82292/testReport)** for PR 19386 at commit

[GitHub] spark issue #19360: [SPARK-22139][CORE]Remove the variable which is never us...

2017-09-28 Thread guoxiaolongzte
Github user guoxiaolongzte commented on the issue: https://github.com/apache/spark/pull/19360 How do you find other similar mistakes? Of course, found a immediately resolved, or behind the code more and more chaos ---

[GitHub] spark issue #19386: [SPARK-22161] [SQL] Add Impala-modified TPC-DS queries

2017-09-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19386 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82291/ Test PASSed. ---

[GitHub] spark issue #19386: [SPARK-22161] [SQL] Add Impala-modified TPC-DS queries

2017-09-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19386 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19386: [SPARK-22161] [SQL] Add Impala-modified TPC-DS queries

2017-09-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19386 **[Test build #82291 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82291/testReport)** for PR 19386 at commit

[GitHub] spark issue #18732: [SPARK-20396][SQL][PySpark] groupby().apply() with panda...

2017-09-28 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18732 I believe I should cc @cloud-fan too BTW. Will take a closer look too soon. --- - To unsubscribe, e-mail:

[GitHub] spark issue #19388: [SPARK-22162] Executors and the driver should use consis...

2017-09-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19388 **[Test build #82304 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82304/testReport)** for PR 19388 at commit

[GitHub] spark issue #19327: [SPARK-22136][SS] Implement stream-stream outer joins.

2017-09-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19327 **[Test build #82305 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82305/testReport)** for PR 19327 at commit

[GitHub] spark issue #19344: [SPARK-22122][SQL] Respect WITH clauses to count input r...

2017-09-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19344 **[Test build #82303 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82303/testReport)** for PR 19344 at commit

[GitHub] spark pull request #18732: [SPARK-20396][SQL][PySpark] groupby().apply() wit...

2017-09-28 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/18732#discussion_r141765717 --- Diff: python/pyspark/sql/tests.py --- @@ -3376,6 +3377,74 @@ def test_vectorized_udf_empty_partition(self): res =

[GitHub] spark pull request #18732: [SPARK-20396][SQL][PySpark] groupby().apply() wit...

2017-09-28 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/18732#discussion_r141765529 --- Diff: python/pyspark/sql/functions.py --- @@ -2206,6 +2207,10 @@ def pandas_udf(f=None, returnType=StringType()): | 8| JOHN

[GitHub] spark pull request #18732: [SPARK-20396][SQL][PySpark] groupby().apply() wit...

2017-09-28 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/18732#discussion_r141765893 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/python/ArrowEvalPythonExec.scala --- @@ -44,14 +44,17 @@ case class

[GitHub] spark issue #19386: [SPARK-22161] [SQL] Add Impala-modified TPC-DS queries

2017-09-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19386 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82294/ Test FAILed. ---

[GitHub] spark issue #19386: [SPARK-22161] [SQL] Add Impala-modified TPC-DS queries

2017-09-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19386 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19386: [SPARK-22161] [SQL] Add Impala-modified TPC-DS queries

2017-09-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19386 **[Test build #82294 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82294/testReport)** for PR 19386 at commit

[GitHub] spark issue #19387: [SPARK-22160][SQL] Make sample points per partition (in ...

2017-09-28 Thread xuanwang14
Github user xuanwang14 commented on the issue: https://github.com/apache/spark/pull/19387 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19387: [SPARK-22160][SQL] Make sample points per partition (in ...

2017-09-28 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/19387 I put up a comment saying this test result should be deterministic, since the sampling uses a fixed seed based on partition id. ---

[GitHub] spark issue #19384: [SPARK-22159][SQL] Make config names consistently end wi...

2017-09-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19384 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82289/ Test FAILed. ---

[GitHub] spark issue #19387: [SPARK-22160][SQL] Make sample points per partition (in ...

2017-09-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19387 **[Test build #82302 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82302/testReport)** for PR 19387 at commit

[GitHub] spark issue #19384: [SPARK-22159][SQL] Make config names consistently end wi...

2017-09-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19384 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19384: [SPARK-22159][SQL] Make config names consistently end wi...

2017-09-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19384 **[Test build #82289 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82289/testReport)** for PR 19384 at commit

[GitHub] spark issue #19370: [SPARK-18136] Fix setup of SPARK_HOME variable on Window...

2017-09-28 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19370 Moving it to Python script sounds a good idea to me if possible in the future.I did have the smae concern with @felixcheung but to me okay to add it after reading the comment above.

[GitHub] spark issue #19344: [SPARK-22122][SQL] Respect WITH clauses to count input r...

2017-09-28 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/19344 @gatorsmile ok, fixed. Also, I checked this code could collect all the relations. --- - To unsubscribe, e-mail:

[GitHub] spark issue #19344: [SPARK-22122][SQL] Respect WITH clauses to count input r...

2017-09-28 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19344 **[Test build #82301 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82301/testReport)** for PR 19344 at commit

[GitHub] spark pull request #19387: [SPARK-22160][SQL] Make sample points per partiti...

2017-09-28 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/19387#discussion_r141764431 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/ConfigBehaviorSuite.scala --- @@ -0,0 +1,64 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #19387: [SPARK-22160][SQL] Make sample points per partiti...

2017-09-28 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/19387#discussion_r141764415 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/ConfigBehaviorSuite.scala --- @@ -0,0 +1,64 @@ +/* + * Licensed to the Apache Software

  1   2   3   4   >