[GitHub] [spark] AmplabJenkins removed a comment on pull request #28804: [SPARK-31973][SQL] Add ability to disable Sort,Spill in Partial aggregation

2020-06-17 Thread GitBox


AmplabJenkins removed a comment on pull request #28804:
URL: https://github.com/apache/spark/pull/28804#issuecomment-645781069







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins commented on pull request #28804: [SPARK-31973][SQL] Add ability to disable Sort,Spill in Partial aggregation

2020-06-17 Thread GitBox


AmplabJenkins commented on pull request #28804:
URL: https://github.com/apache/spark/pull/28804#issuecomment-645781069







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA commented on pull request #28804: [SPARK-31973][SQL] Add ability to disable Sort,Spill in Partial aggregation

2020-06-17 Thread GitBox


SparkQA commented on pull request #28804:
URL: https://github.com/apache/spark/pull/28804#issuecomment-645780769


   **[Test build #124199 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124199/testReport)**
 for PR 28804 at commit 
[`2b3704b`](https://github.com/apache/spark/commit/2b3704b1bd96e0c64890a0d730f840c0ab7da36e).



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] karuppayya commented on a change in pull request #28804: [SPARK-31973][SQL] Add ability to disable Sort,Spill in Partial aggregation

2020-06-17 Thread GitBox


karuppayya commented on a change in pull request #28804:
URL: https://github.com/apache/spark/pull/28804#discussion_r441974540



##
File path: 
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala
##
@@ -2173,6 +2173,13 @@ object SQLConf {
   .checkValue(bit => bit >= 10 && bit <= 30, "The bit value must be in 
[10, 30].")
   .createWithDefault(16)
 
+  val SPILL_PARTIAL_AGGREGATE_DISABLED =
+buildConf("spark.sql.aggregate.spill.partialaggregate.disabled")

Review comment:
   I have renamed the config. Can you please check





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] karuppayya commented on a change in pull request #28804: [SPARK-31973][SQL] Add ability to disable Sort,Spill in Partial aggregation

2020-06-17 Thread GitBox


karuppayya commented on a change in pull request #28804:
URL: https://github.com/apache/spark/pull/28804#discussion_r441974438



##
File path: 
sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/HashAggregateExec.scala
##
@@ -72,6 +74,8 @@ case class HashAggregateExec(
 "peakMemory" -> SQLMetrics.createSizeMetric(sparkContext, "peak memory"),
 "spillSize" -> SQLMetrics.createSizeMetric(sparkContext, "spill size"),
 "aggTime" -> SQLMetrics.createTimingMetric(sparkContext, "time in 
aggregation build"),
+"partialAggSkipped" -> SQLMetrics.createMetric(sparkContext, "Num records" 
+
+  " skipped partial aggregation skipped"),

Review comment:
   done





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] karuppayya commented on pull request #28804: [SPARK-31973][SQL] Add ability to disable Sort,Spill in Partial aggregation

2020-06-17 Thread GitBox


karuppayya commented on pull request #28804:
URL: https://github.com/apache/spark/pull/28804#issuecomment-645779303


   @maropu @cloud-fan Can you please add me to the whitelist to trigger the 
tests?



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] prakharjain09 commented on a change in pull request #28619: [SPARK-21040][CORE] Speculate tasks which are running on decommission executors

2020-06-17 Thread GitBox


prakharjain09 commented on a change in pull request #28619:
URL: https://github.com/apache/spark/pull/28619#discussion_r441973394



##
File path: core/src/main/scala/org/apache/spark/internal/config/package.scala
##
@@ -1842,6 +1842,17 @@ package object config {
   .timeConf(TimeUnit.MILLISECONDS)
   .createOptional
 
+  private[spark] val EXECUTOR_DECOMMISSION_KILL_INTERVAL =
+ConfigBuilder("spark.executor.decommission.killInterval")
+  .doc("Duration after which a decommissioned executor will be killed 
forcefully." +
+"This config is useful for cloud environments where we know in advance 
when " +
+"an executor is going to go down after decommissioning signal i.e. 
around 2 mins " +
+"in aws spot nodes, 1/2 hrs in spot block nodes etc. This config is 
currently " +

Review comment:
   @cloud-fan This config can be set by users based on their setups. If 
they are using AWS spot nodes, timeout can be set to somewhere around 120 
seconds, if they are using fix duration 6hrs spot blocks (say they decommission 
executors at 5:45), timeout can be set to 15 mins and so on.
   
   If user doesn't set this timeout, things will remain as they were and tasks 
running on decommission executors  won't get any special treatment with respect 
to speculation.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #28804: [SPARK-31973][SQL] Add ability to disable Sort,Spill in Partial aggregation

2020-06-17 Thread GitBox


AmplabJenkins removed a comment on pull request #28804:
URL: https://github.com/apache/spark/pull/28804#issuecomment-645778573







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins commented on pull request #28804: [SPARK-31973][SQL] Add ability to disable Sort,Spill in Partial aggregation

2020-06-17 Thread GitBox


AmplabJenkins commented on pull request #28804:
URL: https://github.com/apache/spark/pull/28804#issuecomment-645778573







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait

2020-06-17 Thread GitBox


AmplabJenkins removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-645776165







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait

2020-06-17 Thread GitBox


AmplabJenkins commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-645776165







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait

2020-06-17 Thread GitBox


SparkQA commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-645775862


   **[Test build #124198 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124198/testReport)**
 for PR 28710 at commit 
[`e844c11`](https://github.com/apache/spark/commit/e844c11939d5523d5d7402f5ee5ed6cabd52f8fb).



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait

2020-06-17 Thread GitBox


AmplabJenkins removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-645773820


   Test FAILed.
   Refer to this link for build results (access rights to CI server needed): 
   
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/28814/
   Test FAILed.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] huaxingao commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait

2020-06-17 Thread GitBox


huaxingao commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-645775168


   retest this please



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #28825: [SPARK-32020][SQL][MINOR] Better error message when SPARK_HOME or spark.test.home is not set.

2020-06-17 Thread GitBox


AmplabJenkins removed a comment on pull request #28825:
URL: https://github.com/apache/spark/pull/28825#issuecomment-645774200







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] huaxingao commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait

2020-06-17 Thread GitBox


huaxingao commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-645774417


   I moved ```asBinary``` back to ```LogisticRegressionSummary``` to get rid of 
this
   ```
   
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.ml.classification.LogisticRegressionSummary.asBinary")
   ```
   All the rest of the MiMa problems are InheritedNewAbstractMethodProblem. I 
think those are OK. 



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins commented on pull request #28852: [SPARK-30616][SQL] Introduce TTL config option for SQL Metadata Cache

2020-06-17 Thread GitBox


AmplabJenkins commented on pull request #28852:
URL: https://github.com/apache/spark/pull/28852#issuecomment-645774143







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #28852: [SPARK-30616][SQL] Introduce TTL config option for SQL Metadata Cache

2020-06-17 Thread GitBox


AmplabJenkins removed a comment on pull request #28852:
URL: https://github.com/apache/spark/pull/28852#issuecomment-645774143







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins commented on pull request #28825: [SPARK-32020][SQL][MINOR] Better error message when SPARK_HOME or spark.test.home is not set.

2020-06-17 Thread GitBox


AmplabJenkins commented on pull request #28825:
URL: https://github.com/apache/spark/pull/28825#issuecomment-645774200







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA removed a comment on pull request #28852: [SPARK-30616][SQL] Introduce TTL config option for SQL Metadata Cache

2020-06-17 Thread GitBox


SparkQA removed a comment on pull request #28852:
URL: https://github.com/apache/spark/pull/28852#issuecomment-645698048


   **[Test build #124182 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124182/testReport)**
 for PR 28852 at commit 
[`f03fe24`](https://github.com/apache/spark/commit/f03fe2463ae39f886c5558fa3a307a87d7f549f3).



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait

2020-06-17 Thread GitBox


AmplabJenkins removed a comment on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-645773814


   Merged build finished. Test FAILed.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA removed a comment on pull request #28825: [SPARK-32020][SQL][MINOR] Better error message when SPARK_HOME or spark.test.home is not set.

2020-06-17 Thread GitBox


SparkQA removed a comment on pull request #28825:
URL: https://github.com/apache/spark/pull/28825#issuecomment-645726046


   **[Test build #124187 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124187/testReport)**
 for PR 28825 at commit 
[`0ff8bf0`](https://github.com/apache/spark/commit/0ff8bf0d1b25665e40c9518de4211b9912f8b61f).



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait

2020-06-17 Thread GitBox


AmplabJenkins commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-645773814







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA commented on pull request #28852: [SPARK-30616][SQL] Introduce TTL config option for SQL Metadata Cache

2020-06-17 Thread GitBox


SparkQA commented on pull request #28852:
URL: https://github.com/apache/spark/pull/28852#issuecomment-645773560


   **[Test build #124182 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124182/testReport)**
 for PR 28852 at commit 
[`f03fe24`](https://github.com/apache/spark/commit/f03fe2463ae39f886c5558fa3a307a87d7f549f3).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait

2020-06-17 Thread GitBox


SparkQA commented on pull request #28710:
URL: https://github.com/apache/spark/pull/28710#issuecomment-645773554


   **[Test build #124197 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124197/testReport)**
 for PR 28710 at commit 
[`e844c11`](https://github.com/apache/spark/commit/e844c11939d5523d5d7402f5ee5ed6cabd52f8fb).



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA commented on pull request #28825: [SPARK-32020][SQL][MINOR] Better error message when SPARK_HOME or spark.test.home is not set.

2020-06-17 Thread GitBox


SparkQA commented on pull request #28825:
URL: https://github.com/apache/spark/pull/28825#issuecomment-645773587


   **[Test build #124187 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124187/testReport)**
 for PR 28825 at commit 
[`0ff8bf0`](https://github.com/apache/spark/commit/0ff8bf0d1b25665e40c9518de4211b9912f8b61f).
* This patch passes all tests.
* This patch **does not merge cleanly**.
* This patch adds the following public classes _(experimental)_:
 * `class TableIdentifierParserSuite extends SparkFunSuite with SQLHelper  `
 * `class SQLQueryTestSuite extends QueryTest with SharedSparkSession with 
SQLHelper `



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] gatorsmile commented on pull request #28817: [WIP][SPARK-31197][CORE] Exit the executor once all tasks and migrations are finished built on top of on top of spark20629

2020-06-17 Thread GitBox


gatorsmile commented on pull request #28817:
URL: https://github.com/apache/spark/pull/28817#issuecomment-645771306


   cc @Ngone51 @jiangxb1987 



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] gatorsmile commented on pull request #28818: [WIP][SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

2020-06-17 Thread GitBox


gatorsmile commented on pull request #28818:
URL: https://github.com/apache/spark/pull/28818#issuecomment-645771384


   cc @Ngone51 @jiangxb1987 



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] gatorsmile commented on pull request #28831: [SPARK-31993][SQL] Evaluate children code whenever needed in both varargCounts/varargBuilds for 'concat_ws' with columns having at least o

2020-06-17 Thread GitBox


gatorsmile commented on pull request #28831:
URL: https://github.com/apache/spark/pull/28831#issuecomment-645770760


   cc @rednaxelafx 



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] gatorsmile commented on pull request #28835: [WIP][SPARK-31926][TESTS][FOLLOWUP]Cleanup the thread local variable of hive metastore

2020-06-17 Thread GitBox


gatorsmile commented on pull request #28835:
URL: https://github.com/apache/spark/pull/28835#issuecomment-645770966


   cc @juliuszsompolski 



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] gatorsmile commented on pull request #28836: [SPARK-31561][SQL] Add QUALIFY Clause

2020-06-17 Thread GitBox


gatorsmile commented on pull request #28836:
URL: https://github.com/apache/spark/pull/28836#issuecomment-645770292


   @AngersZh Agree on what @bart-samwel said. Maybe we can revisit this in 
future? Let us first close this PR? 



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] gatorsmile commented on pull request #28841: [SPARK-31962][SQL] Provide option to load files after a specified date when reading from a folder path

2020-06-17 Thread GitBox


gatorsmile commented on pull request #28841:
URL: https://github.com/apache/spark/pull/28841#issuecomment-645769608


   cc @cloud-fan @gengliangwang @MaxGekk 



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #28854: [MINOR][DOCS] Emphasize the Streaming tab is for DStream API

2020-06-17 Thread GitBox


AmplabJenkins removed a comment on pull request #28854:
URL: https://github.com/apache/spark/pull/28854#issuecomment-645757520







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA removed a comment on pull request #28854: [MINOR][DOCS] Emphasize the Streaming tab is for DStream API

2020-06-17 Thread GitBox


SparkQA removed a comment on pull request #28854:
URL: https://github.com/apache/spark/pull/28854#issuecomment-645754673


   **[Test build #124196 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124196/testReport)**
 for PR 28854 at commit 
[`cbfe1ac`](https://github.com/apache/spark/commit/cbfe1acca078aa39d34c90714bbf20fc28d76f0a).



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA commented on pull request #28854: [MINOR][DOCS] Emphasize the Streaming tab is for DStream API

2020-06-17 Thread GitBox


SparkQA commented on pull request #28854:
URL: https://github.com/apache/spark/pull/28854#issuecomment-645757449


   **[Test build #124196 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124196/testReport)**
 for PR 28854 at commit 
[`cbfe1ac`](https://github.com/apache/spark/commit/cbfe1acca078aa39d34c90714bbf20fc28d76f0a).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins commented on pull request #28854: [MINOR][DOCS] Emphasize the Streaming tab is for DStream API

2020-06-17 Thread GitBox


AmplabJenkins commented on pull request #28854:
URL: https://github.com/apache/spark/pull/28854#issuecomment-645757520







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA commented on pull request #28854: [MINOR][DOCS] Emphasize the Streaming tab is for DStream API

2020-06-17 Thread GitBox


SparkQA commented on pull request #28854:
URL: https://github.com/apache/spark/pull/28854#issuecomment-645754673


   **[Test build #124196 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124196/testReport)**
 for PR 28854 at commit 
[`cbfe1ac`](https://github.com/apache/spark/commit/cbfe1acca078aa39d34c90714bbf20fc28d76f0a).



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #28854: [MINOR][DOCS] Emphasize the Streaming tab is for DStream API

2020-06-17 Thread GitBox


AmplabJenkins removed a comment on pull request #28854:
URL: https://github.com/apache/spark/pull/28854#issuecomment-645755010







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins commented on pull request #28854: [MINOR][DOCS] Emphasize the Streaming tab is for DStream API

2020-06-17 Thread GitBox


AmplabJenkins commented on pull request #28854:
URL: https://github.com/apache/spark/pull/28854#issuecomment-645755010







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] xuanyuanking commented on pull request #28854: [MINOR][DOCS] Emphasize the Streaming tab is for DStream API

2020-06-17 Thread GitBox


xuanyuanking commented on pull request #28854:
URL: https://github.com/apache/spark/pull/28854#issuecomment-645753269


   cc @HyukjinKwon @xccui as a minor follow up for SPARK-31792



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] xuanyuanking opened a new pull request #28854: [MINOR][DOCS] Emphasize the Streaming tab is for DStream API

2020-06-17 Thread GitBox


xuanyuanking opened a new pull request #28854:
URL: https://github.com/apache/spark/pull/28854


   ### What changes were proposed in this pull request?
   Emphasize the Streaming tab is for DStream API.
   
   ### Why are the changes needed?
   Some users reported that it's a little confusing of the streaming tab and 
structured streaming tab.
   
   
   ### Does this PR introduce _any_ user-facing change?
   Document change.
   
   ### How was this patch tested?
   N/A
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #28848: [SPARK-32003][CORE] Unregister outputs for executor on fetch failure …

2020-06-17 Thread GitBox


AmplabJenkins removed a comment on pull request #28848:
URL: https://github.com/apache/spark/pull/28848#issuecomment-645748224







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins commented on pull request #28848: [SPARK-32003][CORE] Unregister outputs for executor on fetch failure …

2020-06-17 Thread GitBox


AmplabJenkins commented on pull request #28848:
URL: https://github.com/apache/spark/pull/28848#issuecomment-645748224







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA removed a comment on pull request #28848: [SPARK-32003][CORE] Unregister outputs for executor on fetch failure …

2020-06-17 Thread GitBox


SparkQA removed a comment on pull request #28848:
URL: https://github.com/apache/spark/pull/28848#issuecomment-645701890


   **[Test build #124184 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124184/testReport)**
 for PR 28848 at commit 
[`4c0b98c`](https://github.com/apache/spark/commit/4c0b98cc30cefb9745fa1c1232022bfcb55077cd).



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA commented on pull request #28848: [SPARK-32003][CORE] Unregister outputs for executor on fetch failure …

2020-06-17 Thread GitBox


SparkQA commented on pull request #28848:
URL: https://github.com/apache/spark/pull/28848#issuecomment-645747557


   **[Test build #124184 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124184/testReport)**
 for PR 28848 at commit 
[`4c0b98c`](https://github.com/apache/spark/commit/4c0b98cc30cefb9745fa1c1232022bfcb55077cd).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins commented on pull request #28853: [SPARK-32019][SQL] Add spark.sql.files.minPartitionNum config

2020-06-17 Thread GitBox


AmplabJenkins commented on pull request #28853:
URL: https://github.com/apache/spark/pull/28853#issuecomment-645747457







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #28853: [SPARK-32019][SQL] Add spark.sql.files.minPartitionNum config

2020-06-17 Thread GitBox


AmplabJenkins removed a comment on pull request #28853:
URL: https://github.com/apache/spark/pull/28853#issuecomment-645747457







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA commented on pull request #28853: [SPARK-32019][SQL] Add spark.sql.files.minPartitionNum config

2020-06-17 Thread GitBox


SparkQA commented on pull request #28853:
URL: https://github.com/apache/spark/pull/28853#issuecomment-645747063


   **[Test build #124195 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124195/testReport)**
 for PR 28853 at commit 
[`09bba5c`](https://github.com/apache/spark/commit/09bba5cb67be22aff62956e22c1f999de1ccf547).



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #28123: [SPARK-31350][SQL] Coalesce bucketed tables for sort merge join if applicable

2020-06-17 Thread GitBox


AmplabJenkins removed a comment on pull request #28123:
URL: https://github.com/apache/spark/pull/28123#issuecomment-645743374







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins commented on pull request #28123: [SPARK-31350][SQL] Coalesce bucketed tables for sort merge join if applicable

2020-06-17 Thread GitBox


AmplabJenkins commented on pull request #28123:
URL: https://github.com/apache/spark/pull/28123#issuecomment-645743374







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #28825: [SPARK-32020][SQL][MINOR] Better error message when SPARK_HOME or spark.test.home is not set.

2020-06-17 Thread GitBox


AmplabJenkins removed a comment on pull request #28825:
URL: https://github.com/apache/spark/pull/28825#issuecomment-645743276







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins commented on pull request #28825: [SPARK-32020][SQL][MINOR] Better error message when SPARK_HOME or spark.test.home is not set.

2020-06-17 Thread GitBox


AmplabJenkins commented on pull request #28825:
URL: https://github.com/apache/spark/pull/28825#issuecomment-645743276







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA commented on pull request #28123: [SPARK-31350][SQL] Coalesce bucketed tables for sort merge join if applicable

2020-06-17 Thread GitBox


SparkQA commented on pull request #28123:
URL: https://github.com/apache/spark/pull/28123#issuecomment-645742893


   **[Test build #124194 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124194/testReport)**
 for PR 28123 at commit 
[`e231268`](https://github.com/apache/spark/commit/e23126816cf9e9d538479c389c32b87708d246e8).



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA commented on pull request #28825: [SPARK-32020][SQL][MINOR] Better error message when SPARK_HOME or spark.test.home is not set.

2020-06-17 Thread GitBox


SparkQA commented on pull request #28825:
URL: https://github.com/apache/spark/pull/28825#issuecomment-645742891


   **[Test build #124193 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124193/testReport)**
 for PR 28825 at commit 
[`c3bb548`](https://github.com/apache/spark/commit/c3bb54846f2b31977924f436ef60e1ddfffe1392).



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #28853: [SPARK-32019][SQL] Add spark.sql.files.minPartitionNum config

2020-06-17 Thread GitBox


AmplabJenkins removed a comment on pull request #28853:
URL: https://github.com/apache/spark/pull/28853#issuecomment-645737311


   Test FAILed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/124191/
   Test FAILed.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #28853: [SPARK-32019][SQL] Add spark.sql.files.minPartitionNum config

2020-06-17 Thread GitBox


AmplabJenkins removed a comment on pull request #28853:
URL: https://github.com/apache/spark/pull/28853#issuecomment-645737306


   Merged build finished. Test FAILed.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA removed a comment on pull request #28853: [SPARK-32019][SQL] Add spark.sql.files.minPartitionNum config

2020-06-17 Thread GitBox


SparkQA removed a comment on pull request #28853:
URL: https://github.com/apache/spark/pull/28853#issuecomment-645736601


   **[Test build #124191 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124191/testReport)**
 for PR 28853 at commit 
[`4d776d3`](https://github.com/apache/spark/commit/4d776d3957be629822b61bcff7741e97aa681bcd).



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #28853: [SPARK-32019][SQL] Add spark.sql.files.minPartitionNum config

2020-06-17 Thread GitBox


AmplabJenkins removed a comment on pull request #28853:
URL: https://github.com/apache/spark/pull/28853#issuecomment-645736934







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #28840: [SPARK-31999][SQL] Add refresh function command

2020-06-17 Thread GitBox


AmplabJenkins removed a comment on pull request #28840:
URL: https://github.com/apache/spark/pull/28840#issuecomment-645736929







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins commented on pull request #28853: [SPARK-32019][SQL] Add spark.sql.files.minPartitionNum config

2020-06-17 Thread GitBox


AmplabJenkins commented on pull request #28853:
URL: https://github.com/apache/spark/pull/28853#issuecomment-645737306







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA commented on pull request #28853: [SPARK-32019][SQL] Add spark.sql.files.minPartitionNum config

2020-06-17 Thread GitBox


SparkQA commented on pull request #28853:
URL: https://github.com/apache/spark/pull/28853#issuecomment-645737293


   **[Test build #124191 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124191/testReport)**
 for PR 28853 at commit 
[`4d776d3`](https://github.com/apache/spark/commit/4d776d3957be629822b61bcff7741e97aa681bcd).
* This patch **fails Scala style tests**.
* This patch merges cleanly.
* This patch adds no public classes.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins commented on pull request #28840: [SPARK-31999][SQL] Add refresh function command

2020-06-17 Thread GitBox


AmplabJenkins commented on pull request #28840:
URL: https://github.com/apache/spark/pull/28840#issuecomment-645736929







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins commented on pull request #28853: [SPARK-32019][SQL] Add spark.sql.files.minPartitionNum config

2020-06-17 Thread GitBox


AmplabJenkins commented on pull request #28853:
URL: https://github.com/apache/spark/pull/28853#issuecomment-645736934







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA commented on pull request #28853: [SPARK-32019][SQL] Add spark.sql.files.minPartitionNum config

2020-06-17 Thread GitBox


SparkQA commented on pull request #28853:
URL: https://github.com/apache/spark/pull/28853#issuecomment-645736601


   **[Test build #124191 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124191/testReport)**
 for PR 28853 at commit 
[`4d776d3`](https://github.com/apache/spark/commit/4d776d3957be629822b61bcff7741e97aa681bcd).



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA commented on pull request #28840: [SPARK-31999][SQL] Add refresh function command

2020-06-17 Thread GitBox


SparkQA commented on pull request #28840:
URL: https://github.com/apache/spark/pull/28840#issuecomment-645736604


   **[Test build #124192 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124192/testReport)**
 for PR 28840 at commit 
[`e444943`](https://github.com/apache/spark/commit/e4449430fecd029606c4d48a0d8c82c887704649).



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #28804: [SPARK-31973][SQL] Add ability to disable Sort,Spill in Partial aggregation

2020-06-17 Thread GitBox


AmplabJenkins removed a comment on pull request #28804:
URL: https://github.com/apache/spark/pull/28804#issuecomment-645734522


   Test FAILed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/124189/
   Test FAILed.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA removed a comment on pull request #28804: [SPARK-31973][SQL] Add ability to disable Sort,Spill in Partial aggregation

2020-06-17 Thread GitBox


SparkQA removed a comment on pull request #28804:
URL: https://github.com/apache/spark/pull/28804#issuecomment-645730148


   **[Test build #124189 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124189/testReport)**
 for PR 28804 at commit 
[`5f05aa7`](https://github.com/apache/spark/commit/5f05aa7e2631a56b5cc97206f2799f64151b1789).



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #28804: [SPARK-31973][SQL] Add ability to disable Sort,Spill in Partial aggregation

2020-06-17 Thread GitBox


AmplabJenkins removed a comment on pull request #28804:
URL: https://github.com/apache/spark/pull/28804#issuecomment-645734520


   Merged build finished. Test FAILed.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA commented on pull request #28804: [SPARK-31973][SQL] Add ability to disable Sort,Spill in Partial aggregation

2020-06-17 Thread GitBox


SparkQA commented on pull request #28804:
URL: https://github.com/apache/spark/pull/28804#issuecomment-645734498


   **[Test build #124189 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124189/testReport)**
 for PR 28804 at commit 
[`5f05aa7`](https://github.com/apache/spark/commit/5f05aa7e2631a56b5cc97206f2799f64151b1789).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds no public classes.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins commented on pull request #28804: [SPARK-31973][SQL] Add ability to disable Sort,Spill in Partial aggregation

2020-06-17 Thread GitBox


AmplabJenkins commented on pull request #28804:
URL: https://github.com/apache/spark/pull/28804#issuecomment-645734520







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #28825: [SPARk-31950][SQL][FOLLOW-UP][MINOR] Better error message on SPARK_HOME or…

2020-06-17 Thread GitBox


AmplabJenkins removed a comment on pull request #28825:
URL: https://github.com/apache/spark/pull/28825#issuecomment-645732637







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #28840: [SPARK-31999][SQL] Add refresh function command

2020-06-17 Thread GitBox


AmplabJenkins removed a comment on pull request #28840:
URL: https://github.com/apache/spark/pull/28840#issuecomment-645732187


   Test FAILed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/124185/
   Test FAILed.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins commented on pull request #28825: [SPARk-31950][SQL][FOLLOW-UP][MINOR] Better error message on SPARK_HOME or…

2020-06-17 Thread GitBox


AmplabJenkins commented on pull request #28825:
URL: https://github.com/apache/spark/pull/28825#issuecomment-645732637







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #28840: [SPARK-31999][SQL] Add refresh function command

2020-06-17 Thread GitBox


AmplabJenkins removed a comment on pull request #28840:
URL: https://github.com/apache/spark/pull/28840#issuecomment-645732179


   Merged build finished. Test FAILed.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA commented on pull request #28825: [SPARk-31950][SQL][FOLLOW-UP][MINOR] Better error message on SPARK_HOME or…

2020-06-17 Thread GitBox


SparkQA commented on pull request #28825:
URL: https://github.com/apache/spark/pull/28825#issuecomment-645732367


   **[Test build #124190 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124190/testReport)**
 for PR 28825 at commit 
[`d70a2af`](https://github.com/apache/spark/commit/d70a2aff630ab90797bf90d235d56437b25b2ea1).



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA removed a comment on pull request #28840: [SPARK-31999][SQL] Add refresh function command

2020-06-17 Thread GitBox


SparkQA removed a comment on pull request #28840:
URL: https://github.com/apache/spark/pull/28840#issuecomment-645707597


   **[Test build #124185 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124185/testReport)**
 for PR 28840 at commit 
[`f83fd8b`](https://github.com/apache/spark/commit/f83fd8b8b2f0badc97c8e85c5707b2efc9763952).



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA commented on pull request #28840: [SPARK-31999][SQL] Add refresh function command

2020-06-17 Thread GitBox


SparkQA commented on pull request #28840:
URL: https://github.com/apache/spark/pull/28840#issuecomment-645732026


   **[Test build #124185 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124185/testReport)**
 for PR 28840 at commit 
[`f83fd8b`](https://github.com/apache/spark/commit/f83fd8b8b2f0badc97c8e85c5707b2efc9763952).
* This patch **fails Spark unit tests**.
* This patch merges cleanly.
* This patch adds the following public classes _(experimental)_:
 * `case class RefreshFunction(child: LogicalPlan) extends Command`



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins commented on pull request #28840: [SPARK-31999][SQL] Add refresh function command

2020-06-17 Thread GitBox


AmplabJenkins commented on pull request #28840:
URL: https://github.com/apache/spark/pull/28840#issuecomment-645732179







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #28853: [SPARK-32019][SQL] Add spark.sql.files.minPartitionNum config

2020-06-17 Thread GitBox


AmplabJenkins removed a comment on pull request #28853:
URL: https://github.com/apache/spark/pull/28853#issuecomment-645730963


   Test FAILed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/124188/
   Test FAILed.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] HyukjinKwon commented on a change in pull request #28825: [SPARk-31950][SQL][FOLLOW-UP][MINOR] Better error message on SPARK_HOME or…

2020-06-17 Thread GitBox


HyukjinKwon commented on a change in pull request #28825:
URL: https://github.com/apache/spark/pull/28825#discussion_r441933345



##
File path: 
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/plans/SQLHelper.scala
##
@@ -83,4 +85,11 @@ trait SQLHelper {
   }
 }
   }
+
+  def getSparkHome(): String = {

Review comment:
   I think you could even make it a `lazy val` can call it `sparkHome` I 
guess but shouldn't be a big deal.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA removed a comment on pull request #28853: [SPARK-32019][SQL] Add spark.sql.files.minPartitionNum config

2020-06-17 Thread GitBox


SparkQA removed a comment on pull request #28853:
URL: https://github.com/apache/spark/pull/28853#issuecomment-645730147


   **[Test build #124188 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124188/testReport)**
 for PR 28853 at commit 
[`8ef9f29`](https://github.com/apache/spark/commit/8ef9f29c281073d07d18ef6a539da00f28aa8bd9).



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #28853: [SPARK-32019][SQL] Add spark.sql.files.minPartitionNum config

2020-06-17 Thread GitBox


AmplabJenkins removed a comment on pull request #28853:
URL: https://github.com/apache/spark/pull/28853#issuecomment-645730958


   Merged build finished. Test FAILed.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] HyukjinKwon commented on a change in pull request #28825: [SPARk-31950][SQL][FOLLOW-UP][MINOR] Better error message on SPARK_HOME or…

2020-06-17 Thread GitBox


HyukjinKwon commented on a change in pull request #28825:
URL: https://github.com/apache/spark/pull/28825#discussion_r441932959



##
File path: 
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/plans/SQLHelper.scala
##
@@ -21,6 +21,8 @@ import java.time.ZoneId
 
 import scala.util.control.NonFatal
 
+import org.scalatest.Assertions._

Review comment:
   We could probably explicitly import what we need.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] maropu commented on a change in pull request #28804: [SPARK-31973][SQL] Add ability to disable Sort,Spill in Partial aggregation

2020-06-17 Thread GitBox


maropu commented on a change in pull request #28804:
URL: https://github.com/apache/spark/pull/28804#discussion_r441932845



##
File path: 
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala
##
@@ -2173,6 +2173,13 @@ object SQLConf {
   .checkValue(bit => bit >= 10 && bit <= 30, "The bit value must be in 
[10, 30].")
   .createWithDefault(16)
 
+  val SPILL_PARTIAL_AGGREGATE_DISABLED =
+buildConf("spark.sql.aggregate.spill.partialaggregate.disabled")

Review comment:
   `disabled` -> `enabled` to follow the other config naming.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA commented on pull request #28853: [SPARK-32019][SQL] Add spark.sql.files.minPartitionNum config

2020-06-17 Thread GitBox


SparkQA commented on pull request #28853:
URL: https://github.com/apache/spark/pull/28853#issuecomment-645730946


   **[Test build #124188 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124188/testReport)**
 for PR 28853 at commit 
[`8ef9f29`](https://github.com/apache/spark/commit/8ef9f29c281073d07d18ef6a539da00f28aa8bd9).
* This patch **fails Scala style tests**.
* This patch merges cleanly.
* This patch adds no public classes.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #28804: [SPARK-31973][SQL] Add ability to disable Sort,Spill in Partial aggregation

2020-06-17 Thread GitBox


AmplabJenkins removed a comment on pull request #28804:
URL: https://github.com/apache/spark/pull/28804#issuecomment-645730604







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins commented on pull request #28853: [SPARK-32019][SQL] Add spark.sql.files.minPartitionNum config

2020-06-17 Thread GitBox


AmplabJenkins commented on pull request #28853:
URL: https://github.com/apache/spark/pull/28853#issuecomment-645730958







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] HyukjinKwon commented on a change in pull request #28825: [SPARk-31950][SQL][FOLLOW-UP][MINOR] Better error message on SPARK_HOME or…

2020-06-17 Thread GitBox


HyukjinKwon commented on a change in pull request #28825:
URL: https://github.com/apache/spark/pull/28825#discussion_r441932726



##
File path: 
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/parser/TableIdentifierParserSuite.scala
##
@@ -26,7 +24,7 @@ import org.apache.spark.sql.catalyst.plans.SQLHelper
 import org.apache.spark.sql.catalyst.util.fileToString
 import org.apache.spark.sql.internal.SQLConf
 
-class TableIdentifierParserSuite extends SparkFunSuite with SQLHelper {
+class TableIdentifierParserSuite extends SparkFunSuite with SQLHelper  {

Review comment:
   nit there's one more space here ..





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins commented on pull request #28853: [SPARK-32019][SQL] Add spark.sql.files.minPartitionNum config

2020-06-17 Thread GitBox


AmplabJenkins commented on pull request #28853:
URL: https://github.com/apache/spark/pull/28853#issuecomment-645730525







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins commented on pull request #28804: [SPARK-31973][SQL] Add ability to disable Sort,Spill in Partial aggregation

2020-06-17 Thread GitBox


AmplabJenkins commented on pull request #28804:
URL: https://github.com/apache/spark/pull/28804#issuecomment-645730604







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] maropu commented on a change in pull request #28804: [SPARK-31973][SQL] Add ability to disable Sort,Spill in Partial aggregation

2020-06-17 Thread GitBox


maropu commented on a change in pull request #28804:
URL: https://github.com/apache/spark/pull/28804#discussion_r441932524



##
File path: 
sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/HashAggregateExec.scala
##
@@ -72,6 +74,8 @@ case class HashAggregateExec(
 "peakMemory" -> SQLMetrics.createSizeMetric(sparkContext, "peak memory"),
 "spillSize" -> SQLMetrics.createSizeMetric(sparkContext, "spill size"),
 "aggTime" -> SQLMetrics.createTimingMetric(sparkContext, "time in 
aggregation build"),
+"partialAggSkipped" -> SQLMetrics.createMetric(sparkContext, "Num records" 
+
+  " skipped partial aggregation skipped"),

Review comment:
   skipped ... skipped? How about `number of skipped records for partial 
aggregates`?





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #28853: [SPARK-32019][SQL] Add spark.sql.files.minPartitionNum config

2020-06-17 Thread GitBox


AmplabJenkins removed a comment on pull request #28853:
URL: https://github.com/apache/spark/pull/28853#issuecomment-645730525







This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] ulysses-you commented on pull request #28853: [SPARK-32019][SQL] Add spark.sql.files.minPartitionNum config

2020-06-17 Thread GitBox


ulysses-you commented on pull request #28853:
URL: https://github.com/apache/spark/pull/28853#issuecomment-645730484


   cc @cloud-fan  @maropu 



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA commented on pull request #28853: [SPARK-32019][SQL] Add spark.sql.files.minPartitionNum config

2020-06-17 Thread GitBox


SparkQA commented on pull request #28853:
URL: https://github.com/apache/spark/pull/28853#issuecomment-645730147


   **[Test build #124188 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124188/testReport)**
 for PR 28853 at commit 
[`8ef9f29`](https://github.com/apache/spark/commit/8ef9f29c281073d07d18ef6a539da00f28aa8bd9).



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA commented on pull request #28804: [SPARK-31973][SQL] Add ability to disable Sort,Spill in Partial aggregation

2020-06-17 Thread GitBox


SparkQA commented on pull request #28804:
URL: https://github.com/apache/spark/pull/28804#issuecomment-645730148


   **[Test build #124189 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124189/testReport)**
 for PR 28804 at commit 
[`5f05aa7`](https://github.com/apache/spark/commit/5f05aa7e2631a56b5cc97206f2799f64151b1789).



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #28840: [SPARK-31999][SQL] Add refresh function command

2020-06-17 Thread GitBox


AmplabJenkins removed a comment on pull request #28840:
URL: https://github.com/apache/spark/pull/28840#issuecomment-645729242


   Test FAILed.
   Refer to this link for build results (access rights to CI server needed): 
   https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/124183/
   Test FAILed.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] ulysses-you opened a new pull request #28853: [SPARK-32019][SQL] Add spark.sql.files.minPartitionNum config

2020-06-17 Thread GitBox


ulysses-you opened a new pull request #28853:
URL: https://github.com/apache/spark/pull/28853


   
   
   ### What changes were proposed in this pull request?
   
   Add a new config `spark.sql.files.minPartitionNum` to control file split 
partition in local session.
   
   ### Why are the changes needed?
   
   Aims to control file split partitions in session level.
   More details see discuss in 
[PR-28778](https://github.com/apache/spark/pull/28778).
   
   ### Does this PR introduce _any_ user-facing change?
   
   Yes, new config.
   
   ### How was this patch tested?
   
   Add UT.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #28840: [SPARK-31999][SQL] Add refresh function command

2020-06-17 Thread GitBox


AmplabJenkins removed a comment on pull request #28840:
URL: https://github.com/apache/spark/pull/28840#issuecomment-645729235


   Merged build finished. Test FAILed.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] SparkQA removed a comment on pull request #28840: [SPARK-31999][SQL] Add refresh function command

2020-06-17 Thread GitBox


SparkQA removed a comment on pull request #28840:
URL: https://github.com/apache/spark/pull/28840#issuecomment-645698051


   **[Test build #124183 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124183/testReport)**
 for PR 28840 at commit 
[`c434821`](https://github.com/apache/spark/commit/c434821ddb1d7f335f0ede1e8b3802617b918fe2).



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



[GitHub] [spark] AmplabJenkins removed a comment on pull request #28804: [SPARK-31973][SQL] Add ability to disable Sort,Spill in Partial aggregation

2020-06-17 Thread GitBox


AmplabJenkins removed a comment on pull request #28804:
URL: https://github.com/apache/spark/pull/28804#issuecomment-642999484


   Can one of the admins verify this patch?



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org



  1   2   3   4   5   >