[GitHub] [spark] maropu commented on pull request #28804: [SPARK-31973][SQL] Skip partial aggregates if grouping keys have high cardinality

maropu commented on pull request #28804: URL: https://github.com/apache/spark/pull/28804#issuecomment-673478644 > I don't think AQE can help here. This is partial aggregate and usually there won't be a shuffle right before the partial agg. Hm, I see. Even so, `BasicStatsPlanVisitor`

[GitHub] [spark] HyukjinKwon commented on pull request #29333: [WIP][SPARK-32357][INFRA] Publish failed and succeeded test reports in GitHub Actions

HyukjinKwon commented on pull request #29333: URL: https://github.com/apache/spark/pull/29333#issuecomment-673477836 https://github.com/HyukjinKwon/spark/runs/980576471 should show the example of this PR. Before this PR is merged, the test results would not be reported because the

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29270: [SPARK-32466][TEST][SQL] Add PlanStabilitySuite to detect SparkPlan regression

AmplabJenkins removed a comment on pull request #29270: URL: https://github.com/apache/spark/pull/29270#issuecomment-673477444 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29270: [SPARK-32466][TEST][SQL] Add PlanStabilitySuite to detect SparkPlan regression

AmplabJenkins commented on pull request #29270: URL: https://github.com/apache/spark/pull/29270#issuecomment-673477444 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29270: [SPARK-32466][TEST][SQL] Add PlanStabilitySuite to detect SparkPlan regression

SparkQA commented on pull request #29270: URL: https://github.com/apache/spark/pull/29270#issuecomment-673476667 **[Test build #127412 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127412/testReport)** for PR 29270 at commit

[GitHub] [spark] abhishekd0907 commented on a change in pull request #29242: [SPARK-31448] [PYTHON] Fix storage level used in cache() in dataframe.py

abhishekd0907 commented on a change in pull request #29242: URL: https://github.com/apache/spark/pull/29242#discussion_r469946934 ## File path: python/pyspark/sql/dataframe.py ## @@ -674,7 +674,7 @@ def cache(self): .. note:: The default storage level has changed to

[GitHub] [spark] itskals commented on pull request #29413: [SPARK-32597][CORE] Tune Event Drop in Async Event Queue

itskals commented on pull request #29413: URL: https://github.com/apache/spark/pull/29413#issuecomment-673475255 I was thinking that though spark has many queues and in many cases may be not all queues are used used to the same level at the same time.. I mean, when some queues are

[GitHub] [spark] tgravescs commented on pull request #29413: [SPARK-32597][CORE] Tune Event Drop in Async Event Queue

tgravescs commented on pull request #29413: URL: https://github.com/apache/spark/pull/29413#issuecomment-673474159 > We can think of making it per queue basis any such approach based on the user requirement like I have already done in this PR using the conf(spark.set.optmized.event.queue)

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29360: [SPARK-32542][SQL]Add a Batch in Optimizer to improve performance in multidimensional analysis

AmplabJenkins removed a comment on pull request #29360: URL: https://github.com/apache/spark/pull/29360#issuecomment-673469505 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29360: [SPARK-32542][SQL]Add a Batch in Optimizer to improve performance in multidimensional analysis

AmplabJenkins commented on pull request #29360: URL: https://github.com/apache/spark/pull/29360#issuecomment-673469505 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #29360: [SPARK-32542][SQL]Add a Batch in Optimizer to improve performance in multidimensional analysis

SparkQA removed a comment on pull request #29360: URL: https://github.com/apache/spark/pull/29360#issuecomment-673352065 **[Test build #127404 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127404/testReport)** for PR 29360 at commit

[GitHub] [spark] SparkQA commented on pull request #29360: [SPARK-32542][SQL]Add a Batch in Optimizer to improve performance in multidimensional analysis

SparkQA commented on pull request #29360: URL: https://github.com/apache/spark/pull/29360#issuecomment-673468488 **[Test build #127404 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127404/testReport)** for PR 29360 at commit

[GitHub] [spark] cloud-fan commented on pull request #28804: [SPARK-31973][SQL] Skip partial aggregates if grouping keys have high cardinality

cloud-fan commented on pull request #28804: URL: https://github.com/apache/spark/pull/28804#issuecomment-673464902 I don't think AQE can help here. This is partial aggregate and usually there won't be a shuffle right before the partial agg.

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29414: [SPARK-32106][SQL] Implement script transform in sql/core

AmplabJenkins removed a comment on pull request #29414: URL: https://github.com/apache/spark/pull/29414#issuecomment-673460970 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29414: [SPARK-32106][SQL] Implement script transform in sql/core

AmplabJenkins commented on pull request #29414: URL: https://github.com/apache/spark/pull/29414#issuecomment-673460970 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] mundaym commented on a change in pull request #29407: [SPARK-32588][CORE][TEST] Fix SizeEstimator initialization in tests

mundaym commented on a change in pull request #29407: URL: https://github.com/apache/spark/pull/29407#discussion_r469928734 ## File path: core/src/test/scala/org/apache/spark/util/SizeEstimatorSuite.scala ## @@ -214,6 +216,10 @@ class SizeEstimatorSuite } test("class

[GitHub] [spark] SparkQA removed a comment on pull request #29414: [SPARK-32106][SQL] Implement script transform in sql/core

SparkQA removed a comment on pull request #29414: URL: https://github.com/apache/spark/pull/29414#issuecomment-673341123 **[Test build #127402 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127402/testReport)** for PR 29414 at commit

[GitHub] [spark] maropu commented on pull request #28804: [SPARK-31973][SQL] Skip partial aggregates if grouping keys have high cardinality

maropu commented on pull request #28804: URL: https://github.com/apache/spark/pull/28804#issuecomment-673460228 btw, could this optimization be implemented on the adaptive execution framework (`AdaptiveSparkPlanExec`)? In the initial discussion

[GitHub] [spark] SparkQA commented on pull request #29414: [SPARK-32106][SQL] Implement script transform in sql/core

SparkQA commented on pull request #29414: URL: https://github.com/apache/spark/pull/29414#issuecomment-673459937 **[Test build #127402 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127402/testReport)** for PR 29414 at commit

[GitHub] [spark] cloud-fan commented on a change in pull request #28490: [SPARK-31670][SQL]Resolve Struct Field in Grouping Aggregate with same ExprId

cloud-fan commented on a change in pull request #28490: URL: https://github.com/apache/spark/pull/28490#discussion_r469923244 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala ## @@ -1479,6 +1479,33 @@ class Analyzer( //

[GitHub] [spark] mundaym commented on pull request #29407: [SPARK-32588][CORE][TEST] Fix SizeEstimator initialization in tests

mundaym commented on pull request #29407: URL: https://github.com/apache/spark/pull/29407#issuecomment-673453911 > One related question outside the scope of this PR: since some system properties affect the behavior of the SizeEstimator, should the test suites that re-initialize

[GitHub] [spark] srowen commented on a change in pull request #27369: [SPARK-30654] Bootstrap4 docs upgrade

srowen commented on a change in pull request #27369: URL: https://github.com/apache/spark/pull/27369#discussion_r469920089 ## File path: docs/_layouts/global.html ## @@ -20,12 +20,15 @@

[GitHub] [spark] cloud-fan commented on a change in pull request #28685: [SPARK-27951][SQL] Support ANSI SQL NTH_VALUE window function

cloud-fan commented on a change in pull request #28685: URL: https://github.com/apache/spark/pull/28685#discussion_r469920074 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/window/WindowExecBase.scala ## @@ -137,7 +137,11 @@ trait WindowExecBase extends

[GitHub] [spark] cloud-fan commented on a change in pull request #28685: [SPARK-27951][SQL] Support ANSI SQL NTH_VALUE window function

cloud-fan commented on a change in pull request #28685: URL: https://github.com/apache/spark/pull/28685#discussion_r469919096 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/windowExpressions.scala ## @@ -474,6 +479,55 @@ case class

[GitHub] [spark] cloud-fan commented on a change in pull request #28685: [SPARK-27951][SQL] Support ANSI SQL NTH_VALUE window function

cloud-fan commented on a change in pull request #28685: URL: https://github.com/apache/spark/pull/28685#discussion_r469919096 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/windowExpressions.scala ## @@ -474,6 +479,55 @@ case class

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #29421: [SPARK-32388][SQL] TRANSFORM with schema-less mode should keep the same with hive

AngersZh commented on a change in pull request #29421: URL: https://github.com/apache/spark/pull/29421#discussion_r469918794 ## File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/HiveScriptTransformationExec.scala ## @@ -274,6 +275,14 @@ object

[GitHub] [spark] cloud-fan commented on a change in pull request #28685: [SPARK-27951][SQL] Support ANSI SQL NTH_VALUE window function

cloud-fan commented on a change in pull request #28685: URL: https://github.com/apache/spark/pull/28685#discussion_r469916712 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/windowExpressions.scala ## @@ -474,6 +479,55 @@ case class

[GitHub] [spark] cloud-fan commented on a change in pull request #28685: [SPARK-27951][SQL] Support ANSI SQL NTH_VALUE window function

cloud-fan commented on a change in pull request #28685: URL: https://github.com/apache/spark/pull/28685#discussion_r469916064 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/windowExpressions.scala ## @@ -474,6 +479,55 @@ case class

[GitHub] [spark] cloud-fan commented on a change in pull request #29423: [SPARK-20680][SQL][FOLLOW-UP] Add HiveVoidType in HiveClientImpl

cloud-fan commented on a change in pull request #29423: URL: https://github.com/apache/spark/pull/29423#discussion_r469914410 ## File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveSQLViewSuite.scala ## @@ -137,4 +137,25 @@ class HiveSQLViewSuite

[GitHub] [spark] cloud-fan commented on a change in pull request #29423: [SPARK-20680][SQL][FOLLOW-UP] Add HiveVoidType in HiveClientImpl

cloud-fan commented on a change in pull request #29423: URL: https://github.com/apache/spark/pull/29423#discussion_r469913614 ## File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala ## @@ -1005,7 +1006,8 @@ private[hive] object

[GitHub] [spark] cloud-fan commented on a change in pull request #29342: [SPARK-32399][SQL] Full outer shuffled hash join

cloud-fan commented on a change in pull request #29342: URL: https://github.com/apache/spark/pull/29342#discussion_r469912478 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/ShuffledHashJoinExec.scala ## @@ -71,8 +85,210 @@ case class

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29333: [WIP][SPARK-32357][INFRA] Publish failed and succeeded test reports in GitHub Actions

AmplabJenkins removed a comment on pull request #29333: URL: https://github.com/apache/spark/pull/29333#issuecomment-673441781 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29333: [WIP][SPARK-32357][INFRA] Publish failed and succeeded test reports in GitHub Actions

AmplabJenkins removed a comment on pull request #29333: URL: https://github.com/apache/spark/pull/29333#issuecomment-673439994 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins commented on pull request #29333: [WIP][SPARK-32357][INFRA] Publish failed and succeeded test reports in GitHub Actions

AmplabJenkins commented on pull request #29333: URL: https://github.com/apache/spark/pull/29333#issuecomment-673441781 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29333: [WIP][SPARK-32357][INFRA] Publish failed and succeeded test reports in GitHub Actions

SparkQA commented on pull request #29333: URL: https://github.com/apache/spark/pull/29333#issuecomment-673441231 **[Test build #127411 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127411/testReport)** for PR 29333 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29333: [WIP][SPARK-32357][INFRA] Publish failed and succeeded test reports in GitHub Actions

AmplabJenkins removed a comment on pull request #29333: URL: https://github.com/apache/spark/pull/29333#issuecomment-673439866 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29333: [WIP][SPARK-32357][INFRA] Publish failed and succeeded test reports in GitHub Actions

AmplabJenkins commented on pull request #29333: URL: https://github.com/apache/spark/pull/29333#issuecomment-673439989 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #29333: [WIP][SPARK-32357][INFRA] Publish failed and succeeded test reports in GitHub Actions

AmplabJenkins commented on pull request #29333: URL: https://github.com/apache/spark/pull/29333#issuecomment-673439845 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29333: [WIP][SPARK-32357][INFRA] Publish failed and succeeded test reports in GitHub Actions

AmplabJenkins removed a comment on pull request #29333: URL: https://github.com/apache/spark/pull/29333#issuecomment-673439845 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] maropu commented on a change in pull request #29414: [SPARK-32106][SQL] Implement script transform in sql/core

maropu commented on a change in pull request #29414: URL: https://github.com/apache/spark/pull/29414#discussion_r469898576 ## File path: sql/core/src/test/resources/sql-tests/results/transform.sql.out ## @@ -0,0 +1,224 @@ +-- Automatically generated by SQLQueryTestSuite +--

[GitHub] [spark] maropu commented on a change in pull request #29414: [SPARK-32106][SQL] Implement script transform in sql/core

maropu commented on a change in pull request #29414: URL: https://github.com/apache/spark/pull/29414#discussion_r469898075 ## File path: sql/core/src/test/resources/sql-tests/results/transform.sql.out ## @@ -0,0 +1,224 @@ +-- Automatically generated by SQLQueryTestSuite +--

[GitHub] [spark] AmplabJenkins commented on pull request #29333: [WIP][SPARK-32357][INFRA] Publish failed and succeeded test reports in GitHub Actions

AmplabJenkins commented on pull request #29333: URL: https://github.com/apache/spark/pull/29333#issuecomment-673432519 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29333: [WIP][SPARK-32357][INFRA] Publish failed and succeeded test reports in GitHub Actions

AmplabJenkins removed a comment on pull request #29333: URL: https://github.com/apache/spark/pull/29333#issuecomment-673432519 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #29333: [WIP][SPARK-32357][INFRA] Publish failed and succeeded test reports in GitHub Actions

SparkQA commented on pull request #29333: URL: https://github.com/apache/spark/pull/29333#issuecomment-673432052 **[Test build #127410 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127410/testReport)** for PR 29333 at commit

[GitHub] [spark] SparkQA commented on pull request #29360: [SPARK-32542][SQL]Add a Batch in Optimizer to improve performance in multidimensional analysis

SparkQA commented on pull request #29360: URL: https://github.com/apache/spark/pull/29360#issuecomment-673432002 **[Test build #127409 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127409/testReport)** for PR 29360 at commit

[GitHub] [spark] maropu commented on a change in pull request #29421: [SPARK-32388][SQL]TRANSFORM when schema less should keep same with hive

maropu commented on a change in pull request #29421: URL: https://github.com/apache/spark/pull/29421#discussion_r469892680 ## File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveScriptTransformationSuite.scala ## @@ -182,7 +182,11 @@ class

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29360: [SPARK-32542][SQL]Add a Batch in Optimizer to improve performance in multidimensional analysis

AmplabJenkins removed a comment on pull request #29360: URL: https://github.com/apache/spark/pull/29360#issuecomment-673429679 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29360: [SPARK-32542][SQL]Add a Batch in Optimizer to improve performance in multidimensional analysis

AmplabJenkins commented on pull request #29360: URL: https://github.com/apache/spark/pull/29360#issuecomment-673429679 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] Karl-WangSK removed a comment on pull request #29360: [SPARK-32542][SQL]Add a Batch in Optimizer to improve performance in multidimensional analysis

Karl-WangSK removed a comment on pull request #29360: URL: https://github.com/apache/spark/pull/29360#issuecomment-673426929 retest this please This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] maropu commented on a change in pull request #29421: [SPARK-32388][SQL]TRANSFORM when schema less should keep same with hive

maropu commented on a change in pull request #29421: URL: https://github.com/apache/spark/pull/29421#discussion_r469889961 ## File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/HiveScriptTransformationExec.scala ## @@ -274,6 +275,14 @@ object

[GitHub] [spark] maropu commented on a change in pull request #29421: [SPARK-32388][SQL]TRANSFORM when schema less should keep same with hive

maropu commented on a change in pull request #29421: URL: https://github.com/apache/spark/pull/29421#discussion_r469889057 ## File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/HiveScriptTransformationExec.scala ## @@ -274,6 +275,14 @@ object

[GitHub] [spark] HyukjinKwon commented on a change in pull request #29333: [WIP][SPARK-32357][INFRA] Publish failed and succeeded test reports in GitHub Actions

HyukjinKwon commented on a change in pull request #29333: URL: https://github.com/apache/spark/pull/29333#discussion_r469888902 ## File path: .github/workflows/master.yml ## @@ -27,21 +26,21 @@ jobs: # Kinesis tests depends on external Amazon kinesis service.

[GitHub] [spark] maropu commented on a change in pull request #29421: [SPARK-32388][SQL]TRANSFORM when schema less should keep same with hive

maropu commented on a change in pull request #29421: URL: https://github.com/apache/spark/pull/29421#discussion_r469889057 ## File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/HiveScriptTransformationExec.scala ## @@ -274,6 +275,14 @@ object

[GitHub] [spark] Karl-WangSK commented on pull request #29360: [SPARK-32542][SQL]Add a Batch in Optimizer to improve performance in multidimensional analysis

Karl-WangSK commented on pull request #29360: URL: https://github.com/apache/spark/pull/29360#issuecomment-673426929 retest this please This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] HyukjinKwon commented on pull request #29333: [WIP][SPARK-32357][INFRA] Publish failed and succeeded test reports in GitHub Actions

HyukjinKwon commented on pull request #29333: URL: https://github.com/apache/spark/pull/29333#issuecomment-673426729 I elaborated it in the PR description how it works and why I had to fork two plugins. This is an automated

[GitHub] [spark] HyukjinKwon commented on a change in pull request #29333: [WIP][SPARK-32357][INFRA] Publish failed and succeeded test reports in GitHub Actions

HyukjinKwon commented on a change in pull request #29333: URL: https://github.com/apache/spark/pull/29333#discussion_r469888319 ## File path: .github/workflows/master.yml ## @@ -9,7 +9,6 @@ on: - master jobs: - # TODO(SPARK-32248): Recover JDK 11 builds Review

[GitHub] [spark] SparkQA commented on pull request #29423: [SPARK-20680][SQL][FOLLOW-UP] Add HiveVoidType in HiveClientImpl

SparkQA commented on pull request #29423: URL: https://github.com/apache/spark/pull/29423#issuecomment-673426253 **[Test build #127408 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127408/testReport)** for PR 29423 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29423: [SPARK-20680][SQL][FOLLOW-UP] Add HiveVoidType in HiveClientImpl

AmplabJenkins removed a comment on pull request #29423: URL: https://github.com/apache/spark/pull/29423#issuecomment-673423659 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29423: [SPARK-20680][SQL][FOLLOW-UP] Add HiveVoidType in HiveClientImpl

AmplabJenkins commented on pull request #29423: URL: https://github.com/apache/spark/pull/29423#issuecomment-673423659 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] ulysses-you commented on a change in pull request #29423: [SPARK-20680][SQL][FOLLOW-UP] Add HiveVoidType in HiveClientImpl

ulysses-you commented on a change in pull request #29423: URL: https://github.com/apache/spark/pull/29423#discussion_r469884015 ## File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveSQLViewSuite.scala ## @@ -137,4 +137,24 @@ class HiveSQLViewSuite

[GitHub] [spark] ulysses-you commented on a change in pull request #29423: [SPARK-20680][SQL][FOLLOW-UP] Add HiveVoidType in HiveClientImpl

ulysses-you commented on a change in pull request #29423: URL: https://github.com/apache/spark/pull/29423#discussion_r469883959 ## File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala ## @@ -1005,7 +1006,8 @@ private[hive] object

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29333: [WIP][SPARK-32357][INFRA] Publish failed and succeeded test reports in GitHub Actions

AmplabJenkins removed a comment on pull request #29333: URL: https://github.com/apache/spark/pull/29333#issuecomment-673417732 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29333: [WIP][SPARK-32357][INFRA] Publish failed and succeeded test reports in GitHub Actions

AmplabJenkins commented on pull request #29333: URL: https://github.com/apache/spark/pull/29333#issuecomment-673417732 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29333: [WIP][SPARK-32357][INFRA] Publish failed and succeeded test reports in GitHub Actions

SparkQA commented on pull request #29333: URL: https://github.com/apache/spark/pull/29333#issuecomment-673417245 **[Test build #127407 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127407/testReport)** for PR 29333 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #29342: [SPARK-32399][SQL] Full outer shuffled hash join

AmplabJenkins commented on pull request #29342: URL: https://github.com/apache/spark/pull/29342#issuecomment-673414173 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29342: [SPARK-32399][SQL] Full outer shuffled hash join

AmplabJenkins removed a comment on pull request #29342: URL: https://github.com/apache/spark/pull/29342#issuecomment-673414173 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA removed a comment on pull request #29342: [SPARK-32399][SQL] Full outer shuffled hash join

SparkQA removed a comment on pull request #29342: URL: https://github.com/apache/spark/pull/29342#issuecomment-673348390 **[Test build #127403 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127403/testReport)** for PR 29342 at commit

[GitHub] [spark] SparkQA commented on pull request #29342: [SPARK-32399][SQL] Full outer shuffled hash join

SparkQA commented on pull request #29342: URL: https://github.com/apache/spark/pull/29342#issuecomment-673413402 **[Test build #127403 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127403/testReport)** for PR 29342 at commit

[GitHub] [spark] SaurabhChawla100 edited a comment on pull request #29413: [SPARK-32597][CORE] Tune Event Drop in Async Event Queue

SaurabhChawla100 edited a comment on pull request #29413: URL: https://github.com/apache/spark/pull/29413#issuecomment-673398423 > sorry I'm still not seeing any difference here then increasing the size of the current queue? If both are not really allocating memory for the entire amount

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29414: [SPARK-32106][SQL] Implement script transform in sql/core

AmplabJenkins removed a comment on pull request #29414: URL: https://github.com/apache/spark/pull/29414#issuecomment-673400583 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29414: [SPARK-32106][SQL] Implement script transform in sql/core

AmplabJenkins commented on pull request #29414: URL: https://github.com/apache/spark/pull/29414#issuecomment-673400583 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29414: [SPARK-32106][SQL] Implement script transform in sql/core

SparkQA commented on pull request #29414: URL: https://github.com/apache/spark/pull/29414#issuecomment-673399976 **[Test build #127406 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127406/testReport)** for PR 29414 at commit

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #29421: [SPARK-32388][SQL]TRANSFORM when schema less should keep same with hive

AngersZh commented on a change in pull request #29421: URL: https://github.com/apache/spark/pull/29421#discussion_r469855638 ## File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/HiveScriptTransformationExec.scala ## @@ -274,6 +275,14 @@ object

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #29421: [SPARK-32388][SQL]TRANSFORM when schema less should keep same with hive

AngersZh commented on a change in pull request #29421: URL: https://github.com/apache/spark/pull/29421#discussion_r469855638 ## File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/HiveScriptTransformationExec.scala ## @@ -274,6 +275,14 @@ object

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #29414: [SPARK-32106][SQL] Implement script transform in sql/core

AngersZh commented on a change in pull request #29414: URL: https://github.com/apache/spark/pull/29414#discussion_r469854572 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkScriptTransformationExec.scala ## @@ -0,0 +1,91 @@ +/* + * Licensed to

[GitHub] [spark] SaurabhChawla100 commented on pull request #29413: [SPARK-32597][CORE] Tune Event Drop in Async Event Queue

SaurabhChawla100 commented on pull request #29413: URL: https://github.com/apache/spark/pull/29413#issuecomment-673398423 > sorry I'm still not seeing any difference here then increasing the size of the current queue? If both are not really allocating memory for the entire amount until

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #29414: [SPARK-32106][SQL] Implement script transform in sql/core

AngersZh commented on a change in pull request #29414: URL: https://github.com/apache/spark/pull/29414#discussion_r469852514 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkScriptTransformationExec.scala ## @@ -0,0 +1,91 @@ +/* + * Licensed to

[GitHub] [spark] AmplabJenkins commented on pull request #28685: [SPARK-27951][SQL] Support ANSI SQL NTH_VALUE window function

AmplabJenkins commented on pull request #28685: URL: https://github.com/apache/spark/pull/28685#issuecomment-673384776 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] maropu commented on a change in pull request #29421: [SPARK-32388][SQL]TRANSFORM when schema less should keep same with hive[

maropu commented on a change in pull request #29421: URL: https://github.com/apache/spark/pull/29421#discussion_r469838596 ## File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/HiveScriptTransformationExec.scala ## @@ -274,6 +275,14 @@ object

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28685: [SPARK-27951][SQL] Support ANSI SQL NTH_VALUE window function

AmplabJenkins removed a comment on pull request #28685: URL: https://github.com/apache/spark/pull/28685#issuecomment-673384776 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] maropu commented on a change in pull request #29414: [SPARK-32106][SQL] Implement script transform in sql/core

maropu commented on a change in pull request #29414: URL: https://github.com/apache/spark/pull/29414#discussion_r469834208 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkScriptTransformationExec.scala ## @@ -0,0 +1,91 @@ +/* + * Licensed to the

[GitHub] [spark] SparkQA commented on pull request #28685: [SPARK-27951][SQL] Support ANSI SQL NTH_VALUE window function

SparkQA commented on pull request #28685: URL: https://github.com/apache/spark/pull/28685#issuecomment-673384225 **[Test build #127405 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127405/testReport)** for PR 28685 at commit

[GitHub] [spark] Karl-WangSK removed a comment on pull request #29360: [SPARK-32542][SQL]Add a Batch in Optimizer to improve performance in multidimensional analysis

Karl-WangSK removed a comment on pull request #29360: URL: https://github.com/apache/spark/pull/29360#issuecomment-673382458 retest this please This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] Karl-WangSK commented on pull request #29360: [SPARK-32542][SQL]Add a Batch in Optimizer to improve performance in multidimensional analysis

Karl-WangSK commented on pull request #29360: URL: https://github.com/apache/spark/pull/29360#issuecomment-673382458 retest this please This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] beliefer commented on a change in pull request #28685: [SPARK-27951][SQL] Support ANSI SQL NTH_VALUE window function

beliefer commented on a change in pull request #28685: URL: https://github.com/apache/spark/pull/28685#discussion_r469835363 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/window/WindowFunctionFrame.scala ## @@ -151,10 +168,41 @@ final class

[GitHub] [spark] maropu commented on a change in pull request #29414: [SPARK-32106][SQL] Implement script transform in sql/core

maropu commented on a change in pull request #29414: URL: https://github.com/apache/spark/pull/29414#discussion_r469834208 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkScriptTransformationExec.scala ## @@ -0,0 +1,91 @@ +/* + * Licensed to the

[GitHub] [spark] maropu commented on a change in pull request #29414: [SPARK-32106][SQL] Implement script transform in sql/core

maropu commented on a change in pull request #29414: URL: https://github.com/apache/spark/pull/29414#discussion_r469833912 ## File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/HiveScriptTransformationExec.scala ## @@ -45,6 +45,8 @@ import

[GitHub] [spark] maropu commented on a change in pull request #29414: [SPARK-32106][SQL] Implement script transform in sql/core

maropu commented on a change in pull request #29414: URL: https://github.com/apache/spark/pull/29414#discussion_r469833812 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkScriptTransformationExec.scala ## @@ -0,0 +1,91 @@ +/* + * Licensed to the

[GitHub] [spark] cloud-fan commented on a change in pull request #29342: [SPARK-32399][SQL] Full outer shuffled hash join

cloud-fan commented on a change in pull request #29342: URL: https://github.com/apache/spark/pull/29342#discussion_r469825588 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/ShuffledHashJoinExec.scala ## @@ -71,8 +85,210 @@ case class

[GitHub] [spark] cloud-fan commented on a change in pull request #29342: [SPARK-32399][SQL] Full outer shuffled hash join

cloud-fan commented on a change in pull request #29342: URL: https://github.com/apache/spark/pull/29342#discussion_r469823249 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/ShuffledHashJoinExec.scala ## @@ -71,8 +85,210 @@ case class

[GitHub] [spark] cloud-fan commented on a change in pull request #29342: [SPARK-32399][SQL] Full outer shuffled hash join

cloud-fan commented on a change in pull request #29342: URL: https://github.com/apache/spark/pull/29342#discussion_r469821082 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/ShuffledHashJoinExec.scala ## @@ -71,8 +85,210 @@ case class

[GitHub] [spark] cloud-fan commented on a change in pull request #29342: [SPARK-32399][SQL] Full outer shuffled hash join

cloud-fan commented on a change in pull request #29342: URL: https://github.com/apache/spark/pull/29342#discussion_r469819019 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/ShuffledHashJoinExec.scala ## @@ -71,8 +85,210 @@ case class

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29422: [wip][testing][dnr] Decom fixes

AmplabJenkins removed a comment on pull request #29422: URL: https://github.com/apache/spark/pull/29422#issuecomment-673366843 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29422: [wip][testing][dnr] Decom fixes

AmplabJenkins removed a comment on pull request #29422: URL: https://github.com/apache/spark/pull/29422#issuecomment-673366831 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins commented on pull request #29422: [wip][testing][dnr] Decom fixes

AmplabJenkins commented on pull request #29422: URL: https://github.com/apache/spark/pull/29422#issuecomment-673366831 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #29422: [wip][testing][dnr] Decom fixes

SparkQA removed a comment on pull request #29422: URL: https://github.com/apache/spark/pull/29422#issuecomment-673313561 **[Test build #127401 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127401/testReport)** for PR 29422 at commit

[GitHub] [spark] SparkQA commented on pull request #29422: [wip][testing][dnr] Decom fixes

SparkQA commented on pull request #29422: URL: https://github.com/apache/spark/pull/29422#issuecomment-673366252 **[Test build #127401 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/127401/testReport)** for PR 29422 at commit

[GitHub] [spark] cloud-fan commented on a change in pull request #29342: [SPARK-32399][SQL] Full outer shuffled hash join

cloud-fan commented on a change in pull request #29342: URL: https://github.com/apache/spark/pull/29342#discussion_r469809836 ## File path: core/src/main/java/org/apache/spark/unsafe/map/BytesToBytesMap.java ## @@ -428,6 +428,62 @@ public MapIterator destructiveIterator() {

[GitHub] [spark] cloud-fan commented on a change in pull request #29342: [SPARK-32399][SQL] Full outer shuffled hash join

cloud-fan commented on a change in pull request #29342: URL: https://github.com/apache/spark/pull/29342#discussion_r469807522 ## File path: core/src/main/java/org/apache/spark/unsafe/map/BytesToBytesMap.java ## @@ -428,6 +428,62 @@ public MapIterator destructiveIterator() {

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29422: [wip][testing][dnr] Decom fixes

AmplabJenkins removed a comment on pull request #29422: URL: https://github.com/apache/spark/pull/29422#issuecomment-673353025 Test FAILed. Refer to this link for build results (access rights to CI server needed):

<    1   2   3   4   5   >