[GitHub] [spark] the8thC commented on a diff in pull request #40236: [SPARK-38735][SQL][TESTS] Add tests for the error class: INTERNAL_ERROR

2023-03-05 Thread via GitHub
the8thC commented on code in PR #40236: URL: https://github.com/apache/spark/pull/40236#discussion_r1125670340 ## sql/core/src/test/scala/org/apache/spark/sql/errors/QueryExecutionErrorsSuite.scala: ## @@ -765,6 +770,58 @@ class QueryExecutionErrorsSuite ) } } + +

[GitHub] [spark] LuciferYang commented on pull request #40274: [SPARK-42215][CONNECT] Simplify Scala Client IT tests

2023-03-05 Thread via GitHub
LuciferYang commented on PR #40274: URL: https://github.com/apache/spark/pull/40274#issuecomment-1455094955 On the whole, it is good for me. There is only one question. Spark still uses maven for version release and deploy. But after this pr, the E2E test change to use sbt assembly server j

[GitHub] [spark] LuciferYang opened a new pull request, #40285: [SPARK-42675][CONNECT][TESTS] Drop temp view after test `test temp view`

2023-03-05 Thread via GitHub
LuciferYang opened a new pull request, #40285: URL: https://github.com/apache/spark/pull/40285 ### What changes were proposed in this pull request? This pr just drop temp view after test which create by `test temp view` in `ClientE2ETestSuite`. ### Why are the changes needed?

[GitHub] [spark] LuciferYang commented on a diff in pull request #40278: [SPARK-42670][BUILD] Upgrade maven-surefire-plugin to 3.0.0-M9 & eliminate build warnings

2023-03-05 Thread via GitHub
LuciferYang commented on code in PR #40278: URL: https://github.com/apache/spark/pull/40278#discussion_r1125660591 ## pom.xml: ## @@ -2957,7 +2957,7 @@ ${test.java.home} -DmyKey=yourValue - + Review Comment:

[GitHub] [spark] LuciferYang commented on a diff in pull request #40278: [SPARK-42670][BUILD] Upgrade maven-surefire-plugin to 3.0.0-M9 & eliminate build warnings

2023-03-05 Thread via GitHub
LuciferYang commented on code in PR #40278: URL: https://github.com/apache/spark/pull/40278#discussion_r1125660591 ## pom.xml: ## @@ -2957,7 +2957,7 @@ ${test.java.home} -DmyKey=yourValue - + Review Comment:

[GitHub] [spark] LuciferYang commented on a diff in pull request #40278: [SPARK-42670][BUILD] Upgrade maven-surefire-plugin to 3.0.0-M9 & eliminate build warnings

2023-03-05 Thread via GitHub
LuciferYang commented on code in PR #40278: URL: https://github.com/apache/spark/pull/40278#discussion_r1125659843 ## pom.xml: ## @@ -2957,7 +2957,7 @@ ${test.java.home} -DmyKey=yourValue - + Review Comment:

[GitHub] [spark] panbingkun commented on a diff in pull request #40278: [SPARK-42670][BUILD] Upgrade maven-surefire-plugin to 3.0.0-M9 & eliminate build warnings

2023-03-05 Thread via GitHub
panbingkun commented on code in PR #40278: URL: https://github.com/apache/spark/pull/40278#discussion_r1125659474 ## pom.xml: ## @@ -2957,7 +2957,7 @@ ${test.java.home} -DmyKey=yourValue - + Review Comment:

[GitHub] [spark] LuciferYang commented on a diff in pull request #40278: [SPARK-42670][BUILD] Upgrade maven-surefire-plugin to 3.0.0-M9 & eliminate build warnings

2023-03-05 Thread via GitHub
LuciferYang commented on code in PR #40278: URL: https://github.com/apache/spark/pull/40278#discussion_r1125655202 ## pom.xml: ## @@ -2957,7 +2957,7 @@ ${test.java.home} -DmyKey=yourValue - + Review Comment:

[GitHub] [spark] beliefer commented on a diff in pull request #39091: [SPARK-41527][CONNECT][PYTHON] Implement `DataFrame.observe`

2023-03-05 Thread via GitHub
beliefer commented on code in PR #39091: URL: https://github.com/apache/spark/pull/39091#discussion_r1125655121 ## connector/connect/common/src/main/protobuf/spark/connect/relations.proto: ## @@ -598,3 +599,18 @@ message ToSchema { // The Sever side will update the dataframe

[GitHub] [spark] beliefer commented on pull request #40275: [SPARK-42557][CONNECT] Add Broadcast to functions

2023-03-05 Thread via GitHub
beliefer commented on PR #40275: URL: https://github.com/apache/spark/pull/40275#issuecomment-1455071764 @LuciferYang Thank you. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comme

[GitHub] [spark] beliefer commented on a diff in pull request #40275: [SPARK-42557][CONNECT] Add Broadcast to functions

2023-03-05 Thread via GitHub
beliefer commented on code in PR #40275: URL: https://github.com/apache/spark/pull/40275#discussion_r1125652836 ## connector/connect/client/jvm/src/main/scala/org/apache/spark/sql/functions.scala: ## @@ -1228,6 +1228,22 @@ object functions { def map_from_arrays(keys: Column,

[GitHub] [spark] beliefer commented on a diff in pull request #40275: [SPARK-42557][CONNECT] Add Broadcast to functions

2023-03-05 Thread via GitHub
beliefer commented on code in PR #40275: URL: https://github.com/apache/spark/pull/40275#discussion_r1125652564 ## connector/connect/client/jvm/src/test/scala/org/apache/spark/sql/ClientE2ETestSuite.scala: ## @@ -489,17 +489,31 @@ class ClientE2ETestSuite extends RemoteSparkSess

[GitHub] [spark] beliefer commented on pull request #40275: [SPARK-42557][CONNECT] Add Broadcast to functions

2023-03-05 Thread via GitHub
beliefer commented on PR #40275: URL: https://github.com/apache/spark/pull/40275#issuecomment-1455071084 @LuciferYang I want support the similar `withSQLConf`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

[GitHub] [spark] beliefer commented on a diff in pull request #40275: [SPARK-42557][CONNECT] Add Broadcast to functions

2023-03-05 Thread via GitHub
beliefer commented on code in PR #40275: URL: https://github.com/apache/spark/pull/40275#discussion_r1125651656 ## connector/connect/client/jvm/src/test/scala/org/apache/spark/sql/ClientE2ETestSuite.scala: ## @@ -489,17 +489,31 @@ class ClientE2ETestSuite extends RemoteSparkSess

[GitHub] [spark] LuciferYang commented on pull request #40283: [SPARK-42673][BUILD] Ban Maven 3.9.x for Spark build

2023-03-05 Thread via GitHub
LuciferYang commented on PR #40283: URL: https://github.com/apache/spark/pull/40283#issuecomment-1455043698 cc @dongjoon-hyun -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [spark] LuciferYang commented on pull request #40283: [SPARK-42673][BUILD] Ban Maven 3.9.x for Spark build

2023-03-05 Thread via GitHub
LuciferYang commented on PR #40283: URL: https://github.com/apache/spark/pull/40283#issuecomment-1455041804 no error message with this pr https://pipelines.actions.githubusercontent.com/serviceHosts/c184045e-b556-4e78-b8ef-fb37b2eda9a3/_apis/pipelines/1/runs/63900/signedlogcontent/13?urlExpi

[GitHub] [spark] LuciferYang commented on pull request #40283: [SPARK-42673][BUILD] Ban Maven 3.9.x for Spark build

2023-03-05 Thread via GitHub
LuciferYang commented on PR #40283: URL: https://github.com/apache/spark/pull/40283#issuecomment-1455028122 For check, I add a `./build/mvn -version` before in `java-11-17` GA task without this pr: https://github.com/LuciferYang/spark/actions/runs/4328951282/jobs/7559134224 And

[GitHub] [spark] LuciferYang commented on pull request #40283: [SPARK-42673][BUILD] Ban Maven 3.9.x for Spark build

2023-03-05 Thread via GitHub
LuciferYang commented on PR #40283: URL: https://github.com/apache/spark/pull/40283#issuecomment-1455027425 for example: - https://github.com/apache/spark/actions/runs/4329598884/jobs/7560252913 - https://github.com/apache/spark/actions/runs/4329598884/jobs/7560252970 - https://g

[GitHub] [spark] panbingkun opened a new pull request, #40284: [SPARK-42674][BUILD] Upgrade scalafmt from 3.7.1 to 3.7.2

2023-03-05 Thread via GitHub
panbingkun opened a new pull request, #40284: URL: https://github.com/apache/spark/pull/40284 ### What changes were proposed in this pull request? The pr aims to upgrade scalafmt from 3.7.1 to 3.7.2 ### Why are the changes needed? A. Release note: > https://github.com/scalamet

[GitHub] [spark] LuciferYang opened a new pull request, #40283: [SPARK-42673][BUILD] Ban 3.9.x for Spark Maven build

2023-03-05 Thread via GitHub
LuciferYang opened a new pull request, #40283: URL: https://github.com/apache/spark/pull/40283 ### What changes were proposed in this pull request? ### Why are the changes needed? ### Does this PR introduce _any_ user-facing change? ### How

[GitHub] [spark] HeartSaVioR commented on pull request #40273: [SPARK-42668][SS] Catch exception while trying to close compressed stream in HDFSStateStoreProvider abort

2023-03-05 Thread via GitHub
HeartSaVioR commented on PR #40273: URL: https://github.com/apache/spark/pull/40273#issuecomment-1455022742 Mind retriggering the build, please? Probably simplest way to do is pushing an empty commit. You can retrigger the build in your fork but it won't be reflected here. -- This is an

[GitHub] [spark] itholic commented on a diff in pull request #40282: [SPARK-42672][PYTHON][DOCS] Document error class list

2023-03-05 Thread via GitHub
itholic commented on code in PR #40282: URL: https://github.com/apache/spark/pull/40282#discussion_r1125617055 ## python/docs/source/development/errors.rst: ## @@ -0,0 +1,92 @@ +.. Licensed to the Apache Software Foundation (ASF) under one +or more contributor license agree

[GitHub] [spark] itholic commented on a diff in pull request #40282: [SPARK-42672][PYTHON][DOCS] Document error class list

2023-03-05 Thread via GitHub
itholic commented on code in PR #40282: URL: https://github.com/apache/spark/pull/40282#discussion_r1125617055 ## python/docs/source/development/errors.rst: ## @@ -0,0 +1,92 @@ +.. Licensed to the Apache Software Foundation (ASF) under one +or more contributor license agree

[GitHub] [spark] itholic commented on a diff in pull request #40282: [SPARK-42672][PYTHON][DOCS] Document error class list

2023-03-05 Thread via GitHub
itholic commented on code in PR #40282: URL: https://github.com/apache/spark/pull/40282#discussion_r1125617055 ## python/docs/source/development/errors.rst: ## @@ -0,0 +1,92 @@ +.. Licensed to the Apache Software Foundation (ASF) under one +or more contributor license agree

[GitHub] [spark] itholic opened a new pull request, #40282: [SPARK-42672][PYTHON][DOCS] Document error class list

2023-03-05 Thread via GitHub
itholic opened a new pull request, #40282: URL: https://github.com/apache/spark/pull/40282 ### What changes were proposed in this pull request? This PR proposes to document the PySpark error class list into a part of [Development](https://spark.apache.org/docs/latest/api/python/develo

<    1   2