[GitHub] [spark] MaxGekk commented on a diff in pull request #38507: [SPARK-40372][SQL] Migrate failures of array type checks onto error classes

MaxGekk commented on code in PR #38507: URL: https://github.com/apache/spark/pull/38507#discussion_r1014776457 ## core/src/main/resources/error/error-classes.json: ## @@ -225,14 +230,14 @@ "The should all be of type map, but it's ." ] }, - "MAP_C

[GitHub] [spark] LuciferYang opened a new pull request, #38524: [SPARK-41024][BUILD] Upgrade scala-maven-plugin to 4.7.2

LuciferYang opened a new pull request, #38524: URL: https://github.com/apache/spark/pull/38524 ### What changes were proposed in this pull request? This pr aims upgrade scala-maven-plugin to 4.7.2 ### Why are the changes needed? ### Does this PR introduce _any

[GitHub] [spark] LuciferYang opened a new pull request, #38523: [SPARK-41023][BUILD] Upgrade Jackson to 2.14.0

LuciferYang opened a new pull request, #38523: URL: https://github.com/apache/spark/pull/38523 ### What changes were proposed in this pull request? This pr aims upgrade `Jackson` related dependencies from 2.13.4 to 2.14.0. ### Why are the changes needed? After Jackson

[GitHub] [spark] MaxGekk commented on a diff in pull request #38515: [SPARK-41015][SQL][PROTOBUF] UnitTest null check for data generator

MaxGekk commented on code in PR #38515: URL: https://github.com/apache/spark/pull/38515#discussion_r1014774789 ## sql/catalyst/src/main/scala/org/apache/spark/sql/errors/QueryCompilationErrors.scala: ## @@ -3344,7 +3344,7 @@ private[sql] object QueryCompilationErrors extends Qu

[GitHub] [spark] MaxGekk commented on a diff in pull request #38522: [SPARK-41022][SQL][TESTS] Test the error class: DEFAULT_DATABASE_NOT_EXISTS, INDEX_ALREADY_EXISTS, INDEX_NOT_FOUND, ROUTINE_NOT_FOU

MaxGekk commented on code in PR #38522: URL: https://github.com/apache/spark/pull/38522#discussion_r1014774547 ## sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalogSuite.scala: ## @@ -1578,13 +1578,29 @@ abstract class SessionCatalogSuite extends An

[GitHub] [spark] MaxGekk commented on a diff in pull request #38520: [SPARK-41021][SQL][TESTS] Test some subclasses of error class DATATYPE_MISMATCH

MaxGekk commented on code in PR #38520: URL: https://github.com/apache/spark/pull/38520#discussion_r1014774113 ## sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/ExpressionTypeCheckingSuite.scala: ## @@ -745,4 +745,52 @@ class ExpressionTypeCheckingSuite exten

[GitHub] [spark] AmplabJenkins commented on pull request #38522: [SPARK-41022][SQL][TESTS] Test the error class: DEFAULT_DATABASE_NOT_EXISTS, INDEX_ALREADY_EXISTS, INDEX_NOT_FOUND, ROUTINE_NOT_FOUND

AmplabJenkins commented on PR #38522: URL: https://github.com/apache/spark/pull/38522#issuecomment-1304726977 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] HeartSaVioR commented on a diff in pull request #38503: [SPARK-40940] Remove Multi-stateful operator checkers for streaming queries.

HeartSaVioR commented on code in PR #38503: URL: https://github.com/apache/spark/pull/38503#discussion_r1014774005 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/UnsupportedOperationChecker.scala: ## @@ -157,10 +172,11 @@ object UnsupportedOperationChecker

[GitHub] [spark] MaxGekk commented on a diff in pull request #38490: [SPARK-41009][SQL] Rename the error class `_LEGACY_ERROR_TEMP_1070` to `LOCATION_ALREADY_EXISTS`

MaxGekk commented on code in PR #38490: URL: https://github.com/apache/spark/pull/38490#discussion_r1014773620 ## core/src/main/resources/error/error-classes.json: ## @@ -668,6 +668,24 @@ } } }, + "LOCATION_ALREADY_EXISTS" : { +"message" : [ + "Cannot cr

[GitHub] [spark] MaxGekk commented on pull request #38521: [SPARK-41020][SQL] Rename the error class `_LEGACY_ERROR_TEMP_1019` to `STAR_GROUP_BY_POS`

MaxGekk commented on PR #38521: URL: https://github.com/apache/spark/pull/38521#issuecomment-1304724265 @cloud-fan @srielau @itholic @LuciferYang @panbingkun Please, review this PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Git

[GitHub] [spark] panbingkun commented on pull request #38520: [SPARK-41021][SQL][TESTS] Test some subclasses of error class DATATYPE_MISMATCH

panbingkun commented on PR #38520: URL: https://github.com/apache/spark/pull/38520#issuecomment-1304705843 cc @MaxGekk -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To u

[GitHub] [spark] panbingkun opened a new pull request, #38522: [WIP][SQL][TESTS] Test the error class: DEFAULT_DATABASE_NOT_EXISTS, INDEX_ALREADY_EXISTS, INDEX_NOT_FOUND, ROUTINE_NOT_FOUND

panbingkun opened a new pull request, #38522: URL: https://github.com/apache/spark/pull/38522 ### What changes were proposed in this pull request? ### Why are the changes needed? ### Does this PR introduce _any_ user-facing change? No. ### How was this patch te

[GitHub] [spark] srielau commented on a diff in pull request #38490: [SPARK-41009][SQL] Rename the error class `_LEGACY_ERROR_TEMP_1070` to `LOCATION_ALREADY_EXISTS`

srielau commented on code in PR #38490: URL: https://github.com/apache/spark/pull/38490#discussion_r1014752823 ## core/src/main/resources/error/error-classes.json: ## @@ -668,6 +668,24 @@ } } }, + "LOCATION_ALREADY_EXISTS" : { +"message" : [ + "Cannot cr

[GitHub] [spark] srielau commented on a diff in pull request #38490: [SPARK-41009][SQL] Rename the error class `_LEGACY_ERROR_TEMP_1070` to `LOCATION_ALREADY_EXISTS`

srielau commented on code in PR #38490: URL: https://github.com/apache/spark/pull/38490#discussion_r1014752750 ## core/src/main/resources/error/error-classes.json: ## @@ -668,6 +668,24 @@ } } }, + "LOCATION_ALREADY_EXISTS" : { +"message" : [ + "Cannot cr

[GitHub] [spark] github-actions[bot] closed pull request #34637: [SPARK-37349][SQL] add SQL Rest API parsing logic

github-actions[bot] closed pull request #34637: [SPARK-37349][SQL] add SQL Rest API parsing logic URL: https://github.com/apache/spark/pull/34637 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] github-actions[bot] closed pull request #37009: [SPARK-38292][PYTHON]Support na_filter for pyspark.pandas.read_csv

github-actions[bot] closed pull request #37009: [SPARK-38292][PYTHON]Support na_filter for pyspark.pandas.read_csv URL: https://github.com/apache/spark/pull/37009 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

[GitHub] [spark] github-actions[bot] commented on pull request #37317: [SPARK-39894][SQL] Combine the similar binary comparison in boolean expression.

github-actions[bot] commented on PR #37317: URL: https://github.com/apache/spark/pull/37317#issuecomment-1304677649 We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.

[GitHub] [spark] github-actions[bot] closed pull request #37315: [SPARK-39892][SQL] Use ArrowType.Decimal(precision, scale, bitWidth) instead of ArrowType.Decimal(precision, scale)

github-actions[bot] closed pull request #37315: [SPARK-39892][SQL] Use ArrowType.Decimal(precision, scale, bitWidth) instead of ArrowType.Decimal(precision, scale) URL: https://github.com/apache/spark/pull/37315 -- This is an automated message from the Apache Git Service. To respond to the m

[GitHub] [spark] github-actions[bot] closed pull request #37104: [SPARK-39698][SQL] Use `TakeOrderedAndProject` if maxRows below the `spark.sql.execution.topKSortMaxRowsThreshold`

github-actions[bot] closed pull request #37104: [SPARK-39698][SQL] Use `TakeOrderedAndProject` if maxRows below the `spark.sql.execution.topKSortMaxRowsThreshold` URL: https://github.com/apache/spark/pull/37104 -- This is an automated message from the Apache Git Service. To respond to the me

[GitHub] [spark] github-actions[bot] commented on pull request #37065: [SPARK-38699][SQL] Use error classes in the execution errors of dictionary encoding

github-actions[bot] commented on PR #37065: URL: https://github.com/apache/spark/pull/37065#issuecomment-1304677664 We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.

[GitHub] [spark] github-actions[bot] commented on pull request #37235: [SPARK-39824][PYTHON][PS] Introduce index where and putmask func in pyspark

github-actions[bot] commented on PR #37235: URL: https://github.com/apache/spark/pull/37235#issuecomment-1304677659 We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.

[GitHub] [spark] github-actions[bot] closed pull request #37309: [SPARK-39871][CORE] Jmx http interface supported for SparkHistoryServer

github-actions[bot] closed pull request #37309: [SPARK-39871][CORE] Jmx http interface supported for SparkHistoryServer URL: https://github.com/apache/spark/pull/37309 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the U

[GitHub] [spark] jzhuge commented on a diff in pull request #35636: [SPARK-31357][SQL][WIP] Catalog API for view metadata

jzhuge commented on code in PR #35636: URL: https://github.com/apache/spark/pull/35636#discussion_r1014698488 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/V2ViewDescription.scala: ## @@ -0,0 +1,55 @@ +/* + * Licensed to the Apache Software Foundation (AS

[GitHub] [spark] jzhuge commented on a diff in pull request #35636: [SPARK-31357][SQL][WIP] Catalog API for view metadata

jzhuge commented on code in PR #35636: URL: https://github.com/apache/spark/pull/35636#discussion_r1014695991 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/V2ViewDescription.scala: ## @@ -0,0 +1,55 @@ +/* + * Licensed to the Apache Software Foundation (AS

[GitHub] [spark] jzhuge commented on a diff in pull request #37556: [SPARK-39799][SQL] DataSourceV2: View catalog interface

jzhuge commented on code in PR #37556: URL: https://github.com/apache/spark/pull/37556#discussion_r1014686160 ## sql/catalyst/src/main/java/org/apache/spark/sql/connector/catalog/ViewCatalog.java: ## @@ -0,0 +1,157 @@ +/* + * Licensed to the Apache Software Foundation (ASF) unde

[GitHub] [spark] SandishKumarHN commented on a diff in pull request #38515: [SPARK-41015][SQL][PROTOBUF] UnitTest null check for data generator

SandishKumarHN commented on code in PR #38515: URL: https://github.com/apache/spark/pull/38515#discussion_r1014688948 ## sql/catalyst/src/main/scala/org/apache/spark/sql/errors/QueryCompilationErrors.scala: ## @@ -3330,7 +3330,7 @@ private[sql] object QueryCompilationErrors exte

[GitHub] [spark] SandishKumarHN commented on a diff in pull request #38515: [SPARK-41015][SQL][PROTOBUF] UnitTest null check for data generator

SandishKumarHN commented on code in PR #38515: URL: https://github.com/apache/spark/pull/38515#discussion_r1014688453 ## connector/protobuf/src/test/scala/org/apache/spark/sql/protobuf/ProtobufCatalystDataConversionSuite.scala: ## @@ -123,16 +123,21 @@ class ProtobufCatalystData

[GitHub] [spark] jzhuge commented on pull request #37556: [SPARK-39799][SQL] DataSourceV2: View catalog interface

jzhuge commented on PR #37556: URL: https://github.com/apache/spark/pull/37556#issuecomment-1304612855 Puzzled by this pyspark test failures. Seems unrelated. ``` [info] compiling 25 Scala sources to /__w/spark/spark/connector/docker-integration-tests/target/scala-2.12/test-classes ..

[GitHub] [spark] jzhuge commented on a diff in pull request #37556: [SPARK-39799][SQL] DataSourceV2: View catalog interface

jzhuge commented on code in PR #37556: URL: https://github.com/apache/spark/pull/37556#discussion_r1014686160 ## sql/catalyst/src/main/java/org/apache/spark/sql/connector/catalog/ViewCatalog.java: ## @@ -0,0 +1,157 @@ +/* + * Licensed to the Apache Software Foundation (ASF) unde

[GitHub] [spark] rangadi commented on a diff in pull request #38515: [SPARK-41015][SQL][PROTOBUF] UnitTest null check for data generator

rangadi commented on code in PR #38515: URL: https://github.com/apache/spark/pull/38515#discussion_r1014676031 ## connector/protobuf/src/test/scala/org/apache/spark/sql/protobuf/ProtobufCatalystDataConversionSuite.scala: ## @@ -123,16 +123,21 @@ class ProtobufCatalystDataConvers

[GitHub] [spark] srowen commented on a diff in pull request #38500: [SPARK-41007][SQL] Add missing serializer for java.math.BigInteger

srowen commented on code in PR #38500: URL: https://github.com/apache/spark/pull/38500#discussion_r1014673309 ## sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/JavaTypeInferenceSuite.scala: ## @@ -0,0 +1,41 @@ +/* + * Licensed to the Apache Software Foundation (ASF) u

[GitHub] [spark] MaxGekk opened a new pull request, #38521: [SPARK-41020][SQL] Rename the error class `_LEGACY_ERROR_TEMP_1019` to `STAR_GROUP_BY_POS`

MaxGekk opened a new pull request, #38521: URL: https://github.com/apache/spark/pull/38521 ### What changes were proposed in this pull request? In the PR, I propose to rename the legacy error class `_LEGACY_ERROR_TEMP_1019` to `STAR_GROUP_BY_POS`. ### Why are the changes needed?

[GitHub] [spark] rangadi commented on a diff in pull request #38515: [SPARK-41015][SQL][PROTOBUF] UnitTest null check for data generator

rangadi commented on code in PR #38515: URL: https://github.com/apache/spark/pull/38515#discussion_r1014668826 ## connector/protobuf/src/test/scala/org/apache/spark/sql/protobuf/ProtobufCatalystDataConversionSuite.scala: ## @@ -123,16 +123,21 @@ class ProtobufCatalystDataConvers

[GitHub] [spark] rangadi commented on a diff in pull request #38515: [SPARK-41015][SQL][PROTOBUF] UnitTest null check for data generator

rangadi commented on code in PR #38515: URL: https://github.com/apache/spark/pull/38515#discussion_r1014660407 ## connector/protobuf/src/test/scala/org/apache/spark/sql/protobuf/ProtobufCatalystDataConversionSuite.scala: ## @@ -123,16 +123,21 @@ class ProtobufCatalystDataConvers

[GitHub] [spark] panbingkun opened a new pull request, #38520: [WIP][SQL][TESTS] Check error classes in DataTypeMismatch...

panbingkun opened a new pull request, #38520: URL: https://github.com/apache/spark/pull/38520 ### What changes were proposed in this pull request? ### Why are the changes needed? ### Does this PR introduce _any_ user-facing change? ### How was this patch teste

[GitHub] [spark] dwsmith1983 commented on pull request #38510: [MINOR][DOC] revisions for spark sql performance tuning to improve readability and grammar

dwsmith1983 commented on PR #38510: URL: https://github.com/apache/spark/pull/38510#issuecomment-1304565735 @srowen masters seems to be fine now. I merged mastered into my branch to see if it will fix the tests. -- This is an automated message from the Apache Git Service. To respond to th

[GitHub] [spark] MaxGekk opened a new pull request, #38519: [MINOR][SQL] Remove unused an error class and query error methods

MaxGekk opened a new pull request, #38519: URL: https://github.com/apache/spark/pull/38519 ### What changes were proposed in this pull request? In the PR, I propose to remove the unused error class `CANNOT_INFER_DATE` and some unused methods/parameters from `QueryExecutionErrors` and `Qu

[GitHub] [spark] MaxGekk commented on a diff in pull request #38514: [SPARK-41019][SQL] Provide a query context to `failAnalysis()`

MaxGekk commented on code in PR #38514: URL: https://github.com/apache/spark/pull/38514#discussion_r1014646614 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala: ## @@ -555,14 +554,7 @@ trait CheckAnalysis extends PredicateHelper with Loo

[GitHub] [spark] MaxGekk commented on a diff in pull request #38104: [SPARK-40663][SQL] Migrate execution errors onto error classes: _LEGACY_ERROR_TEMP_2000-2025

MaxGekk commented on code in PR #38104: URL: https://github.com/apache/spark/pull/38104#discussion_r1014645923 ## sql/catalyst/src/main/scala/org/apache/spark/sql/errors/QueryExecutionErrors.scala: ## @@ -334,27 +345,41 @@ private[sql] object QueryExecutionErrors extends QueryE

[GitHub] [spark] MaxGekk commented on a diff in pull request #38123: [SPARK-40663][SQL] Migrate execution errors onto error classes: _LEGACY_ERROR_TEMP_2101-2125

MaxGekk commented on code in PR #38123: URL: https://github.com/apache/spark/pull/38123#discussion_r1014645753 ## sql/catalyst/src/main/scala/org/apache/spark/sql/errors/QueryExecutionErrors.scala: ## @@ -1156,114 +1156,180 @@ private[sql] object QueryExecutionErrors extends Qu

[GitHub] [spark] LuciferYang commented on a diff in pull request #38514: [SPARK-41019][SQL] Provide a query context to `failAnalysis()`

LuciferYang commented on code in PR #38514: URL: https://github.com/apache/spark/pull/38514#discussion_r1014645614 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala: ## @@ -555,14 +554,7 @@ trait CheckAnalysis extends PredicateHelper with

[GitHub] [spark] MaxGekk commented on a diff in pull request #38344: [SPARK-40777][SQL][PROTOBUF] Protobuf import support and move error-classes.

MaxGekk commented on code in PR #38344: URL: https://github.com/apache/spark/pull/38344#discussion_r1014644624 ## sql/catalyst/src/main/scala/org/apache/spark/sql/errors/QueryCompilationErrors.scala: ## @@ -3212,4 +3212,179 @@ private[sql] object QueryCompilationErrors extends

[GitHub] [spark] MaxGekk commented on a diff in pull request #38514: [SPARK-41019][SQL] Provide a query context to `failAnalysis()`

MaxGekk commented on code in PR #38514: URL: https://github.com/apache/spark/pull/38514#discussion_r1014643347 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala: ## @@ -82,24 +82,24 @@ trait CheckAnalysis extends PredicateHelper with Look

[GitHub] [spark] LuciferYang commented on a diff in pull request #38514: [SPARK-41019][SQL] Provide a query context to `failAnalysis()`

LuciferYang commented on code in PR #38514: URL: https://github.com/apache/spark/pull/38514#discussion_r1014641828 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala: ## @@ -82,24 +82,24 @@ trait CheckAnalysis extends PredicateHelper with

[GitHub] [spark] srowen commented on pull request #38510: [MINOR][DOC] revisions for spark sql performance tuning to improve readability and grammar

srowen commented on PR #38510: URL: https://github.com/apache/spark/pull/38510#issuecomment-1304551471 Yeah it's not related, looks like an error with the master branch right now. We can wait a bit for someone to fix that `/home/runner/work/spark/spark/connector/connect/src/test/scala/or

[GitHub] [spark] AmplabJenkins commented on pull request #38515: [SPARK-41015][SQL][PROTOBUF] UnitTest null check for data generator

AmplabJenkins commented on PR #38515: URL: https://github.com/apache/spark/pull/38515#issuecomment-1304540917 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] HyukjinKwon commented on pull request #38516: [SPARK-32380][SQL] Fixing access of HBase table via Hive from Spark

HyukjinKwon commented on PR #38516: URL: https://github.com/apache/spark/pull/38516#issuecomment-1304537957 Also merged to branch-3.3 and branch-3.2. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] HyukjinKwon closed pull request #38516: [SPARK-32380][SQL] Fixing access of HBase table via Hive from Spark

HyukjinKwon closed pull request #38516: [SPARK-32380][SQL] Fixing access of HBase table via Hive from Spark URL: https://github.com/apache/spark/pull/38516 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] HyukjinKwon commented on pull request #38516: [SPARK-32380][SQL] Fixing access of HBase table via Hive from Spark

HyukjinKwon commented on PR #38516: URL: https://github.com/apache/spark/pull/38516#issuecomment-1304537551 Merged to master. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [spark] dwsmith1983 commented on pull request #38510: [MINOR][DOC] revisions for spark sql performance tuning to improve readability and grammar

dwsmith1983 commented on PR #38510: URL: https://github.com/apache/spark/pull/38510#issuecomment-1304519148 Anyone know why it failed this run? None of code has been touched. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [spark] AmplabJenkins commented on pull request #38517: [WIP][SPARK-39591][SS] Async Progress Tracking

AmplabJenkins commented on PR #38517: URL: https://github.com/apache/spark/pull/38517#issuecomment-1304497085 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] AmplabJenkins commented on pull request #38518: [SPARK-33349][K8S] Reset the executor pods watcher when we receive a version changed from k8s

AmplabJenkins commented on PR #38518: URL: https://github.com/apache/spark/pull/38518#issuecomment-1304497079 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] peter-toth commented on a diff in pull request #38513: [SPARK-40903][SQL][FOLLOWUP] Cast canonicalized Add as its original data type if necessary

peter-toth commented on code in PR #38513: URL: https://github.com/apache/spark/pull/38513#discussion_r1014619447 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/arithmetic.scala: ## @@ -481,12 +481,12 @@ case class Add( // TODO: do not reorder cons

[GitHub] [spark] peter-toth commented on a diff in pull request #38513: [SPARK-40903][SQL][FOLLOWUP] Cast canonicalized Add as its original data type if necessary

peter-toth commented on code in PR #38513: URL: https://github.com/apache/spark/pull/38513#discussion_r1014619447 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/arithmetic.scala: ## @@ -481,12 +481,12 @@ case class Add( // TODO: do not reorder cons

[GitHub] [spark] EnricoMi commented on a diff in pull request #38223: [SPARK-40770][PYTHON] Improved error messages for applyInPandas for schema mismatch

EnricoMi commented on code in PR #38223: URL: https://github.com/apache/spark/pull/38223#discussion_r1014612015 ## python/pyspark/worker.py: ## @@ -159,27 +226,13 @@ def wrapped(left_key_series, left_value_series, right_key_series, right_value_se key_series = left_

[GitHub] [spark] zhengruifeng commented on a diff in pull request #38468: [SPARK-41005][CONNECT][PYTHON] Arrow-based collect

zhengruifeng commented on code in PR #38468: URL: https://github.com/apache/spark/pull/38468#discussion_r1014600233 ## connector/connect/src/main/scala/org/apache/spark/sql/connect/service/SparkConnectStreamHandler.scala: ## @@ -117,10 +129,91 @@ class SparkConnectStreamHandler(

[GitHub] [spark] Dam1029 opened a new pull request, #38518: Reset the executor pods watcher when we receive a version changed fro…

Dam1029 opened a new pull request, #38518: URL: https://github.com/apache/spark/pull/38518 ### What changes were proposed in this pull request? This is a straight application of #33349 onto master, reset executor pods watcher when we encounter too old resource version (https://issues.apa

[GitHub] [spark] dwsmith1983 commented on a diff in pull request #38510: [MINOR][DOC] revisions for spark sql performance tuning to improve readability and grammar

dwsmith1983 commented on code in PR #38510: URL: https://github.com/apache/spark/pull/38510#discussion_r1014599711 ## docs/sql-performance-tuning.md: ## @@ -77,8 +77,8 @@ that these options will be deprecated in future release as more optimizations ar spark.sql.files.openC

[GitHub] [spark] MaxGekk commented on a diff in pull request #38422: [SPARK-40948][SQL] Introduce new error class: PATH_NOT_FOUND

MaxGekk commented on code in PR #38422: URL: https://github.com/apache/spark/pull/38422#discussion_r1014596462 ## sql/core/src/test/scala/org/apache/spark/sql/DataFrameSuite.scala: ## @@ -2344,7 +2344,7 @@ class DataFrameSuite extends QueryTest val e = intercept[AnalysisExc

[GitHub] [spark] MaxGekk commented on a diff in pull request #38422: [SPARK-40948][SQL] Introduce new error class: PATH_NOT_FOUND

MaxGekk commented on code in PR #38422: URL: https://github.com/apache/spark/pull/38422#discussion_r1014596349 ## R/pkg/tests/fulltests/test_sparkSQL.R: ## @@ -3990,12 +3990,21 @@ test_that("Call DataFrameWriter.load() API in Java without path and check argume expect_error(r

[GitHub] [spark] MaxGekk commented on pull request #38514: [SPARK-41019][SQL] Provide a query context to `failAnalysis()`

MaxGekk commented on PR #38514: URL: https://github.com/apache/spark/pull/38514#issuecomment-1304428215 @itholic @srielau @LuciferYang @panbingkun @cloud-fan @gengliangwang Could you review this PR, please. -- This is an automated message from the Apache Git Service. To respond to the mes

[GitHub] [spark] MaxGekk commented on a diff in pull request #38490: [SPARK-41009][SQL] Rename the error class `_LEGACY_ERROR_TEMP_1070` to `LOCATION_ALREADY_EXISTS`

MaxGekk commented on code in PR #38490: URL: https://github.com/apache/spark/pull/38490#discussion_r1014594711 ## core/src/main/resources/error/error-classes.json: ## @@ -668,6 +668,24 @@ } } }, + "LOCATION_ALREADY_EXISTS" : { +"message" : [ + "Cannot cr