[GitHub] [spark] mridulm commented on pull request #38091: [SPARK-40096][CORE][TESTS][FOLLOW-UP] Fix flaky test case

mridulm commented on PR #38091: URL: https://github.com/apache/spark/pull/38091#issuecomment-131768 Let us see if the recent fix addresses the issue - else we can take that route @LuciferYang -- This is an automated message from the Apache Git Service. To respond to the message, plea

[GitHub] [spark] felipepessoto commented on pull request #37616: [SPARK-40178][PYTHON][SQL] Fix partitioning hint parameters in PySpark

felipepessoto commented on PR #37616: URL: https://github.com/apache/spark/pull/37616#issuecomment-1312224727 For Scala is expected to need to call `.expr`, or we need to fix it as well? -- This is an automated message from the Apache Git Service. To respond to the message, please log on t

[GitHub] [spark] grundprinzip commented on a diff in pull request #38631: [SPARK-40809] [CONNECT] [FOLLOW] Support `alias()` in Python client

grundprinzip commented on code in PR #38631: URL: https://github.com/apache/spark/pull/38631#discussion_r1020586736 ## python/pyspark/sql/connect/column.py: ## @@ -82,6 +82,74 @@ def to_plan(self, session: "RemoteSparkSession") -> "proto.Expression": def __str__(self) -> s

[GitHub] [spark] AmplabJenkins commented on pull request #38615: [SPARK-41109][SQL] Rename the error class _LEGACY_ERROR_TEMP_1216 to INVALID_LIKE_PATTERN

AmplabJenkins commented on PR #38615: URL: https://github.com/apache/spark/pull/38615#issuecomment-1312240034 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] AmplabJenkins commented on pull request #38611: [SPARK-41107][PYTHON][INFRA][TEST] Install memory-profiler in the CI

AmplabJenkins commented on PR #38611: URL: https://github.com/apache/spark/pull/38611#issuecomment-1312240061 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] xinrong-meng commented on a diff in pull request #38611: [SPARK-41107][PYTHON][INFRA][TEST] Install memory-profiler in the CI

xinrong-meng commented on code in PR #38611: URL: https://github.com/apache/spark/pull/38611#discussion_r1020600278 ## dev/infra/Dockerfile: ## @@ -32,7 +32,7 @@ RUN $APT_INSTALL software-properties-common git libxml2-dev pkg-config curl wget RUN update-alternatives --set java

[GitHub] [spark] amaliujia opened a new pull request, #38632: [SPARK-41116][CONNECT] Input relation can be optional for Project in Connect proto

amaliujia opened a new pull request, #38632: URL: https://github.com/apache/spark/pull/38632 ### What changes were proposed in this pull request? I was writing test cases to test expressions and realized that we can allow `Project` without input plan. For example, `SELECT 1` i

[GitHub] [spark] amaliujia commented on pull request #38632: [SPARK-41116][CONNECT] Input relation can be optional for Project in Connect proto

amaliujia commented on PR #38632: URL: https://github.com/apache/spark/pull/38632#issuecomment-1312268172 R: @cloud-fan -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] github-actions[bot] commented on pull request #37346: [SPARK-37210][CORE][SQL] Allow forced use of staging directory

github-actions[bot] commented on PR #37346: URL: https://github.com/apache/spark/pull/37346#issuecomment-1312282414 We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.

[GitHub] [spark] github-actions[bot] closed pull request #37365: [SPARK-39938][PYTHON][PS] Accept all inputs of prefix/suffix which implement __str__ in add_predix/add_suffix

github-actions[bot] closed pull request #37365: [SPARK-39938][PYTHON][PS] Accept all inputs of prefix/suffix which implement __str__ in add_predix/add_suffix URL: https://github.com/apache/spark/pull/37365 -- This is an automated message from the Apache Git Service. To respond to the message

[GitHub] [spark] github-actions[bot] closed pull request #37355: [SPARK-39930][SQL] Introduce Cache Hints

github-actions[bot] closed pull request #37355: [SPARK-39930][SQL] Introduce Cache Hints URL: https://github.com/apache/spark/pull/37355 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] viirya commented on a diff in pull request #38626: [SPARK-38959][SQL][FOLLOWUP] Do not optimize subqueries twice

viirya commented on code in PR #38626: URL: https://github.com/apache/spark/pull/38626#discussion_r1020639233 ## sql/core/src/main/scala/org/apache/spark/sql/execution/SparkOptimizer.scala: ## @@ -51,8 +51,10 @@ class SparkOptimizer( Batch("Optimize Metadata Only Query", On

[GitHub] [spark] viirya commented on a diff in pull request #38626: [SPARK-38959][SQL][FOLLOWUP] Do not optimize subqueries twice

viirya commented on code in PR #38626: URL: https://github.com/apache/spark/pull/38626#discussion_r1020639496 ## sql/core/src/main/scala/org/apache/spark/sql/execution/SparkOptimizer.scala: ## @@ -51,8 +51,10 @@ class SparkOptimizer( Batch("Optimize Metadata Only Query", On

[GitHub] [spark] viirya commented on a diff in pull request #38626: [SPARK-38959][SQL][FOLLOWUP] Do not optimize subqueries twice

viirya commented on code in PR #38626: URL: https://github.com/apache/spark/pull/38626#discussion_r1020639613 ## sql/core/src/main/scala/org/apache/spark/sql/execution/SparkOptimizer.scala: ## @@ -51,8 +51,10 @@ class SparkOptimizer( Batch("Optimize Metadata Only Query", On

[GitHub] [spark] WeichenXu123 commented on a diff in pull request #37734: [SPARK-40264][ML] add batch_infer_udf function to pyspark.ml.functions

WeichenXu123 commented on code in PR #37734: URL: https://github.com/apache/spark/pull/37734#discussion_r1020654939 ## python/pyspark/ml/functions.py: ## @@ -106,6 +138,602 @@ def array_to_vector(col: Column) -> Column: return Column(sc._jvm.org.apache.spark.ml.functions.a

[GitHub] [spark] Dam1029 commented on pull request #38518: [SPARK-33349][K8S] Reset the executor pods watcher when we receive a version changed from k8s

Dam1029 commented on PR #38518: URL: https://github.com/apache/spark/pull/38518#issuecomment-1312350979 @dongjoon-hyun @Ngone51 Could you help take a look? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [spark] AmplabJenkins commented on pull request #38607: [SPARK-40938][CONNECT][PYTHON][FOLLOW-UP] Fix SubqueryAlias without the child plan when constructing Connect proto in the Python client

AmplabJenkins commented on PR #38607: URL: https://github.com/apache/spark/pull/38607#issuecomment-1312357530 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] AmplabJenkins commented on pull request #38603: [SPARK-41101][PYTHON][PROTOBUF] Message classname support for PYSPARK-PROTOBUF

AmplabJenkins commented on PR #38603: URL: https://github.com/apache/spark/pull/38603#issuecomment-1312357548 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] AmplabJenkins commented on pull request #38601: [WIP][SPARK-41100][INFRA] Upgrade Ubuntu to latest

AmplabJenkins commented on PR #38601: URL: https://github.com/apache/spark/pull/38601#issuecomment-1312357558 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] MaxGekk commented on a diff in pull request #38615: [SPARK-41109][SQL] Rename the error class _LEGACY_ERROR_TEMP_1216 to INVALID_LIKE_PATTERN

MaxGekk commented on code in PR #38615: URL: https://github.com/apache/spark/pull/38615#discussion_r1020704358 ## core/src/main/resources/error/error-classes.json: ## @@ -630,6 +630,11 @@ "Input schema can only contain STRING as a key type for a MAP." ] }, + "I

<    1   2