[jira] [Resolved] (SPARK-52181) Increase variant size limit to 128MiB

2025-05-16 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-52181. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 50927 [https://gith

[jira] [Updated] (SPARK-52165) [SDP] Setup build scaffolding for the pipelines project

2025-05-16 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-52165: --- Affects Version/s: 4.1.0 (was: 4.0.0) > [SDP] Setup build scaffol

[jira] [Assigned] (SPARK-52165) [SDP] Setup build scaffolding for the pipelines project

2025-05-16 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang reassigned SPARK-52165: -- Assignee: Jon Mio > [SDP] Setup build scaffolding for the pipelines project > ---

[jira] [Resolved] (SPARK-52165) [SDP] Setup build scaffolding for the pipelines project

2025-05-16 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-52165. Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 50905 [https:

[jira] [Assigned] (SPARK-52202) Reduce PMD exclusion rules

2025-05-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-52202: - Assignee: Dongjoon Hyun > Reduce PMD exclusion rules > -- > >

[jira] [Updated] (SPARK-52202) Reduce PMD exclusion rules

2025-05-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-52202: --- Labels: pull-request-available (was: ) > Reduce PMD exclusion rules > -

[jira] [Created] (SPARK-52202) Reduce PMD exclusion rules

2025-05-16 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-52202: - Summary: Reduce PMD exclusion rules Key: SPARK-52202 URL: https://issues.apache.org/jira/browse/SPARK-52202 Project: Spark Issue Type: Sub-task C

[jira] [Resolved] (SPARK-52201) Revisit `PMD` annotations

2025-05-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-52201. --- Fix Version/s: kubernetes-operator-0.2.0 Resolution: Fixed This is resolved via http

[jira] [Resolved] (SPARK-52200) Make `StatusRecorder.objectMapper` static

2025-05-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-52200. --- Fix Version/s: kubernetes-operator-0.2.0 Resolution: Fixed Issue resolved by pull req

[jira] [Assigned] (SPARK-52200) Make `StatusRecorder.objectMapper` static

2025-05-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-52200: - Assignee: Dongjoon Hyun > Make `StatusRecorder.objectMapper` static > -

[jira] [Assigned] (SPARK-52201) Revisit `PMD` annotations

2025-05-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-52201: - Assignee: Dongjoon Hyun > Revisit `PMD` annotations > - > >

[jira] [Updated] (SPARK-52201) Revisit `PMD` annotations

2025-05-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-52201: --- Labels: pull-request-available (was: ) > Revisit `PMD` annotations > --

[jira] [Created] (SPARK-52201) Revisit `PMD` annotations

2025-05-16 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-52201: - Summary: Revisit `PMD` annotations Key: SPARK-52201 URL: https://issues.apache.org/jira/browse/SPARK-52201 Project: Spark Issue Type: Sub-task Co

[jira] [Resolved] (SPARK-52199) Remove `Spark 3.4.3` bindings in `spark-versions` E2E test

2025-05-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-52199. --- Fix Version/s: kubernetes-operator-0.2.0 Resolution: Fixed Issue resolved by pull req

[jira] [Assigned] (SPARK-52199) Remove `Spark 3.4.3` bindings in `spark-versions` E2E test

2025-05-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-52199: - Assignee: Dongjoon Hyun > Remove `Spark 3.4.3` bindings in `spark-versions` E2E test >

[jira] [Created] (SPARK-52200) Make `StatusRecorder.objectMapper` static

2025-05-16 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-52200: - Summary: Make `StatusRecorder.objectMapper` static Key: SPARK-52200 URL: https://issues.apache.org/jira/browse/SPARK-52200 Project: Spark Issue Type: Sub-t

[jira] [Updated] (SPARK-52200) Make `StatusRecorder.objectMapper` static

2025-05-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-52200: --- Labels: pull-request-available (was: ) > Make `StatusRecorder.objectMapper` static > --

[jira] [Resolved] (SPARK-52188) State store source can't read from RocksDB

2025-05-16 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-52188. -- Fix Version/s: 4.0.0 4.1.0 Assignee: Eric Marnadi Resolution

[jira] [Updated] (SPARK-52199) Remove `Spark 3.4.3` bindings in `spark-versions` E2E test

2025-05-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-52199: --- Labels: pull-request-available (was: ) > Remove `Spark 3.4.3` bindings in `spark-versions`

[jira] [Created] (SPARK-52199) Remove `Spark 3.4.3` bindings in `spark-versions` E2E test

2025-05-16 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-52199: - Summary: Remove `Spark 3.4.3` bindings in `spark-versions` E2E test Key: SPARK-52199 URL: https://issues.apache.org/jira/browse/SPARK-52199 Project: Spark

[jira] [Resolved] (SPARK-52197) Upgrade `gRPC Swift` to 2.2.1

2025-05-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-52197. --- Fix Version/s: connect-swift-0.2.0 Resolution: Fixed Issue resolved by pull request 1

[jira] [Created] (SPARK-52198) Fix keySchema in KeyWithIndexToValueStore for join state format v3 with column families

2025-05-16 Thread Livia Zhu (Jira)
Livia Zhu created SPARK-52198: - Summary: Fix keySchema in KeyWithIndexToValueStore for join state format v3 with column families Key: SPARK-52198 URL: https://issues.apache.org/jira/browse/SPARK-52198 Pro

[jira] [Updated] (SPARK-52197) Upgrade `gRPC Swift` to 2.2.1

2025-05-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-52197: --- Labels: pull-request-available (was: ) > Upgrade `gRPC Swift` to 2.2.1 > --

[jira] [Created] (SPARK-52197) Upgrade `gRPC Swift` to 2.2.1

2025-05-16 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-52197: - Summary: Upgrade `gRPC Swift` to 2.2.1 Key: SPARK-52197 URL: https://issues.apache.org/jira/browse/SPARK-52197 Project: Spark Issue Type: Sub-task

[jira] [Resolved] (SPARK-52196) Promote `SparkSQLRepl` code to `Spark SQL REPL` example

2025-05-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-52196. --- Fix Version/s: connect-swift-0.2.0 Resolution: Fixed Issue resolved by pull request 1

[jira] [Updated] (SPARK-52196) Promote `SparkSQLRepl` code to `Spark SQL REPL` example

2025-05-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-52196: --- Labels: pull-request-available (was: ) > Promote `SparkSQLRepl` code to `Spark SQL REPL` ex

[jira] [Updated] (SPARK-52195) Fix initial state column dropping issue for Python TWS

2025-05-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-52195: --- Labels: pull-request-available (was: ) > Fix initial state column dropping issue for Python

[jira] [Created] (SPARK-52195) Fix initial state column dropping issue for Python TWS

2025-05-16 Thread Bo Gao (Jira)
Bo Gao created SPARK-52195: -- Summary: Fix initial state column dropping issue for Python TWS Key: SPARK-52195 URL: https://issues.apache.org/jira/browse/SPARK-52195 Project: Spark Issue Type: Task

[jira] [Resolved] (SPARK-52193) Add `Spark Connect` port to Spark Driver pod and service

2025-05-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-52193. --- Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 50925 [https://

[jira] [Resolved] (SPARK-52194) Add `spark-connect-server.yaml` example

2025-05-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-52194. --- Fix Version/s: kubernetes-operator-0.2.0 Resolution: Fixed Issue resolved by pull req

[jira] [Updated] (SPARK-52156) Put legacy CREATE TEMPORARY TABLE ... USING provider under the flag

2025-05-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-52156: -- Parent: SPARK-51166 Issue Type: Sub-task (was: Improvement) > Put legacy CREATE TEMPO

[jira] [Assigned] (SPARK-52194) Add `spark-connect-server.yaml` example

2025-05-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-52194: - Assignee: Dongjoon Hyun > Add `spark-connect-server.yaml` example > ---

[jira] [Updated] (SPARK-52194) Add `spark-connect-server.yaml` example

2025-05-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-52194: --- Labels: pull-request-available (was: ) > Add `spark-connect-server.yaml` example >

[jira] [Created] (SPARK-52194) Add `spark-connect-server.yaml` example

2025-05-16 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-52194: - Summary: Add `spark-connect-server.yaml` example Key: SPARK-52194 URL: https://issues.apache.org/jira/browse/SPARK-52194 Project: Spark Issue Type: Sub-tas

[jira] [Updated] (SPARK-52101) Disable Inline Forcing for all rCTEs

2025-05-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-52101: --- Labels: pull-request-available (was: ) > Disable Inline Forcing for all rCTEs > ---

[jira] [Assigned] (SPARK-52156) Put legacy CREATE TEMPORARY TABLE ... USING provider under the flag

2025-05-16 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-52156: --- Assignee: Mihailo Milosevic > Put legacy CREATE TEMPORARY TABLE ... USING provider under th

[jira] [Updated] (SPARK-52156) Put legacy CREATE TEMPORARY TABLE ... USING provider under the flag

2025-05-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-52156: --- Labels: pull-request-available (was: ) > Put legacy CREATE TEMPORARY TABLE ... USING provid

[jira] [Resolved] (SPARK-52156) Put legacy CREATE TEMPORARY TABLE ... USING provider under the flag

2025-05-16 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-52156. - Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 50902 [https://gith

[jira] [Resolved] (SPARK-52174) Enable `spark.checkpoint.compress` by default

2025-05-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-52174. --- Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 50908 [https://

[jira] [Updated] (SPARK-52174) Enable `spark.checkpoint.compress` by default

2025-05-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-52174: --- Labels: pull-request-available (was: ) > Enable `spark.checkpoint.compress` by default > --

[jira] [Assigned] (SPARK-52174) Enable `spark.checkpoint.compress` by default

2025-05-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-52174: - Assignee: Dongjoon Hyun > Enable `spark.checkpoint.compress` by default > -

[jira] [Updated] (SPARK-52184) Wrap external engine syntax errors in a unified spark error

2025-05-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-52184: --- Labels: pull-request-available (was: ) > Wrap external engine syntax errors in a unified sp

[jira] [Updated] (SPARK-52185) Automate the thread dump collection for Spark applications

2025-05-16 Thread Gabor Roczei (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Roczei updated SPARK-52185: - Description: When a Java program runs for a long time without giving you any feedback/output, h

[jira] [Updated] (SPARK-52188) State store source can't read from RocksDB

2025-05-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-52188: --- Labels: pull-request-available (was: ) > State store source can't read from RocksDB > -

[jira] [Commented] (SPARK-52188) State store source can't read from RocksDB

2025-05-16 Thread Eric Marnadi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17952255#comment-17952255 ] Eric Marnadi commented on SPARK-52188: -- I have the PR here: https://github.com/apac

[jira] [Commented] (SPARK-52188) State store source can't read from RocksDB

2025-05-16 Thread Adam Binford (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17952246#comment-17952246 ] Adam Binford commented on SPARK-52188: -- Yeah if you just change that Utils.isTestin

[jira] [Commented] (SPARK-52188) State store source can't read from RocksDB

2025-05-16 Thread Eric Marnadi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17952241#comment-17952241 ] Eric Marnadi commented on SPARK-52188: -- Looks like this is a real bug from running

[jira] [Commented] (SPARK-52188) State store source can't read from RocksDB

2025-05-16 Thread Eric Marnadi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17952239#comment-17952239 ] Eric Marnadi commented on SPARK-52188: -- I remember adding this code because of the

[jira] [Commented] (SPARK-52188) State store source can't read from RocksDB

2025-05-16 Thread Eric Marnadi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17952235#comment-17952235 ] Eric Marnadi commented on SPARK-52188: -- [~kimahriman] Can you provide a repro?  >

[jira] [Updated] (SPARK-52192) MLCache loading path check

2025-05-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-52192: --- Labels: pull-request-available (was: ) > MLCache loading path check > -

[jira] [Updated] (SPARK-52185) Automate the thread dump collection for Spark applications

2025-05-16 Thread Gabor Roczei (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Roczei updated SPARK-52185: - Description: When a Java program runs for a long time without giving you any feedback/output, h

[jira] [Updated] (SPARK-52165) [SDP] Setup build scaffolding for the pipelines project

2025-05-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-52165: --- Labels: pull-request-available (was: ) > [SDP] Setup build scaffolding for the pipelines pr

[jira] [Created] (SPARK-52192) MLCache loading path check

2025-05-16 Thread Weichen Xu (Jira)
Weichen Xu created SPARK-52192: -- Summary: MLCache loading path check Key: SPARK-52192 URL: https://issues.apache.org/jira/browse/SPARK-52192 Project: Spark Issue Type: Sub-task Compone

[jira] [Updated] (SPARK-52191) Remove Java deserializer in model local path loader

2025-05-16 Thread Weichen Xu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weichen Xu updated SPARK-52191: --- Summary: Remove Java deserializer in model local path loader (was: Remove Java deserializer in mode

[jira] [Updated] (SPARK-52191) Remove Java deserializer in model local path loader

2025-05-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-52191: --- Labels: pull-request-available (was: ) > Remove Java deserializer in model local path loade

[jira] [Updated] (SPARK-52191) Remove Java deserializer in model local path loader

2025-05-16 Thread Weichen Xu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weichen Xu updated SPARK-52191: --- Description: Remove Java deserializer in model local path loader. Java deserializer is unsafe, remo

[jira] [Created] (SPARK-52191) Remove Java deserializer in model reader from local path

2025-05-16 Thread Weichen Xu (Jira)
Weichen Xu created SPARK-52191: -- Summary: Remove Java deserializer in model reader from local path Key: SPARK-52191 URL: https://issues.apache.org/jira/browse/SPARK-52191 Project: Spark Issue Ty

[jira] [Resolved] (SPARK-52190) Revise `Helm` chart README

2025-05-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-52190. --- Fix Version/s: kubernetes-operator-0.2.0 Resolution: Fixed Issue resolved by pull req

[jira] [Assigned] (SPARK-52190) Revise `Helm` chart README

2025-05-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-52190: - Assignee: Dongjoon Hyun > Revise `Helm` chart README > -- > >

[jira] [Updated] (SPARK-52190) Revise `Helm` chart README

2025-05-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-52190: --- Labels: pull-request-available (was: ) > Revise `Helm` chart README > -

[jira] [Assigned] (SPARK-52173) Fix OwnerReference Patching for Spark Application Driver Resources

2025-05-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-52173: - Assignee: Zhou JIANG > Fix OwnerReference Patching for Spark Application Driver Resourc

[jira] [Updated] (SPARK-52173) Fix OwnerReference Patching for Spark Application Driver Resources

2025-05-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-52173: --- Labels: pull-request-available (was: ) > Fix OwnerReference Patching for Spark Application

[jira] [Resolved] (SPARK-52173) Fix OwnerReference Patching for Spark Application Driver Resources

2025-05-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-52173. --- Fix Version/s: kubernetes-operator-0.2.0 Resolution: Fixed Issue resolved by pull req

[jira] [Created] (SPARK-52190) Revise `Helm` chart README

2025-05-16 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-52190: - Summary: Revise `Helm` chart README Key: SPARK-52190 URL: https://issues.apache.org/jira/browse/SPARK-52190 Project: Spark Issue Type: Sub-task C

[jira] [Updated] (SPARK-52181) Increase variant size limit to 128MiB

2025-05-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-52181: --- Labels: pull-request-available (was: ) > Increase variant size limit to 128MiB > --

[jira] [Commented] (SPARK-52185) Automate the thread dump collection for Spark applications

2025-05-16 Thread Gabor Roczei (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17952164#comment-17952164 ] Gabor Roczei commented on SPARK-52185: -- Created a PR for this: https://github.com/a

[jira] [Updated] (SPARK-52134) Spark Connect execution

2025-05-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-52134: --- Labels: pull-request-available (was: ) > Spark Connect execution > ---

[jira] [Updated] (SPARK-52157) Use earlier computed name for OuterReference in single-pass resolver

2025-05-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-52157: --- Labels: pull-request-available (was: ) > Use earlier computed name for OuterReference in si

[jira] [Updated] (SPARK-52158) Add InvalidInputErrors object to centralize errors in SparkConnectPlanner

2025-05-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-52158: --- Labels: pull-request-available (was: ) > Add InvalidInputErrors object to centralize errors

[jira] [Updated] (SPARK-52122) Fix DefaultParamsReader RCE vulnerability

2025-05-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-52122: --- Labels: pull-request-available (was: ) > Fix DefaultParamsReader RCE vulnerability > --

[jira] [Created] (SPARK-52189) Add new golden file tests for NATURAL JOIN with GROUP BY, HAVING, ORDER BY and LCAs

2025-05-16 Thread Vladimir Golubev (Jira)
Vladimir Golubev created SPARK-52189: Summary: Add new golden file tests for NATURAL JOIN with GROUP BY, HAVING, ORDER BY and LCAs Key: SPARK-52189 URL: https://issues.apache.org/jira/browse/SPARK-52189

[jira] [Updated] (SPARK-52126) Revert rename for TWS utility classes for forward compatibility in Spark Connect

2025-05-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-52126: --- Labels: pull-request-available (was: ) > Revert rename for TWS utility classes for forward

[jira] [Created] (SPARK-52188) State store source can't read from RocksDB

2025-05-16 Thread Adam Binford (Jira)
Adam Binford created SPARK-52188: Summary: State store source can't read from RocksDB Key: SPARK-52188 URL: https://issues.apache.org/jira/browse/SPARK-52188 Project: Spark Issue Type: Improv

[jira] [Updated] (SPARK-52108) Enable `GitHub Pages`

2025-05-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-52108: --- Labels: pull-request-available (was: ) > Enable `GitHub Pages` > - > >

[jira] [Updated] (SPARK-52107) Add `workflow_dispatch` to `pages.yml` to support manual trigger

2025-05-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-52107: --- Labels: pull-request-available (was: ) > Add `workflow_dispatch` to `pages.yml` to support

[jira] [Created] (SPARK-52187) Introduce Join Pushdown to Datasource V2

2025-05-16 Thread Petar Vasiljevic (Jira)
Petar Vasiljevic created SPARK-52187: Summary: Introduce Join Pushdown to Datasource V2 Key: SPARK-52187 URL: https://issues.apache.org/jira/browse/SPARK-52187 Project: Spark Issue Type:

[jira] [Created] (SPARK-52186) An Optimization for EXISTS Subquery

2025-05-16 Thread KeKe Zhu (Jira)
KeKe Zhu created SPARK-52186: Summary: An Optimization for EXISTS Subquery Key: SPARK-52186 URL: https://issues.apache.org/jira/browse/SPARK-52186 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-52186) An Optimization for EXISTS Subquery

2025-05-16 Thread KeKe Zhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] KeKe Zhu updated SPARK-52186: - Affects Version/s: 3.5.0 3.4.0 (was: 3.5.4)

[jira] [Created] (SPARK-52185) Automate the thread dump collection for Spark applications

2025-05-16 Thread Gabor Roczei (Jira)
Gabor Roczei created SPARK-52185: Summary: Automate the thread dump collection for Spark applications Key: SPARK-52185 URL: https://issues.apache.org/jira/browse/SPARK-52185 Project: Spark Is

[jira] [Assigned] (SPARK-52183) Update `SparkSQLRepl` example to show up to 10k rows

2025-05-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-52183: - Assignee: Dongjoon Hyun > Update `SparkSQLRepl` example to show up to 10k rows > --

[jira] [Resolved] (SPARK-52183) Update `SparkSQLRepl` example to show up to 10k rows

2025-05-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-52183. --- Fix Version/s: connect-swift-0.2.0 Resolution: Fixed Issue resolved by pull request 1

[jira] [Resolved] (SPARK-52182) Support `withWatermark` for `DataFrame`

2025-05-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-52182. --- Fix Version/s: connect-swift-0.2.0 Resolution: Fixed Issue resolved by pull request 1

[jira] [Updated] (SPARK-52158) Add InvalidInputErrors object to centralize errors in SparkConnectPlanner

2025-05-16 Thread Yihong He (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yihong He updated SPARK-52158: -- Description: (was: The errors thrown in the SparkConnectPlanner currently do not really use the Ne

[jira] [Resolved] (SPARK-52180) Create artifacts for output and log files in dry-run build

2025-05-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-52180. -- Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 50912 [https://gi

[jira] [Assigned] (SPARK-52180) Create artifacts for output and log files in dry-run build

2025-05-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-52180: Assignee: Hyukjin Kwon > Create artifacts for output and log files in dry-run build > ---

[jira] [Assigned] (SPARK-52122) Fix DefaultParamsReader RCE vulnerability

2025-05-16 Thread Weichen Xu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weichen Xu reassigned SPARK-52122: -- Assignee: Weichen Xu > Fix DefaultParamsReader RCE vulnerability > --

[jira] [Resolved] (SPARK-52122) Fix DefaultParamsReader RCE vulnerability

2025-05-16 Thread Weichen Xu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weichen Xu resolved SPARK-52122. Fix Version/s: 4.1.0 4.0.0 Resolution: Fixed Issue resolved by pull requ

[jira] [Updated] (SPARK-52158) Add InvalidInputErrors object to centralize errors in SparkConnectPlanner

2025-05-16 Thread Yihong He (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yihong He updated SPARK-52158: -- Summary: Add InvalidInputErrors object to centralize errors in SparkConnectPlanner (was: NERF errors

[jira] [Created] (SPARK-52184) Wrap external engine syntax errors in a unified spark error

2025-05-16 Thread Alek Jarmov (Jira)
Alek Jarmov created SPARK-52184: --- Summary: Wrap external engine syntax errors in a unified spark error Key: SPARK-52184 URL: https://issues.apache.org/jira/browse/SPARK-52184 Project: Spark Is

[jira] [Resolved] (SPARK-52157) Use earlier computed name for OuterReference in single-pass resolver

2025-05-16 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-52157. - Fix Version/s: 4.1.0 Resolution: Fixed Issue resolved by pull request 50903 [https://gith

[jira] [Assigned] (SPARK-52157) Use earlier computed name for OuterReference in single-pass resolver

2025-05-16 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-52157: --- Assignee: Mihailo Aleksic > Use earlier computed name for OuterReference in single-pass res

[jira] [Assigned] (SPARK-52134) Spark Connect execution

2025-05-16 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-52134: --- Assignee: David Milicevic > Spark Connect execution > --- > >

[jira] [Resolved] (SPARK-52134) Spark Connect execution

2025-05-16 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-52134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-52134. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 50895 [https://gith

[jira] [Resolved] (SPARK-51301) Bump zstd-jni 1.5.7-1

2025-05-16 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie resolved SPARK-51301. -- Resolution: Fixed Issue resolved by pull request 50057 https://github.com/apache/spark/pull/50057 > B

[jira] [Commented] (SPARK-51035) Upgrade dependencies exposed to critical and high CVEs

2025-05-16 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17952027#comment-17952027 ] Yang Jie commented on SPARK-51035: -- Upgrading Derby requires waiting for the Derby comm

[jira] [Assigned] (SPARK-51301) Bump zstd-jni 1.5.7-1

2025-05-16 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-51301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie reassigned SPARK-51301: Assignee: Cheng Pan > Bump zstd-jni 1.5.7-1 > - > > Key: SPAR