[jira] [Created] (SPARK-50360) StateStoreChangelogReader should read state store version from the changelog

2024-11-19 Thread Wei Liu (Jira)
Wei Liu created SPARK-50360: --- Summary: StateStoreChangelogReader should read state store version from the changelog Key: SPARK-50360 URL: https://issues.apache.org/jira/browse/SPARK-50360 Project: Spark

[jira] [Updated] (SPARK-50358) Update postgres docker image to 17.1

2024-11-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-50358: --- Labels: pull-request-available (was: ) > Update postgres docker image to 17.1 > ---

[jira] [Resolved] (SPARK-50352) Refine docstrings for window/aggregation functions - part 3

2024-11-19 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-50352. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 48891 [https://gi

[jira] [Resolved] (SPARK-50358) Update postgres docker image to 17.1

2024-11-19 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-50358. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 48897 [https://gi

[jira] [Assigned] (SPARK-48344) [M1] SQL Script execution

2024-11-19 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-48344: --- Assignee: Milan Dankovic > [M1] SQL Script execution > - > >

[jira] [Resolved] (SPARK-48344) [M1] SQL Script execution

2024-11-19 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-48344. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 48879 [https://gith

[jira] [Updated] (SPARK-50359) Upgrade PyArrow to 18.0

2024-11-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-50359: --- Labels: pull-request-available (was: ) > Upgrade PyArrow to 18.0 > ---

[jira] [Created] (SPARK-50356) Skipped stages are inconsitent on UI

2024-11-19 Thread Peter Toth (Jira)
Peter Toth created SPARK-50356: -- Summary: Skipped stages are inconsitent on UI Key: SPARK-50356 URL: https://issues.apache.org/jira/browse/SPARK-50356 Project: Spark Issue Type: Bug Co

[jira] [Updated] (SPARK-7008) An implementation of Factorization Machine (LibFM)

2024-11-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-7008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-7008: -- Labels: features pull-request-available (was: features) > An implementation of Factorization M

[jira] [Assigned] (SPARK-50352) Refine docstrings for window/aggregation functions - part 3

2024-11-19 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-50352: Assignee: Ruifeng Zheng > Refine docstrings for window/aggregation functions - part 3 > -

[jira] [Created] (SPARK-50358) Update postgres docker image to 17.1

2024-11-19 Thread Pan Bingkun (Jira)
Pan Bingkun created SPARK-50358: --- Summary: Update postgres docker image to 17.1 Key: SPARK-50358 URL: https://issues.apache.org/jira/browse/SPARK-50358 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-50357) Support `interrupt(Tag|All|Operation)` APIs for PySpark

2024-11-19 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-50357: --- Summary: Support `interrupt(Tag|All|Operation)` APIs for PySpark Key: SPARK-50357 URL: https://issues.apache.org/jira/browse/SPARK-50357 Project: Spark Issue T

[jira] [Updated] (SPARK-50341) PySpark - Use UDS for communication between JVM and Streaming Python worker

2024-11-19 Thread Bo Gao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bo Gao updated SPARK-50341: --- Summary: PySpark - Use UDS for communication between JVM and Streaming Python worker (was: PySpark - Use UD

[jira] [Updated] (SPARK-50356) Skipped stages are inconsitent on UI

2024-11-19 Thread Peter Toth (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Toth updated SPARK-50356: --- Priority: Minor (was: Major) > Skipped stages are inconsitent on UI > -

[jira] [Updated] (SPARK-50356) Skipped stages are inconsitent on UI

2024-11-19 Thread Peter Toth (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Toth updated SPARK-50356: --- Description: The following snipett produces inconsitent UI state: {noformat} import scala.collection

[jira] [Updated] (SPARK-50356) Skipped stages are inconsitent on UI

2024-11-19 Thread Peter Toth (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Toth updated SPARK-50356: --- Description: The following snipett produces inconsitent UI state: {noformat} import scala.collection

[jira] [Updated] (SPARK-50356) Skipped stages are inconsitent on UI

2024-11-19 Thread Peter Toth (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Toth updated SPARK-50356: --- Attachment: (was: Screenshot 2024-11-19 at 16.07.00.png) > Skipped stages are inconsitent on UI

[jira] [Updated] (SPARK-50356) Skipped stages are inconsitent on UI

2024-11-19 Thread Peter Toth (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Toth updated SPARK-50356: --- Attachment: Screenshot 2024-11-19 at 16.07.20.png Screenshot 2024-11-19 at 16.07.06.p

[jira] [Updated] (SPARK-50356) Skipped stages are inconsitent on UI

2024-11-19 Thread Peter Toth (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Toth updated SPARK-50356: --- Description: The following snipett produces inconsitent UI state: {noformat} import scala.collection

[jira] [Updated] (SPARK-50354) No need to set initialInputBufferOffset when Aggregate mode is Complete

2024-11-19 Thread zhixingheyi-tian (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhixingheyi-tian updated SPARK-50354: - Description: Here: [https://github.com/apache/spark/blob/87a5b37ec3c4b383a5938144612c071

[jira] [Updated] (SPARK-50354) No need to set initialInputBufferOffset when Aggregate mode is Complete

2024-11-19 Thread zhixingheyi-tian (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhixingheyi-tian updated SPARK-50354: - Description: Here: [https://github.com/apache/spark/blob/87a5b37ec3c4b383a5938144612c071

[jira] [Assigned] (SPARK-50315) Custom metrics for V1FallbackWriters

2024-11-19 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-50315: --- Assignee: Jan-Ole Sasse > Custom metrics for V1FallbackWriters > --

[jira] [Resolved] (SPARK-50315) Custom metrics for V1FallbackWriters

2024-11-19 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-50315. - Resolution: Fixed Issue resolved by pull request 48867 [https://github.com/apache/spark/pull/488

[jira] [Updated] (SPARK-50354) No need to set initialInputBufferOffset when Aggregate mode is Complete

2024-11-19 Thread zhixingheyi-tian (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhixingheyi-tian updated SPARK-50354: - Description: Here: [https://github.com/apache/spark/blob/87a5b37ec3c4b383a5938144612c071

[jira] [Updated] (SPARK-50354) No need to set initialInputBufferOffset when Aggregate mode is Complete

2024-11-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-50354: --- Labels: pull-request-available (was: ) > No need to set initialInputBufferOffset when Aggre

[jira] [Updated] (SPARK-37455) Replace hash with sort aggregate if child is already sorted

2024-11-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-37455: --- Labels: pull-request-available (was: ) > Replace hash with sort aggregate if child is alrea

[jira] [Updated] (SPARK-50354) No need to set initialInputBufferOffset when Aggregate mode is Complete

2024-11-19 Thread zhixingheyi-tian (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhixingheyi-tian updated SPARK-50354: - Description: Here: [https://github.com/apache/spark/blob/87a5b37ec3c4b383a5938144612c071

[jira] [Updated] (SPARK-50354) No need to set initialInputBufferOffset when Aggregate mode is Complete

2024-11-19 Thread zhixingheyi-tian (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhixingheyi-tian updated SPARK-50354: - Description: Here: [https://github.com/apache/spark/blob/87a5b37ec3c4b383a5938144612c071

[jira] [Created] (SPARK-50354) No need to set initialInputBufferOffset when Aggregate mode is Complete

2024-11-19 Thread zhixingheyi-tian (Jira)
zhixingheyi-tian created SPARK-50354: Summary: No need to set initialInputBufferOffset when Aggregate mode is Complete Key: SPARK-50354 URL: https://issues.apache.org/jira/browse/SPARK-50354 Proje

[jira] [Created] (SPARK-50352) [SPARK-50335][PYTHON][DOCS] Refine docstrings for window/aggregation functions - part 3

2024-11-19 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-50352: - Summary: [SPARK-50335][PYTHON][DOCS] Refine docstrings for window/aggregation functions - part 3 Key: SPARK-50352 URL: https://issues.apache.org/jira/browse/SPARK-50352

[jira] [Updated] (SPARK-50353) Refactor ResolveSQLOnFile

2024-11-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-50353: --- Labels: pull-request-available (was: ) > Refactor ResolveSQLOnFile > --

[jira] [Created] (SPARK-50353) Refactor ResolveSQLOnFile

2024-11-19 Thread Mihailo Aleksic (Jira)
Mihailo Aleksic created SPARK-50353: --- Summary: Refactor ResolveSQLOnFile Key: SPARK-50353 URL: https://issues.apache.org/jira/browse/SPARK-50353 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-35110) Handle ANSI intervals in WindowExecBase

2024-11-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-35110: --- Labels: pull-request-available (was: ) > Handle ANSI intervals in WindowExecBase >

[jira] [Updated] (SPARK-50352) Refine docstrings for window/aggregation functions - part 3

2024-11-19 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-50352: -- Summary: Refine docstrings for window/aggregation functions - part 3 (was: [SPARK-50335][PYTH

[jira] [Assigned] (SPARK-50350) Avro: add new function `schema_of_avro`

2024-11-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-50350: -- Assignee: Pan Bingkun (was: Apache Spark) > Avro: add new function `schema_of_avro`

[jira] [Assigned] (SPARK-50175) Change collation precedence

2024-11-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-50175: -- Assignee: Apache Spark > Change collation precedence > --- >

[jira] [Assigned] (SPARK-50175) Change collation precedence

2024-11-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-50175: -- Assignee: (was: Apache Spark) > Change collation precedence > ---

[jira] [Created] (SPARK-50351) javaoptions execution method.

2024-11-19 Thread TiSong Zhang (Jira)
TiSong Zhang created SPARK-50351: Summary: javaoptions execution method. Key: SPARK-50351 URL: https://issues.apache.org/jira/browse/SPARK-50351 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-50350) Avro: add new function `schema_of_avro`

2024-11-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-50350: --- Labels: pull-request-available (was: ) > Avro: add new function `schema_of_avro` >

[jira] [Created] (SPARK-50350) Avro: add new function `schema_of_avro`

2024-11-19 Thread Pan Bingkun (Jira)
Pan Bingkun created SPARK-50350: --- Summary: Avro: add new function `schema_of_avro` Key: SPARK-50350 URL: https://issues.apache.org/jira/browse/SPARK-50350 Project: Spark Issue Type: New Feature

[jira] [Created] (SPARK-50349) Set max value for spark.sql.autoBroadcastJoinThreshold to 8gb if more than 8gb is passed

2024-11-19 Thread Lakshminarayana Chari (Jira)
Lakshminarayana Chari created SPARK-50349: - Summary: Set max value for spark.sql.autoBroadcastJoinThreshold to 8gb if more than 8gb is passed Key: SPARK-50349 URL: https://issues.apache.org/jira/browse/SPA

[jira] [Updated] (SPARK-50348) Make static import Python data source configurable

2024-11-19 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-50348: --- Labels: pull-request-available (was: ) > Make static import Python data source configurable

[jira] [Updated] (SPARK-50348) Make static import Python data source configurable

2024-11-19 Thread Allison Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-50348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allison Wang updated SPARK-50348: - Description: Currently, Spark will automatically discover and makeĀ  > Make static import Python

[jira] [Created] (SPARK-50348) Make static import Python data source configurable

2024-11-19 Thread Allison Wang (Jira)
Allison Wang created SPARK-50348: Summary: Make static import Python data source configurable Key: SPARK-50348 URL: https://issues.apache.org/jira/browse/SPARK-50348 Project: Spark Issue Type