[jira] [Updated] (SPARK-46378) still remove Sort after converting Aggregate to Project

2023-12-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46378: --- Labels: pull-request-available (was: ) > still remove Sort after converting Aggregate to

[jira] [Created] (SPARK-46378) still remove Sort after converting Aggregate to Project

2023-12-11 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-46378: --- Summary: still remove Sort after converting Aggregate to Project Key: SPARK-46378 URL: https://issues.apache.org/jira/browse/SPARK-46378 Project: Spark Issue

[jira] [Updated] (SPARK-46376) Simplify the code to generate the Spark tarball `filename` in the `HiveExternalCatalogVersionsSuite`.

2023-12-11 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-46376: - Summary: Simplify the code to generate the Spark tarball `filename` in the

[jira] [Updated] (SPARK-46376) Simplify the way to generate the Spark tarball `filename` in the `HiveExternalCatalogVersionsSuite`.

2023-12-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46376: --- Labels: pull-request-available (was: ) > Simplify the way to generate the Spark tarball

[jira] [Created] (SPARK-46376) Simplify the way to generate the Spark tarball `filename` in the `HiveExternalCatalogVersionsSuite`.

2023-12-11 Thread Yang Jie (Jira)
Yang Jie created SPARK-46376: Summary: Simplify the way to generate the Spark tarball `filename` in the `HiveExternalCatalogVersionsSuite`. Key: SPARK-46376 URL: https://issues.apache.org/jira/browse/SPARK-46376

[jira] [Resolved] (SPARK-46240) Add ExecutedPlanPrepRules to SparkSessionExtensions

2023-12-11 Thread jiang13021 (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiang13021 resolved SPARK-46240. Resolution: Won't Do As the discussion in [https://github.com/apache/spark/pull/44254] columnar

[jira] [Created] (SPARK-46375) Add documentation for Python data source API

2023-12-11 Thread Allison Wang (Jira)
Allison Wang created SPARK-46375: Summary: Add documentation for Python data source API Key: SPARK-46375 URL: https://issues.apache.org/jira/browse/SPARK-46375 Project: Spark Issue Type:

[jira] [Updated] (SPARK-46371) Clean up outdated items in `.rat-excludes`

2023-12-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46371: --- Labels: pull-request-available (was: ) > Clean up outdated items in `.rat-excludes` >

[jira] [Commented] (SPARK-46032) connect: cannot assign instance of java.lang.invoke.SerializedLambda to field org.apache.spark.rdd.MapPartitionsRDD.f

2023-12-11 Thread Bobby Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17795594#comment-17795594 ] Bobby Wang commented on SPARK-46032: Thx [~haldefaiz] ,  I tried your patch with another two

[jira] [Assigned] (SPARK-46360) Enhance error message debugging with new `getMessage` API

2023-12-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-46360: Assignee: Haejoon Lee > Enhance error message debugging with new `getMessage` API >

[jira] [Resolved] (SPARK-46360) Enhance error message debugging with new `getMessage` API

2023-12-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-46360. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44292

[jira] [Commented] (SPARK-46374) Array Indexing is 1-based via ANSI SQL Standard

2023-12-11 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17795584#comment-17795584 ] Kent Yao commented on SPARK-46374: -- cc [~cloud_fan] [~dongjoon] [~gurwls223] [~maropu] > Array

[jira] [Updated] (SPARK-46374) Array Indexing is 1-based via ANSI SQL Standard

2023-12-11 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-46374: - Description: SPARK-28402 was closed by the reporter, because he might think it's a postgres specific

[jira] [Updated] (SPARK-46374) Array Indexing is 1-based via ANSI SQL Standard

2023-12-11 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-46374: - Description: SPARK-28402 was closed by the reporter, because he might think it's a postgres specific

[jira] [Updated] (SPARK-46374) Array Indexing is 1-based via ANSI SQL Standard

2023-12-11 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-46374: - Description: SPARK-28402 was closed > Array Indexing is 1-based via ANSI SQL Standard >

[jira] [Updated] (SPARK-46374) Array Indexing is 1-based via ANSI SQL Standard

2023-12-11 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-46374: - Description: SPARK-28402 was closed by the reporter, because he might think it's a postgres specific

[jira] [Commented] (SPARK-46373) Create DataFrame Bug

2023-12-11 Thread Bleibtreu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17795581#comment-17795581 ] Bleibtreu commented on SPARK-46373: --- now, It can be bypassed using Java api {code:java} def

[jira] [Created] (SPARK-46374) Array Indexing is 1-based via ANSI SQL Standard

2023-12-11 Thread Kent Yao (Jira)
Kent Yao created SPARK-46374: Summary: Array Indexing is 1-based via ANSI SQL Standard Key: SPARK-46374 URL: https://issues.apache.org/jira/browse/SPARK-46374 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-46373) Create DataFrame Bug

2023-12-11 Thread jiaqi.zhu (Jira)
jiaqi.zhu created SPARK-46373: - Summary: Create DataFrame Bug Key: SPARK-46373 URL: https://issues.apache.org/jira/browse/SPARK-46373 Project: Spark Issue Type: Bug Components: Spark

[jira] [Created] (SPARK-46372) "Invalid call to toAttribute on unresolved object" instead UNRESOLVED_COLUMN.WITH_SUGGESTION on INSERT statement

2023-12-11 Thread Serge Rielau (Jira)
Serge Rielau created SPARK-46372: Summary: "Invalid call to toAttribute on unresolved object" instead UNRESOLVED_COLUMN.WITH_SUGGESTION on INSERT statement Key: SPARK-46372 URL:

[jira] [Created] (SPARK-46371) Clean up outdated items in `.rat-excludes`

2023-12-11 Thread BingKun Pan (Jira)
BingKun Pan created SPARK-46371: --- Summary: Clean up outdated items in `.rat-excludes` Key: SPARK-46371 URL: https://issues.apache.org/jira/browse/SPARK-46371 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-43427) Unsigned integer types are deserialized as signed numeric equivalents

2023-12-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-43427: Assignee: Parth Upadhyay > Unsigned integer types are deserialized as signed numeric

[jira] [Resolved] (SPARK-43427) Unsigned integer types are deserialized as signed numeric equivalents

2023-12-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-43427. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43773

[jira] [Updated] (SPARK-46369) Remove `kill` link from RELAUNCHING drivers in MasterPage

2023-12-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-46369: -- Fix Version/s: 3.5.1 3.4.3 > Remove `kill` link from RELAUNCHING drivers

[jira] [Resolved] (SPARK-46369) Remove `kill` link from RELAUNCHING drivers in MasterPage

2023-12-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-46369. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44301

[jira] [Updated] (SPARK-38668) Spark on Kubernetes: add separate pod watcher service to reduce pressure on K8s API server

2023-12-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-38668: --- Labels: pull-request-available (was: ) > Spark on Kubernetes: add separate pod watcher

[jira] [Resolved] (SPARK-46351) Require an error class in AnalysisException

2023-12-11 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-46351. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44277

[jira] [Commented] (SPARK-46370) Fix bug when querying from table after changing defaults

2023-12-11 Thread Daniel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17795514#comment-17795514 ] Daniel commented on SPARK-46370: The bug is here:   {{drop table if exists t;}} {{create table t(i

[jira] [Created] (SPARK-46370) Fix bug when querying from table after changing defaults

2023-12-11 Thread Daniel (Jira)
Daniel created SPARK-46370: -- Summary: Fix bug when querying from table after changing defaults Key: SPARK-46370 URL: https://issues.apache.org/jira/browse/SPARK-46370 Project: Spark Issue Type:

[jira] [Updated] (SPARK-46369) Remove `kill` link from RELAUNCHING drivers in MasterPage

2023-12-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-46369: -- Affects Version/s: 3.5.0 3.4.2 2.4.8

[jira] [Assigned] (SPARK-46369) Remove `kill` link from RELAUNCHING drivers in MasterPage

2023-12-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-46369: - Assignee: Dongjoon Hyun > Remove `kill` link from RELAUNCHING drivers in MasterPage >

[jira] [Updated] (SPARK-46369) Remove `kill` link from RELAUNCHING drivers in MasterPage

2023-12-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46369: --- Labels: pull-request-available (was: ) > Remove `kill` link from RELAUNCHING drivers in

[jira] [Created] (SPARK-46369) Remove `kill` link from RELAUNCHING drivers in MasterPage

2023-12-11 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-46369: - Summary: Remove `kill` link from RELAUNCHING drivers in MasterPage Key: SPARK-46369 URL: https://issues.apache.org/jira/browse/SPARK-46369 Project: Spark

[jira] [Assigned] (SPARK-38615) SQL Error Attribution Framework

2023-12-11 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang reassigned SPARK-38615: -- Assignee: Apache Spark > SQL Error Attribution Framework >

[jira] [Resolved] (SPARK-38615) SQL Error Attribution Framework

2023-12-11 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-38615. Resolution: Implemented > SQL Error Attribution Framework >

[jira] [Resolved] (SPARK-40174) Add query contexts to SparkException

2023-12-11 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-40174. Resolution: Won't Fix > Add query contexts to SparkException >

[jira] [Updated] (SPARK-38615) SQL Error Attribution Framework

2023-12-11 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-38615: --- Attachment: SQL Error Attribution Framework.pdf > SQL Error Attribution Framework >

[jira] [Updated] (SPARK-38615) SQL Error Attribution Framework

2023-12-11 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-38615: --- Description: Currently,  there is not enough error context for runtime ANSI failures. In

[jira] [Updated] (SPARK-46368) Support `/readyz` API

2023-12-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-46368: -- Description: Like https://kubernetes.io/docs/reference/using-api/health-checks/, we need to

[jira] [Updated] (SPARK-46351) Require an error class in AnalysisException

2023-12-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46351: --- Labels: pull-request-available (was: ) > Require an error class in AnalysisException >

[jira] [Created] (SPARK-46368) Support `/readyz` API

2023-12-11 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-46368: - Summary: Support `/readyz` API Key: SPARK-46368 URL: https://issues.apache.org/jira/browse/SPARK-46368 Project: Spark Issue Type: Sub-task

[jira] [Assigned] (SPARK-46353) Refactor to improve `RegisterWorker` unit test

2023-12-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-46353: - Assignee: Dongjoon Hyun > Refactor to improve `RegisterWorker` unit test >

[jira] [Updated] (SPARK-38615) SQL Error Attribution Framework

2023-12-11 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-38615: --- Epic Link: (was: SPARK-35030) > SQL Error Attribution Framework >

[jira] [Resolved] (SPARK-46273) Support INSERT INTO/OVERWRITE using DSv2 sources

2023-12-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-46273. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44213

[jira] [Assigned] (SPARK-46273) Support INSERT INTO/OVERWRITE using DSv2 sources

2023-12-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-46273: Assignee: Allison Wang > Support INSERT INTO/OVERWRITE using DSv2 sources >

[jira] [Resolved] (SPARK-46347) Reorganize `RollingTests `

2023-12-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-46347. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44281

[jira] [Assigned] (SPARK-46347) Reorganize `RollingTests `

2023-12-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-46347: Assignee: Ruifeng Zheng > Reorganize `RollingTests ` > -- > >

[jira] [Created] (SPARK-46367) PartitioningPreservingUnaryExecNode breaks KeyGroupedPartitioning

2023-12-11 Thread Jira
Никита Соколов created SPARK-46367: -- Summary: PartitioningPreservingUnaryExecNode breaks KeyGroupedPartitioning Key: SPARK-46367 URL: https://issues.apache.org/jira/browse/SPARK-46367 Project: Spark

[jira] [Resolved] (SPARK-46355) XML: Close InputStreamReader on read completion

2023-12-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-46355. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44287

[jira] [Assigned] (SPARK-46355) XML: Close InputStreamReader on read completion

2023-12-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-46355: Assignee: Sandip Agarwala > XML: Close InputStreamReader on read completion >

[jira] [Resolved] (SPARK-46358) Mino refactor of `ResponseValidator#verifyResponse`

2023-12-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-46358. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44291

[jira] [Assigned] (SPARK-46358) Mino refactor of `ResponseValidator#verifyResponse`

2023-12-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-46358: - Assignee: Yang Jie > Mino refactor of `ResponseValidator#verifyResponse` >

[jira] [Resolved] (SPARK-46356) Upgrade `sbt-assembly` to 2.1.5, `sbt-checkstyle-plugin` to 4.0.1

2023-12-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-46356. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44288

[jira] [Created] (SPARK-46366) Use with expression to avoid duplicating expressions in BETWEEN operation

2023-12-11 Thread Aleksandar Tomic (Jira)
Aleksandar Tomic created SPARK-46366: Summary: Use with expression to avoid duplicating expressions in BETWEEN operation Key: SPARK-46366 URL: https://issues.apache.org/jira/browse/SPARK-46366

[jira] [Updated] (SPARK-46365) Spark 3.5.0 Regression: Window Function Combination Yields Null Values

2023-12-11 Thread Boris PEREVALOV (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Boris PEREVALOV updated SPARK-46365: Description: When combining two window functions (first one to get the previous non-null

[jira] [Created] (SPARK-46365) Spark 3.5.0 Regression: Window Function Combination Yields Null Values

2023-12-11 Thread Boris PEREVALOV (Jira)
Boris PEREVALOV created SPARK-46365: --- Summary: Spark 3.5.0 Regression: Window Function Combination Yields Null Values Key: SPARK-46365 URL: https://issues.apache.org/jira/browse/SPARK-46365

[jira] [Updated] (SPARK-46349) Prevent SortOrder from Accepting Nested SortOrder Instances

2023-12-11 Thread Majid Hajiheidari (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Majid Hajiheidari updated SPARK-46349: -- Labels: (was: pull-request-available) > Prevent SortOrder from Accepting Nested

[jira] [Updated] (SPARK-46364) Push calculation from Aggregate through Expand

2023-12-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46364: --- Labels: pull-request-available (was: ) > Push calculation from Aggregate through Expand >

[jira] [Created] (SPARK-46364) Push calculation from Aggregate through Expand

2023-12-11 Thread Wan Kun (Jira)
Wan Kun created SPARK-46364: --- Summary: Push calculation from Aggregate through Expand Key: SPARK-46364 URL: https://issues.apache.org/jira/browse/SPARK-46364 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-46310) Cannot deploy Spark application using VolcanoFeatureStep to specify podGroupTemplate file

2023-12-11 Thread Lucca Sergi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17795320#comment-17795320 ] Lucca Sergi commented on SPARK-46310: - Just saw [https://github.com/volcano-sh/volcano/issues/3250] 

[jira] [Created] (SPARK-46363) Improve the java text block with java15 feature

2023-12-11 Thread Jiaan Geng (Jira)
Jiaan Geng created SPARK-46363: -- Summary: Improve the java text block with java15 feature Key: SPARK-46363 URL: https://issues.apache.org/jira/browse/SPARK-46363 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-45649) Unify the prepare framework for `OffsetWindowFunctionFrame`

2023-12-11 Thread Jiaan Geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiaan Geng resolved SPARK-45649. Resolution: Fixed > Unify the prepare framework for `OffsetWindowFunctionFrame` >

[jira] [Created] (SPARK-46362) calculation error

2023-12-11 Thread Paride Casulli (Jira)
Paride Casulli created SPARK-46362: -- Summary: calculation error Key: SPARK-46362 URL: https://issues.apache.org/jira/browse/SPARK-46362 Project: Spark Issue Type: Bug Components:

[jira] [Updated] (SPARK-46361) Add spark dataset chunk read API (python only)

2023-12-11 Thread Weichen Xu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weichen Xu updated SPARK-46361: --- Description: *Proposed API:* {code:java} def persist_dataframe_as_chunks(dataframe: DataFrame) ->

[jira] [Updated] (SPARK-46361) Add spark dataset chunk read API (python only)

2023-12-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46361: --- Labels: pull-request-available (was: ) > Add spark dataset chunk read API (python only) >

[jira] [Created] (SPARK-46361) Add spark dataset chunk read API (python only)

2023-12-11 Thread Weichen Xu (Jira)
Weichen Xu created SPARK-46361: -- Summary: Add spark dataset chunk read API (python only) Key: SPARK-46361 URL: https://issues.apache.org/jira/browse/SPARK-46361 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-46302) Fix maven daily testing

2023-12-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-46302: -- Assignee: (was: Apache Spark) > Fix maven daily testing >

[jira] [Assigned] (SPARK-46302) Fix maven daily testing

2023-12-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-46302: -- Assignee: Apache Spark > Fix maven daily testing > --- > >

[jira] [Assigned] (SPARK-46273) Support INSERT INTO/OVERWRITE using DSv2 sources

2023-12-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-46273: -- Assignee: (was: Apache Spark) > Support INSERT INTO/OVERWRITE using DSv2 sources

[jira] [Assigned] (SPARK-46273) Support INSERT INTO/OVERWRITE using DSv2 sources

2023-12-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-46273: -- Assignee: Apache Spark > Support INSERT INTO/OVERWRITE using DSv2 sources >

[jira] [Assigned] (SPARK-46273) Support INSERT INTO/OVERWRITE using DSv2 sources

2023-12-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-46273: -- Assignee: (was: Apache Spark) > Support INSERT INTO/OVERWRITE using DSv2 sources

[jira] [Assigned] (SPARK-46273) Support INSERT INTO/OVERWRITE using DSv2 sources

2023-12-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-46273: -- Assignee: Apache Spark > Support INSERT INTO/OVERWRITE using DSv2 sources >

[jira] [Assigned] (SPARK-46273) Support INSERT INTO/OVERWRITE using DSv2 sources

2023-12-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-46273: -- Assignee: (was: Apache Spark) > Support INSERT INTO/OVERWRITE using DSv2 sources

[jira] [Assigned] (SPARK-46273) Support INSERT INTO/OVERWRITE using DSv2 sources

2023-12-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-46273: -- Assignee: Apache Spark > Support INSERT INTO/OVERWRITE using DSv2 sources >

[jira] [Assigned] (SPARK-46302) Fix maven daily testing

2023-12-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-46302: -- Assignee: (was: Apache Spark) > Fix maven daily testing >

[jira] [Assigned] (SPARK-46302) Fix maven daily testing

2023-12-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-46302: -- Assignee: Apache Spark > Fix maven daily testing > --- > >

[jira] [Updated] (SPARK-28989) Introduce ANSI SQL Dialect

2023-12-11 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-28989: --- Summary: Introduce ANSI SQL Dialect (was: Add `spark.sql.ansi.enabled`) > Introduce ANSI

[jira] [Updated] (SPARK-35161) Error-handling SQL functions for ANSI SQL mode migration

2023-12-11 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-35161: --- Summary: Error-handling SQL functions for ANSI SQL mode migration (was: Error-handling SQL

[jira] [Updated] (SPARK-46356) Upgrade `sbt-assembly` to 2.1.5, `sbt-checkstyle-plugin` to 4.0.1

2023-12-11 Thread BingKun Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BingKun Pan updated SPARK-46356: Summary: Upgrade `sbt-assembly` to 2.1.5, `sbt-checkstyle-plugin` to 4.0.1 (was: Upgrade

[jira] [Commented] (SPARK-44003) DynamicPartitionDataSingleWriter is being starved by Parquet MemoryManager

2023-12-11 Thread Rafal Wojdyla (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17795227#comment-17795227 ] Rafal Wojdyla commented on SPARK-44003: --- Reported this issue in Parquet: