[jira] [Updated] (SPARK-48499) Use Math.abs to get positive numbers

[ https://issues.apache.org/jira/browse/SPARK-48499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48499: --- Labels: pull-request-available (was: ) > Use Math.abs to get positive numbers > ---

[jira] [Updated] (SPARK-48499) Use Math.abs to get positive numbers

[ https://issues.apache.org/jira/browse/SPARK-48499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junqing Li updated SPARK-48499: --- Summary: Use Math.abs to get positive numbers (was: Use Math.abs to get Unsigned Numbers) > Use Ma

[jira] [Created] (SPARK-48499) Use Math.abs to get Unsigned Numbers

Junqing Li created SPARK-48499: -- Summary: Use Math.abs to get Unsigned Numbers Key: SPARK-48499 URL: https://issues.apache.org/jira/browse/SPARK-48499 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-48497) Add user guide for batch data source write API

[ https://issues.apache.org/jira/browse/SPARK-48497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48497: --- Labels: pull-request-available (was: ) > Add user guide for batch data source write API > -

[jira] [Created] (SPARK-48497) Add user guide for batch data source write API

Allison Wang created SPARK-48497: Summary: Add user guide for batch data source write API Key: SPARK-48497 URL: https://issues.apache.org/jira/browse/SPARK-48497 Project: Spark Issue Type: Su

[jira] [Assigned] (SPARK-48490) Unescapes any literals for message of MessageWithContext

[ https://issues.apache.org/jira/browse/SPARK-48490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang reassigned SPARK-48490: -- Assignee: BingKun Pan > Unescapes any literals for message of MessageWithContext > --

[jira] [Resolved] (SPARK-48490) Unescapes any literals for message of MessageWithContext

[ https://issues.apache.org/jira/browse/SPARK-48490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-48490. Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46824 [https:

[jira] [Assigned] (SPARK-48391) use addAll instead of add function in TaskMetrics to accelerate

[ https://issues.apache.org/jira/browse/SPARK-48391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-48391: --- Assignee: jiahong.li > use addAll instead of add function in TaskMetrics to accelerate >

[jira] [Resolved] (SPARK-48391) use addAll instead of add function in TaskMetrics to accelerate

[ https://issues.apache.org/jira/browse/SPARK-48391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-48391. - Fix Version/s: 3.5.2 4.0.0 Resolution: Fixed Issue resolved by pull re

[jira] [Updated] (SPARK-48466) Wrap empty relation propagation in a dedicated node

[ https://issues.apache.org/jira/browse/SPARK-48466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48466: --- Labels: pull-request-available (was: ) > Wrap empty relation propagation in a dedicated nod

[jira] [Updated] (SPARK-48491) Refactor HiveWindowFunctionQuerySuite in BeforeAll and AfterAll

[ https://issues.apache.org/jira/browse/SPARK-48491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48491: --- Labels: pull-request-available (was: ) > Refactor HiveWindowFunctionQuerySuite in BeforeAll

[jira] [Created] (SPARK-48495) Document planned approach to shredding

David Cashman created SPARK-48495: - Summary: Document planned approach to shredding Key: SPARK-48495 URL: https://issues.apache.org/jira/browse/SPARK-48495 Project: Spark Issue Type: Sub-task

[jira] [Resolved] (SPARK-48465) Avoid no-op empty relation propagation in AQE

[ https://issues.apache.org/jira/browse/SPARK-48465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-48465. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46814 [https://gith

[jira] [Updated] (SPARK-48494) Update airlift:aircompressor to 0.27

[ https://issues.apache.org/jira/browse/SPARK-48494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48494: --- Labels: pull-request-available (was: ) > Update airlift:aircompressor to 0.27 > ---

[jira] [Updated] (SPARK-48493) Enhance Python Datasource Reader with Arrow Batch Support for Improved Performance

[ https://issues.apache.org/jira/browse/SPARK-48493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luca Canali updated SPARK-48493: Description: This enhancement adds an option to the Python Datasource Reader to yield Arrow batch

[jira] [Updated] (SPARK-48493) Enhance Python Datasource Reader with Arrow Batch Support for Improved Performance

[ https://issues.apache.org/jira/browse/SPARK-48493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48493: --- Labels: pull-request-available (was: ) > Enhance Python Datasource Reader with Arrow Batch

[jira] [Created] (SPARK-48493) Enhance Python Datasource Reader with Arrow Batch Support for Improved Performance

Luca Canali created SPARK-48493: --- Summary: Enhance Python Datasource Reader with Arrow Batch Support for Improved Performance Key: SPARK-48493 URL: https://issues.apache.org/jira/browse/SPARK-48493 Proj

[jira] [Updated] (SPARK-48492) batch-read parquet files written by streaming returns non-nullable fields in schema

[ https://issues.apache.org/jira/browse/SPARK-48492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julien Peloton updated SPARK-48492: --- Description: Hello, In the documentation, it is stated that > When reading Parquet files,

[jira] [Updated] (SPARK-48492) batch-read parquet files written by streaming returns non-nullable fields in schema

[ https://issues.apache.org/jira/browse/SPARK-48492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julien Peloton updated SPARK-48492: --- Description: Hello, In the documentation, it is stated that > When reading Parquet files,

[jira] [Updated] (SPARK-48492) batch-read parquet files written by streaming returns non-nullable fields in schema

[ https://issues.apache.org/jira/browse/SPARK-48492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julien Peloton updated SPARK-48492: --- Description: Hello, In the documentation, it is stated that > When reading Parquet files,

[jira] [Updated] (SPARK-48492) batch-read parquet files written by streaming returns non-nullable fields in schema

[ https://issues.apache.org/jira/browse/SPARK-48492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julien Peloton updated SPARK-48492: --- Description: Hello, In the documentation, it is stated that > When reading Parquet files,

[jira] [Updated] (SPARK-48492) batch-read parquet files written by streaming returns non-nullable fields in schema

[ https://issues.apache.org/jira/browse/SPARK-48492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julien Peloton updated SPARK-48492: --- Description: Hello, In the documentation, it is stated that > When reading Parquet files,

[jira] [Updated] (SPARK-48492) batch-read parquet files written by streaming returns non-nullable fields in schema

[ https://issues.apache.org/jira/browse/SPARK-48492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julien Peloton updated SPARK-48492: --- Description: Hello, In the documentation, it is stated that > When reading Parquet files,

[jira] [Updated] (SPARK-48492) batch-read parquet files written by streaming returns non-nullable fields in schema

[ https://issues.apache.org/jira/browse/SPARK-48492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julien Peloton updated SPARK-48492: --- Description: Hello, In the documentation, it is stated that > When reading Parquet files,

[jira] [Created] (SPARK-48492) batch-read parquet files written by streaming returns non-nullable fields in schema

Julien Peloton created SPARK-48492: -- Summary: batch-read parquet files written by streaming returns non-nullable fields in schema Key: SPARK-48492 URL: https://issues.apache.org/jira/browse/SPARK-48492

[jira] [Created] (SPARK-48491) Refactor HiveWindowFunctionQuerySuite in BeforeAll and AfterAll

Rui Wang created SPARK-48491: Summary: Refactor HiveWindowFunctionQuerySuite in BeforeAll and AfterAll Key: SPARK-48491 URL: https://issues.apache.org/jira/browse/SPARK-48491 Project: Spark Issu

[jira] [Updated] (SPARK-47172) Upgrade Transport block cipher mode to GCM

[ https://issues.apache.org/jira/browse/SPARK-47172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Weis updated SPARK-47172: --- Due Date: 12/Jun/24 (was: 26/Mar/24) > Upgrade Transport block cipher mode to GCM > ---

[jira] [Resolved] (SPARK-48430) Fix map value extraction when map contains collated strings

[ https://issues.apache.org/jira/browse/SPARK-48430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-48430. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46758 [https://gith

[jira] [Resolved] (SPARK-48476) NPE thrown when delimiter set to null in CSV

[ https://issues.apache.org/jira/browse/SPARK-48476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-48476. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46810 [https://gith

[jira] [Assigned] (SPARK-48476) NPE thrown when delimiter set to null in CSV

[ https://issues.apache.org/jira/browse/SPARK-48476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-48476: --- Assignee: Milan Stefanovic > NPE thrown when delimiter set to null in CSV > ---

[jira] [Updated] (SPARK-48490) Unescapes any literals for message of MessageWithContext

[ https://issues.apache.org/jira/browse/SPARK-48490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48490: --- Labels: pull-request-available (was: ) > Unescapes any literals for message of MessageWithC

[jira] [Assigned] (SPARK-48484) V2Write use the same TaskAttemptId for different task attempts

[ https://issues.apache.org/jira/browse/SPARK-48484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie reassigned SPARK-48484: Assignee: Jackey Lee > V2Write use the same TaskAttemptId for different task attempts > -

[jira] [Created] (SPARK-48490) Unescapes any literals for message of MessageWithContext

BingKun Pan created SPARK-48490: --- Summary: Unescapes any literals for message of MessageWithContext Key: SPARK-48490 URL: https://issues.apache.org/jira/browse/SPARK-48490 Project: Spark Issue

[jira] [Resolved] (SPARK-48484) V2Write use the same TaskAttemptId for different task attempts

[ https://issues.apache.org/jira/browse/SPARK-48484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie resolved SPARK-48484. -- Fix Version/s: 3.4.4 3.5.2 4.0.0 Resolution: Fixed Issue

[jira] [Updated] (SPARK-48489) Throw an user-facing error when reading invalid schema from text DataSource

[ https://issues.apache.org/jira/browse/SPARK-48489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48489: --- Labels: pull-request-available (was: ) > Throw an user-facing error when reading invalid sc

[jira] [Created] (SPARK-48489) Throw an user-facing error when reading invalid schema from text DataSource

Stefan Bukorovic created SPARK-48489: Summary: Throw an user-facing error when reading invalid schema from text DataSource Key: SPARK-48489 URL: https://issues.apache.org/jira/browse/SPARK-48489 P

[jira] [Updated] (SPARK-47690) Hash aggregate support for strings with collation

[ https://issues.apache.org/jira/browse/SPARK-47690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47690: --- Labels: pull-request-available (was: ) > Hash aggregate support for strings with collation

[jira] [Updated] (SPARK-48488) Restore the original logic of methods `log[info|warning|error]` in `SparkSubmit`

[ https://issues.apache.org/jira/browse/SPARK-48488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48488: --- Labels: pull-request-available (was: ) > Restore the original logic of methods `log[info|wa

[jira] [Created] (SPARK-48488) Restore the original logic of methods `log[info|warning|error]` in `SparkSubmit`

BingKun Pan created SPARK-48488: --- Summary: Restore the original logic of methods `log[info|warning|error]` in `SparkSubmit` Key: SPARK-48488 URL: https://issues.apache.org/jira/browse/SPARK-48488 Projec

[jira] [Updated] (SPARK-48487) Update License & Notice according to the dependency changes

[ https://issues.apache.org/jira/browse/SPARK-48487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48487: --- Labels: pull-request-available (was: ) > Update License & Notice according to the dependenc

[jira] [Created] (SPARK-48487) Update License & Notice according to the dependency changes

Kent Yao created SPARK-48487: Summary: Update License & Notice according to the dependency changes Key: SPARK-48487 URL: https://issues.apache.org/jira/browse/SPARK-48487 Project: Spark Issue Ty

[jira] [Assigned] (SPARK-48476) NPE thrown when delimiter set to null in CSV

[ https://issues.apache.org/jira/browse/SPARK-48476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-48476: -- Assignee: (was: Apache Spark) > NPE thrown when delimiter set to null in CSV > --

[jira] [Assigned] (SPARK-48476) NPE thrown when delimiter set to null in CSV

[ https://issues.apache.org/jira/browse/SPARK-48476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-48476: -- Assignee: Apache Spark > NPE thrown when delimiter set to null in CSV > -

[jira] [Assigned] (SPARK-47258) Assign error classes to SHOW CREATE TABLE errors

[ https://issues.apache.org/jira/browse/SPARK-47258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-47258: -- Assignee: Apache Spark > Assign error classes to SHOW CREATE TABLE errors > -

[jira] [Assigned] (SPARK-47258) Assign error classes to SHOW CREATE TABLE errors

[ https://issues.apache.org/jira/browse/SPARK-47258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-47258: -- Assignee: (was: Apache Spark) > Assign error classes to SHOW CREATE TABLE errors

[jira] [Commented] (SPARK-48486) The Dynamic Partition Pruning (DPP) feature in Spark can cause the generated Directed Acyclic Graph (DAG) to expand.

[ https://issues.apache.org/jira/browse/SPARK-48486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17851000#comment-17851000 ] chenfengbin commented on SPARK-48486: - The reason for the problem starting from Spar

[jira] [Updated] (SPARK-48486) The Dynamic Partition Pruning (DPP) feature in Spark can cause the generated Directed Acyclic Graph (DAG) to expand.

[ https://issues.apache.org/jira/browse/SPARK-48486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chenfengbin updated SPARK-48486: Attachment: create_table.sql > The Dynamic Partition Pruning (DPP) feature in Spark can cause the

[jira] [Updated] (SPARK-48486) The Dynamic Partition Pruning (DPP) feature in Spark can cause the generated Directed Acyclic Graph (DAG) to expand.

[ https://issues.apache.org/jira/browse/SPARK-48486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chenfengbin updated SPARK-48486: Attachment: create_table.sql.rtf > The Dynamic Partition Pruning (DPP) feature in Spark can cause

[jira] [Updated] (SPARK-48486) The Dynamic Partition Pruning (DPP) feature in Spark can cause the generated Directed Acyclic Graph (DAG) to expand.

[ https://issues.apache.org/jira/browse/SPARK-48486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chenfengbin updated SPARK-48486: Attachment: (was: create_table.sql.rtf) > The Dynamic Partition Pruning (DPP) feature in Spark

[jira] [Updated] (SPARK-48486) The Dynamic Partition Pruning (DPP) feature in Spark can cause the generated Directed Acyclic Graph (DAG) to expand.

[ https://issues.apache.org/jira/browse/SPARK-48486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chenfengbin updated SPARK-48486: Description: The Dynamic Partition Pruning (DPP) feature in Spark can cause the generated Directe

[jira] [Updated] (SPARK-48486) The Dynamic Partition Pruning (DPP) feature in Spark can cause the generated Directed Acyclic Graph (DAG) to expand.

[ https://issues.apache.org/jira/browse/SPARK-48486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chenfengbin updated SPARK-48486: Description: The Dynamic Partition Pruning (DPP) feature in Spark can cause the generated Directe

[jira] [Updated] (SPARK-48486) The Dynamic Partition Pruning (DPP) feature in Spark can cause the generated Directed Acyclic Graph (DAG) to expand.

[ https://issues.apache.org/jira/browse/SPARK-48486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chenfengbin updated SPARK-48486: Description: The Dynamic Partition Pruning (DPP) feature in Spark can cause the generated Directe

[jira] [Updated] (SPARK-48486) The Dynamic Partition Pruning (DPP) feature in Spark can cause the generated Directed Acyclic Graph (DAG) to expand.

[ https://issues.apache.org/jira/browse/SPARK-48486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chenfengbin updated SPARK-48486: Description: The Dynamic Partition Pruning (DPP) feature in Spark can cause the generated Directe

[jira] [Created] (SPARK-48486) The Dynamic Partition Pruning (DPP) feature in Spark can cause the generated Directed Acyclic Graph (DAG) to expand.

chenfengbin created SPARK-48486: --- Summary: The Dynamic Partition Pruning (DPP) feature in Spark can cause the generated Directed Acyclic Graph (DAG) to expand. Key: SPARK-48486 URL: https://issues.apache.org/jira/br

[jira] [Updated] (SPARK-48485) Support interruptTag and interruptAll in streaming queries

[ https://issues.apache.org/jira/browse/SPARK-48485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-48485: --- Labels: pull-request-available (was: ) > Support interruptTag and interruptAll in streaming

[jira] [Created] (SPARK-48485) Support interruptTag and interruptAll in streaming queries

Hyukjin Kwon created SPARK-48485: Summary: Support interruptTag and interruptAll in streaming queries Key: SPARK-48485 URL: https://issues.apache.org/jira/browse/SPARK-48485 Project: Spark Is

[jira] [Updated] (SPARK-38506) Push partial aggregation through join

[ https://issues.apache.org/jira/browse/SPARK-38506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-38506: --- Labels: pull-request-available (was: ) > Push partial aggregation through join > --