[jira] [Assigned] (SPARK-46825) Build Spark only once when building docs

2024-01-23 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-46825: Assignee: Nicholas Chammas > Build Spark only once when building docs >

[jira] [Resolved] (SPARK-46825) Build Spark only once when building docs

2024-01-23 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-46825. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44865

[jira] [Updated] (SPARK-46826) Reset grpcio installation version of `Python linter dependencies for branch-3.4/branch-3.5`

2024-01-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46826: --- Labels: pull-request-available (was: ) > Reset grpcio installation version of `Python

[jira] [Updated] (SPARK-46826) Reset the installed grpcio version of `Python linter dependencies for branch-3.4/branch-3.5`

2024-01-23 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-46826: - Summary: Reset the installed grpcio version of `Python linter dependencies for branch-3.4/branch-3.5`

[jira] [Updated] (SPARK-46826) Reset grpcio installation version of `Python linter dependencies for branch-3.4/branch-3.5`

2024-01-23 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-46826: - Summary: Reset grpcio installation version of `Python linter dependencies for branch-3.4/branch-3.5`

[jira] [Created] (SPARK-46826) Reset the installed version of grpcio in `Python linter dependencies for branch-3.4/branch-3.5`

2024-01-23 Thread Yang Jie (Jira)
Yang Jie created SPARK-46826: Summary: Reset the installed version of grpcio in `Python linter dependencies for branch-3.4/branch-3.5` Key: SPARK-46826 URL: https://issues.apache.org/jira/browse/SPARK-46826

[jira] [Assigned] (SPARK-46733) Simplify the ContextCleaner|BlockManager by the exit operation only depend on interrupt thread

2024-01-23 Thread Mridul Muralidharan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-46733: --- Assignee: Jiaan Geng > Simplify the ContextCleaner|BlockManager by the

[jira] [Resolved] (SPARK-46733) Simplify the ContextCleaner|BlockManager by the exit operation only depend on interrupt thread

2024-01-23 Thread Mridul Muralidharan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-46733. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request

[jira] [Resolved] (SPARK-46822) Respect spark.sql.legacy.charVarcharAsString when casting jdbc type to catalyst type in jdbc

2024-01-23 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-46822. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44860

[jira] [Updated] (SPARK-46825) Build Spark only once when building docs

2024-01-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46825: --- Labels: pull-request-available (was: ) > Build Spark only once when building docs >

[jira] [Created] (SPARK-46825) Build Spark only once when building docs

2024-01-23 Thread Nicholas Chammas (Jira)
Nicholas Chammas created SPARK-46825: Summary: Build Spark only once when building docs Key: SPARK-46825 URL: https://issues.apache.org/jira/browse/SPARK-46825 Project: Spark Issue Type:

[jira] [Updated] (SPARK-46824) Enable test without optional dependency on PyPy

2024-01-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46824: --- Labels: pull-request-available (was: ) > Enable test without optional dependency on PyPy >

[jira] [Created] (SPARK-46824) Enable test without optional dependency on PyPy

2024-01-23 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-46824: --- Summary: Enable test without optional dependency on PyPy Key: SPARK-46824 URL: https://issues.apache.org/jira/browse/SPARK-46824 Project: Spark Issue Type:

[jira] [Updated] (SPARK-46819) Port error class data to automation-friendly format

2024-01-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46819: --- Labels: pull-request-available (was: ) > Port error class data to automation-friendly

[jira] [Updated] (SPARK-46820) Fix error message regression by restoring new_msg

2024-01-23 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-46820: Description: >>> from pyspark.sql.types import StructType, StructField, StringType, >>>

[jira] [Updated] (SPARK-46820) Fix error message regression by restoring new_msg

2024-01-23 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-46820: Summary: Fix error message regression by restoring new_msg (was: Improve error message when

[jira] [Updated] (SPARK-46823) LocalDataToArrowConversion should check the nullability

2024-01-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46823: --- Labels: pull-request-available (was: ) > LocalDataToArrowConversion should check the

[jira] [Created] (SPARK-46823) LocalDataToArrowConversion should check the nullability

2024-01-23 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-46823: - Summary: LocalDataToArrowConversion should check the nullability Key: SPARK-46823 URL: https://issues.apache.org/jira/browse/SPARK-46823 Project: Spark

[jira] [Updated] (SPARK-46822) Respect spark.sql.legacy.charVarcharAsString when casting jdbc type to catalyst type in jdbc

2024-01-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46822: --- Labels: pull-request-available (was: ) > Respect spark.sql.legacy.charVarcharAsString when

[jira] [Created] (SPARK-46822) Respect spark.sql.legacy.charVarcharAsString when casting jdbc type to catalyst type in jdbc

2024-01-23 Thread Kent Yao (Jira)
Kent Yao created SPARK-46822: Summary: Respect spark.sql.legacy.charVarcharAsString when casting jdbc type to catalyst type in jdbc Key: SPARK-46822 URL: https://issues.apache.org/jira/browse/SPARK-46822

[jira] [Updated] (SPARK-46812) Make `mapInPandas` / mapInArrow` support ResourceProfile (Stage-Level scheduling)

2024-01-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46812: --- Labels: pull-request-available (was: ) > Make `mapInPandas` / mapInArrow` support

[jira] [Updated] (SPARK-43306) Migrate `ValueError` from Spark SQL types into error class

2024-01-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-43306: --- Labels: pull-request-available (was: ) > Migrate `ValueError` from Spark SQL types into

[jira] [Updated] (SPARK-46820) Improve error message when createDataFrame have illegal nullable

2024-01-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46820: --- Labels: pull-request-available (was: ) > Improve error message when createDataFrame have

[jira] [Assigned] (SPARK-46811) Github Action: TypeError: Cannot read properties of undefined (reading 'head_sha')

2024-01-23 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-46811: Assignee: Kent Yao > Github Action: TypeError: Cannot read properties of undefined

[jira] [Resolved] (SPARK-46811) Github Action: TypeError: Cannot read properties of undefined (reading 'head_sha')

2024-01-23 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-46811. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44850

[jira] [Resolved] (SPARK-43403) GET /history//1/jobs/ failed: java.lang.IllegalStateException: DB is closed

2024-01-23 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-43403. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44851

[jira] [Assigned] (SPARK-43403) GET /history//1/jobs/ failed: java.lang.IllegalStateException: DB is closed

2024-01-23 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao reassigned SPARK-43403: Assignee: Zhou Yifan > GET /history//1/jobs/ failed: java.lang.IllegalStateException: DB is >

[jira] [Commented] (SPARK-46810) Clarify error class terminology

2024-01-23 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17810156#comment-17810156 ] Haejoon Lee commented on SPARK-46810: - cc [~maxgekk] who drives SQL side error message improvement

[jira] [Resolved] (SPARK-46821) Remove pandas dependency from assertDataFrameEqual properly

2024-01-23 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee resolved SPARK-46821. - Resolution: Fixed > Remove pandas dependency from assertDataFrameEqual properly >

[jira] [Assigned] (SPARK-46728) Check pandas installation properly

2024-01-23 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-46728: Assignee: Haejoon Lee > Check pandas installation properly >

[jira] [Resolved] (SPARK-46728) Check pandas installation properly

2024-01-23 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-46728. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44745

[jira] [Updated] (SPARK-44815) Cache Schema of DF

2024-01-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-44815: --- Labels: pull-request-available (was: ) > Cache Schema of DF > -- > >

[jira] [Created] (SPARK-46821) Remove pandas dependency from assertDataFrameEqual properly

2024-01-23 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-46821: --- Summary: Remove pandas dependency from assertDataFrameEqual properly Key: SPARK-46821 URL: https://issues.apache.org/jira/browse/SPARK-46821 Project: Spark

[jira] [Created] (SPARK-46820) Improve error message when createDataFrame have illegal nullable

2024-01-23 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-46820: --- Summary: Improve error message when createDataFrame have illegal nullable Key: SPARK-46820 URL: https://issues.apache.org/jira/browse/SPARK-46820 Project: Spark

[jira] [Assigned] (SPARK-46817) Fix `spark-daemon.sh` usage by adding `decommission` command

2024-01-23 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-46817: - Assignee: Dongjoon Hyun > Fix `spark-daemon.sh` usage by adding `decommission` command

[jira] [Resolved] (SPARK-46817) Fix `spark-daemon.sh` usage by adding `decommission` command

2024-01-23 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-46817. --- Fix Version/s: 3.4.3 3.5.1 4.0.0 Resolution:

[jira] [Created] (SPARK-46819) Port error class data to automation-friendly format

2024-01-23 Thread Nicholas Chammas (Jira)
Nicholas Chammas created SPARK-46819: Summary: Port error class data to automation-friendly format Key: SPARK-46819 URL: https://issues.apache.org/jira/browse/SPARK-46819 Project: Spark

[jira] [Updated] (SPARK-46810) Clarify error class terminology

2024-01-23 Thread Nicholas Chammas (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-46810: - Description: We use inconsistent terminology when talking about error classes. I'd like

[jira] [Resolved] (SPARK-46764) Reorganize Ruby script to build API docs

2024-01-23 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-46764. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44791

[jira] [Assigned] (SPARK-46764) Reorganize Ruby script to build API docs

2024-01-23 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-46764: Assignee: Nicholas Chammas > Reorganize Ruby script to build API docs >

[jira] [Updated] (SPARK-46818) Improve error messages for range with non-foldable input

2024-01-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46818: --- Labels: pull-request-available (was: ) > Improve error messages for range with

[jira] [Created] (SPARK-46818) Improve error messages for range with non-foldable input

2024-01-23 Thread Allison Wang (Jira)
Allison Wang created SPARK-46818: Summary: Improve error messages for range with non-foldable input Key: SPARK-46818 URL: https://issues.apache.org/jira/browse/SPARK-46818 Project: Spark

[jira] [Updated] (SPARK-46817) Fix `spark-daemon.sh` usage by adding `decommission` command

2024-01-23 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-46817: -- Affects Version/s: 3.3.4 3.5.0 3.4.2

[jira] [Updated] (SPARK-46817) Fix `spark-daemon.sh` usage by adding `decommission` command

2024-01-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46817: --- Labels: pull-request-available (was: ) > Fix `spark-daemon.sh` usage by adding

[jira] [Created] (SPARK-46817) Fix `spark-daemon.sh` usage by adding `decommission` command

2024-01-23 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-46817: - Summary: Fix `spark-daemon.sh` usage by adding `decommission` command Key: SPARK-46817 URL: https://issues.apache.org/jira/browse/SPARK-46817 Project: Spark

[jira] [Updated] (SPARK-46817) Fix `spark-daemon.sh` usage by adding `decommission` command

2024-01-23 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-46817: -- Summary: Fix `spark-daemon.sh` usage by adding `decommission` command (was: Fix

[jira] [Updated] (SPARK-46817) Fix `spark-daemon.sh` usage msg by adding `decommission` command

2024-01-23 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-46817: -- Summary: Fix `spark-daemon.sh` usage msg by adding `decommission` command (was: Fix

[jira] [Updated] (SPARK-46816) Base implementation for new operator and state API changes

2024-01-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46816: --- Labels: pull-request-available (was: ) > Base implementation for new operator and state

[jira] [Commented] (SPARK-46816) Base implementation for new operator and state API changes

2024-01-23 Thread Anish Shrigondekar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17810071#comment-17810071 ] Anish Shrigondekar commented on SPARK-46816: PR here:

[jira] [Created] (SPARK-46816) Base implementation for new operator and state API changes

2024-01-23 Thread Anish Shrigondekar (Jira)
Anish Shrigondekar created SPARK-46816: -- Summary: Base implementation for new operator and state API changes Key: SPARK-46816 URL: https://issues.apache.org/jira/browse/SPARK-46816 Project: Spark

[jira] [Updated] (SPARK-45939) SPIP: Structured Streaming - Arbitrary State API v2

2024-01-23 Thread Anish Shrigondekar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anish Shrigondekar updated SPARK-45939: --- Epic Link: SPARK-46815 > SPIP: Structured Streaming - Arbitrary State API v2 >

[jira] [Updated] (SPARK-46815) Structured Streaming - Arbitrary State API v2

2024-01-23 Thread Anish Shrigondekar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anish Shrigondekar updated SPARK-46815: --- Description: We are planning to introduce a new operator forĀ [Spark Structured

[jira] [Created] (SPARK-46815) Structured Streaming - Arbitrary State API v2

2024-01-23 Thread Anish Shrigondekar (Jira)
Anish Shrigondekar created SPARK-46815: -- Summary: Structured Streaming - Arbitrary State API v2 Key: SPARK-46815 URL: https://issues.apache.org/jira/browse/SPARK-46815 Project: Spark

[jira] [Resolved] (SPARK-46795) Replace UnsupportedOperationException by SparkUnsupportedOperationException in sql/core

2024-01-23 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-46795. -- Resolution: Fixed Issue resolved by pull request 44772 [https://github.com/apache/spark/pull/44772]

[jira] [Commented] (SPARK-45404) Support AWS_ENDPOINT_URL env variable

2024-01-23 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17810058#comment-17810058 ] Dongjoon Hyun commented on SPARK-45404: --- Thank you, [~ste...@apache.org]. > Support

[jira] [Resolved] (SPARK-46794) Incorrect results due to inferred predicate from checkpoint with subquery

2024-01-23 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-46794. --- Fix Version/s: 3.4.3 3.5.1 4.0.0 Resolution:

[jira] [Updated] (SPARK-46746) Attach codec extension to avro datasource files

2024-01-23 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-46746: -- Parent: SPARK-44111 Issue Type: Sub-task (was: Improvement) > Attach codec extension

[jira] [Updated] (SPARK-46759) Codec xz and zstandard support compression level for avro files

2024-01-23 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-46759: -- Parent: SPARK-44111 Issue Type: Sub-task (was: Improvement) > Codec xz and zstandard

[jira] [Updated] (SPARK-46772) Benchmarking Avro with Compression Codecs

2024-01-23 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-46772: -- Summary: Benchmarking Avro with Compression Codecs (was: Benchmarking Avro with ZSTD ) >

[jira] [Resolved] (SPARK-46772) Benchmarking Avro with ZSTD

2024-01-23 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-46772. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44849

[jira] [Created] (SPARK-46814) Build and Run with Java 21

2024-01-23 Thread Madhavan (Jira)
Madhavan created SPARK-46814: Summary: Build and Run with Java 21 Key: SPARK-46814 URL: https://issues.apache.org/jira/browse/SPARK-46814 Project: Spark Issue Type: New Feature

[jira] [Updated] (SPARK-46813) Don't set the executor id to "driver" when SparkContext is created by the executor side

2024-01-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46813: --- Labels: pull-request-available (was: ) > Don't set the executor id to "driver" when

[jira] [Created] (SPARK-46813) Don't set the executor id to "driver" when SparkContext is created by the executor side

2024-01-23 Thread xiaoping.huang (Jira)
xiaoping.huang created SPARK-46813: -- Summary: Don't set the executor id to "driver" when SparkContext is created by the executor side Key: SPARK-46813 URL: https://issues.apache.org/jira/browse/SPARK-46813

[jira] [Created] (SPARK-46812) Make `mapInPandas` / mapInArrow` support ResourceProfile (Stage-Level scheduling)

2024-01-23 Thread Bobby Wang (Jira)
Bobby Wang created SPARK-46812: -- Summary: Make `mapInPandas` / mapInArrow` support ResourceProfile (Stage-Level scheduling) Key: SPARK-46812 URL: https://issues.apache.org/jira/browse/SPARK-46812

[jira] [Closed] (SPARK-44460) Pass user auth credential to Python workers for foreachBatch and listener

2024-01-23 Thread Wei Liu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Liu closed SPARK-44460. --- > Pass user auth credential to Python workers for foreachBatch and listener >

[jira] [Resolved] (SPARK-46683) Write a subquery generator that generates subqueries of different variations to increase testing coverage in this area

2024-01-23 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-46683. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44599

[jira] [Reopened] (SPARK-46417) do not fail when calling getTable and throwException is false

2024-01-23 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao reopened SPARK-46417: -- Assignee: (was: Wenchen Fan) Ok to be reverted by

[jira] [Updated] (SPARK-46417) do not fail when calling getTable and throwException is false

2024-01-23 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-46417: - Fix Version/s: (was: 4.0.0) (was: 3.5.1) (was: 3.4.3)

[jira] [Updated] (SPARK-46811) Github Action: TypeError: Cannot read properties of undefined (reading 'head_sha')

2024-01-23 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-46811: - Summary: Github Action: TypeError: Cannot read properties of undefined (reading 'head_sha') (was:

[jira] [Resolved] (SPARK-46763) ReplaceDeduplicateWithAggregate fails when non-grouping keys have duplicate attributes

2024-01-23 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-46763. - Fix Version/s: 3.4.3 3.5.1 4.0.0 Resolution: Fixed

[jira] [Assigned] (SPARK-46763) ReplaceDeduplicateWithAggregate fails when non-grouping keys have duplicate attributes

2024-01-23 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-46763: --- Assignee: Nikhil Sheoran > ReplaceDeduplicateWithAggregate fails when non-grouping keys

[jira] [Updated] (SPARK-46811) Github ActionL: TypeError: Cannot read properties of undefined (reading 'head_sha')

2024-01-23 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46811: --- Labels: pull-request-available (was: ) > Github ActionL: TypeError: Cannot read properties

[jira] [Created] (SPARK-46811) Github ActionL: TypeError: Cannot read properties of undefined (reading 'head_sha')

2024-01-23 Thread Kent Yao (Jira)
Kent Yao created SPARK-46811: Summary: Github ActionL: TypeError: Cannot read properties of undefined (reading 'head_sha') Key: SPARK-46811 URL: https://issues.apache.org/jira/browse/SPARK-46811

[jira] [Updated] (SPARK-46810) Clarify error class terminology

2024-01-23 Thread Nicholas Chammas (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicholas Chammas updated SPARK-46810: - Description: We use inconsistent terminology when talking about error classes. I'd like

[jira] [Commented] (SPARK-46810) Clarify error class terminology

2024-01-23 Thread Nicholas Chammas (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17809804#comment-17809804 ] Nicholas Chammas commented on SPARK-46810: -- [~itholic] [~gurwls223] - What do you think? cc

[jira] [Created] (SPARK-46810) Clarify error class terminology

2024-01-23 Thread Nicholas Chammas (Jira)
Nicholas Chammas created SPARK-46810: Summary: Clarify error class terminology Key: SPARK-46810 URL: https://issues.apache.org/jira/browse/SPARK-46810 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-46590) Coalesce partiton assert error after skew join optimization

2024-01-23 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-46590. - Fix Version/s: 3.5.1 4.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-46590) Coalesce partiton assert error after skew join optimization

2024-01-23 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-46590: --- Assignee: Jackey Lee > Coalesce partiton assert error after skew join optimization >