[jira] [Updated] (SPARK-45943) DataSourceV2Relation.computeStats throws IllegalStateException in test mode

2023-11-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45943: --- Labels: pull-request-available (was: ) > DataSourceV2Relation.computeStats throws

[jira] [Commented] (SPARK-42694) Data duplication and loss occur after executing 'insert overwrite...' in Spark 3.1.1

2023-11-16 Thread FengZhou (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17787074#comment-17787074 ] FengZhou commented on SPARK-42694: -- No. Everything is OK, all the tasks are successful. > Data

[jira] [Updated] (SPARK-45971) Correct the package name of `SparkCollectionUtils`

2023-11-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45971: --- Labels: pull-request-available (was: ) > Correct the package name of

[jira] [Resolved] (SPARK-45962) Remove treatEmptyValuesAsNulls and use nullValue option instead in XML

2023-11-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-45962. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43852

[jira] [Created] (SPARK-45971) Correct the package name of `SparkCollectionUtils`

2023-11-16 Thread Yang Jie (Jira)
Yang Jie created SPARK-45971: Summary: Correct the package name of `SparkCollectionUtils` Key: SPARK-45971 URL: https://issues.apache.org/jira/browse/SPARK-45971 Project: Spark Issue Type:

[jira] [Commented] (SPARK-45699) Fix "Widening conversion from `TypeA` to `TypeB` is deprecated because it loses precision"

2023-11-16 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17787053#comment-17787053 ] Yang Jie commented on SPARK-45699: -- [~hannahkamundson] Is there any progress on this ticket? > Fix

[jira] [Resolved] (SPARK-45966) Add missing methods for API reference.

2023-11-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-45966. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43860

[jira] [Assigned] (SPARK-45966) Add missing methods for API reference.

2023-11-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-45966: - Assignee: Haejoon Lee > Add missing methods for API reference. >

[jira] [Resolved] (SPARK-45968) Upgrade github docker action to latest version

2023-11-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-45968. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43862

[jira] [Created] (SPARK-45970) Provide partitioning expressions in Java as same as Scala

2023-11-16 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-45970: Summary: Provide partitioning expressions in Java as same as Scala Key: SPARK-45970 URL: https://issues.apache.org/jira/browse/SPARK-45970 Project: Spark

[jira] [Updated] (SPARK-45969) Document configuration change of executor failure tracker

2023-11-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45969: --- Labels: pull-request-available (was: ) > Document configuration change of executor failure

[jira] [Created] (SPARK-45969) Document configuration change of executor failure tracker

2023-11-16 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-45969: - Summary: Document configuration change of executor failure tracker Key: SPARK-45969 URL: https://issues.apache.org/jira/browse/SPARK-45969 Project: Spark Issue

[jira] [Assigned] (SPARK-45762) Shuffle managers defined in user jars are not available for some launch modes

2023-11-16 Thread Mridul Muralidharan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan reassigned SPARK-45762: --- Assignee: Alessandro Bellina > Shuffle managers defined in user jars are

[jira] [Resolved] (SPARK-45762) Shuffle managers defined in user jars are not available for some launch modes

2023-11-16 Thread Mridul Muralidharan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mridul Muralidharan resolved SPARK-45762. - Resolution: Fixed Issue resolved by pull request 43627

[jira] [Updated] (SPARK-44021) Add spark.sql.files.maxPartitionNum

2023-11-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-44021: --- Labels: pull-request-available (was: ) > Add spark.sql.files.maxPartitionNum >

[jira] [Updated] (SPARK-45966) Add missing methods for API reference.

2023-11-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45966: --- Labels: pull-request-available (was: ) > Add missing methods for API reference. >

[jira] [Resolved] (SPARK-45964) Remove private[sql] in XML and JSON package under catalyst package

2023-11-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-45964. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43856

[jira] [Updated] (SPARK-45967) Upgrade jackson to 2.16.0

2023-11-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45967: --- Labels: pull-request-available (was: ) > Upgrade jackson to 2.16.0 >

[jira] [Assigned] (SPARK-45964) Remove private[sql] in XML and JSON package under catalyst package

2023-11-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-45964: - Assignee: Hyukjin Kwon > Remove private[sql] in XML and JSON package under catalyst

[jira] [Created] (SPARK-45967) Upgrade jackson to 2.16.0

2023-11-16 Thread BingKun Pan (Jira)
BingKun Pan created SPARK-45967: --- Summary: Upgrade jackson to 2.16.0 Key: SPARK-45967 URL: https://issues.apache.org/jira/browse/SPARK-45967 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-45966) Add missing methods for API reference.

2023-11-16 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-45966: --- Summary: Add missing methods for API reference. Key: SPARK-45966 URL: https://issues.apache.org/jira/browse/SPARK-45966 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-45965) Move DSv2 partitioning expressions into functions.partitioning

2023-11-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45965: --- Labels: pull-request-available (was: ) > Move DSv2 partitioning expressions into

[jira] [Assigned] (SPARK-45952) Use built-in math constant in math functions

2023-11-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-45952: Assignee: Ruifeng Zheng > Use built-in math constant in math functions >

[jira] [Resolved] (SPARK-45952) Use built-in math constant in math functions

2023-11-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45952. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43837

[jira] [Updated] (SPARK-45964) Remove private[sql] in XML and JSON package under catalyst package

2023-11-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45964: --- Labels: pull-request-available (was: ) > Remove private[sql] in XML and JSON package under

[jira] [Updated] (SPARK-40909) Reuse the broadcast exchange for bloom filter

2023-11-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-40909: --- Labels: pull-request-available (was: ) > Reuse the broadcast exchange for bloom filter >

[jira] [Updated] (SPARK-44669) Parquet/ORC files written using Hive Serde should has file extension

2023-11-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-44669: --- Labels: pull-request-available (was: ) > Parquet/ORC files written using Hive Serde should

[jira] [Assigned] (SPARK-45912) Enhancement of XSDToSchema API: Change to HDFS API for cloud storage accessibility

2023-11-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-45912: Assignee: Shujing Yang > Enhancement of XSDToSchema API: Change to HDFS API for cloud

[jira] [Updated] (SPARK-44704) Cleanup shuffle files from host node after migration due to graceful decommissioning

2023-11-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-44704: --- Labels: pull-request-available (was: ) > Cleanup shuffle files from host node after

[jira] [Resolved] (SPARK-45912) Enhancement of XSDToSchema API: Change to HDFS API for cloud storage accessibility

2023-11-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45912. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43789

[jira] [Created] (SPARK-45964) Remove private[sql] in XML and JSON package under catalyst package

2023-11-16 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-45964: Summary: Remove private[sql] in XML and JSON package under catalyst package Key: SPARK-45964 URL: https://issues.apache.org/jira/browse/SPARK-45964 Project: Spark

[jira] [Updated] (SPARK-45963) Restore documentation for DSv2 API

2023-11-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45963: --- Labels: pull-request-available (was: ) > Restore documentation for DSv2 API >

[jira] [Created] (SPARK-45963) Restore documentation for DSv2 API

2023-11-16 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-45963: Summary: Restore documentation for DSv2 API Key: SPARK-45963 URL: https://issues.apache.org/jira/browse/SPARK-45963 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-45959) Abusing DataSet.withColumn can cause huge tree with severe perf degradation

2023-11-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45959: --- Labels: pull-request-available (was: ) > Abusing DataSet.withColumn can cause huge tree

[jira] [Assigned] (SPARK-45950) Fix IvyTestUtils#createIvyDescriptor function and make common-utils module can run tests on GitHub Action

2023-11-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-45950: Assignee: Yang Jie > Fix IvyTestUtils#createIvyDescriptor function and make common-utils

[jira] [Resolved] (SPARK-45950) Fix IvyTestUtils#createIvyDescriptor function and make common-utils module can run tests on GitHub Action

2023-11-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45950. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43834

[jira] [Resolved] (SPARK-45960) Add Python 3.10 to the Daily Python Github Action job

2023-11-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45960. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43847

[jira] [Updated] (SPARK-45961) Document `spark.master.*` configurations

2023-11-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45961: -- Fix Version/s: 3.4.2 > Document `spark.master.*` configurations >

[jira] [Updated] (SPARK-45961) Document `spark.master.*` configurations

2023-11-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45961: -- Fix Version/s: 3.5.1 > Document `spark.master.*` configurations >

[jira] [Assigned] (SPARK-45961) Document `spark.master.*` configurations

2023-11-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-45961: - Assignee: Dongjoon Hyun > Document `spark.master.*` configurations >

[jira] [Resolved] (SPARK-45961) Document `spark.master.*` configurations

2023-11-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-45961. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43848

[jira] [Updated] (SPARK-45962) Remove treatEmptyValuesAsNulls and use nullValue option instead in XML

2023-11-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45962: --- Labels: pull-request-available (was: ) > Remove treatEmptyValuesAsNulls and use nullValue

[jira] [Updated] (SPARK-45961) Document `spark.master.*` configurations

2023-11-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45961: --- Labels: pull-request-available (was: ) > Document `spark.master.*` configurations >

[jira] [Created] (SPARK-45961) Document `spark.master.*` configurations

2023-11-16 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-45961: - Summary: Document `spark.master.*` configurations Key: SPARK-45961 URL: https://issues.apache.org/jira/browse/SPARK-45961 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-45958) Upgrade Arrow to 14.0.1

2023-11-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-45958. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43846

[jira] [Resolved] (SPARK-45946) Fix use of deprecated FileUtils write in RocksDBSuite

2023-11-16 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-45946. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43832

[jira] [Updated] (SPARK-45960) Add Python 3.10 to the Daily Python Github Action job

2023-11-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45960: --- Labels: pull-request-available (was: ) > Add Python 3.10 to the Daily Python Github Action

[jira] [Created] (SPARK-45960) Add Python 3.10 to the Daily Python Github Action job

2023-11-16 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-45960: - Summary: Add Python 3.10 to the Daily Python Github Action job Key: SPARK-45960 URL: https://issues.apache.org/jira/browse/SPARK-45960 Project: Spark

[jira] [Resolved] (SPARK-45953) Add Python 3.10 to Infra docker image

2023-11-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-45953. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43840

[jira] [Assigned] (SPARK-45953) Add Python 3.10 to Infra docker image

2023-11-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-45953: - Assignee: Dongjoon Hyun > Add Python 3.10 to Infra docker image >

[jira] [Updated] (SPARK-45959) Abusing DataSet.withColumn can cause huge tree with severe perf degradation

2023-11-16 Thread Asif (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Asif updated SPARK-45959: - Priority: Minor (was: Major) > Abusing DataSet.withColumn can cause huge tree with severe perf degradation >

[jira] [Commented] (SPARK-45959) Abusing DataSet.withColumn can cause huge tree with severe perf degradation

2023-11-16 Thread Asif (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17786941#comment-17786941 ] Asif commented on SPARK-45959: -- will create a PR for the same.. > Abusing DataSet.withColumn can cause

[jira] [Created] (SPARK-45959) Abusing DataSet.withColumn can cause huge tree with severe perf degradation

2023-11-16 Thread Asif (Jira)
Asif created SPARK-45959: Summary: Abusing DataSet.withColumn can cause huge tree with severe perf degradation Key: SPARK-45959 URL: https://issues.apache.org/jira/browse/SPARK-45959 Project: Spark

[jira] [Updated] (SPARK-45958) Upgrade Arrow to 14.0.1

2023-11-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45958: --- Labels: pull-request-available (was: ) > Upgrade Arrow to 14.0.1 > ---

[jira] [Created] (SPARK-45958) Upgrade Arrow to 14.0.1

2023-11-16 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-45958: - Summary: Upgrade Arrow to 14.0.1 Key: SPARK-45958 URL: https://issues.apache.org/jira/browse/SPARK-45958 Project: Spark Issue Type: Task

[jira] [Updated] (SPARK-44924) Add configurations for FileStreamSource cached files

2023-11-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-44924: --- Labels: pull-request-available (was: ) > Add configurations for FileStreamSource cached

[jira] [Commented] (SPARK-45956) Upgrade ZooKeeper to 3.7.2

2023-11-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17786919#comment-17786919 ] Dongjoon Hyun commented on SPARK-45956: --- I collected this as a subtask of SPARK-44111 to give more

[jira] [Updated] (SPARK-45956) Upgrade ZooKeeper to 3.7.2

2023-11-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45956: -- Parent: SPARK-44111 Issue Type: Sub-task (was: Dependency upgrade) > Upgrade

[jira] [Commented] (SPARK-44118) Support K8s scheduling gates

2023-11-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17786916#comment-17786916 ] Dongjoon Hyun commented on SPARK-44118: --- We will revisit this after the feature reaches `GA` and

[jira] [Updated] (SPARK-44118) Support K8s scheduling gates

2023-11-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-44118: -- Description:

[jira] [Updated] (SPARK-44118) Support K8s scheduling gates

2023-11-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-44118: -- Parent: (was: SPARK-44111) Issue Type: Improvement (was: Sub-task) > Support K8s

[jira] [Commented] (SPARK-44118) Support K8s scheduling gates

2023-11-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17786913#comment-17786913 ] Dongjoon Hyun commented on SPARK-44118: --- This is excluded from Apache Spark 4.0.0 scope because

[jira] [Updated] (SPARK-45957) SQL on streaming Temp view fails

2023-11-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45957: --- Labels: pull-request-available (was: ) > SQL on streaming Temp view fails >

[jira] [Updated] (SPARK-45957) SQL on streaming Temp view fails

2023-11-16 Thread Raghu Angadi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi updated SPARK-45957: - Description: The following code fails in the last step with Spark Connect. The root cause is

[jira] [Resolved] (SPARK-45955) Collapse Support for Flamegraph and thread dump details

2023-11-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-45955. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43842

[jira] [Created] (SPARK-45957) SQL on streaming Temp view fails

2023-11-16 Thread Raghu Angadi (Jira)
Raghu Angadi created SPARK-45957: Summary: SQL on streaming Temp view fails Key: SPARK-45957 URL: https://issues.apache.org/jira/browse/SPARK-45957 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-45955) Collapse Support for Flamegraph and thread dump details

2023-11-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-45955: - Assignee: Kent Yao > Collapse Support for Flamegraph and thread dump details >

[jira] [Updated] (SPARK-45956) Upgrade ZooKeeper to 3.7.2

2023-11-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45956: --- Labels: pull-request-available (was: ) > Upgrade ZooKeeper to 3.7.2 >

[jira] [Updated] (SPARK-45956) Upgrade ZooKeeper to 3.7.2

2023-11-16 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-45956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bjørn Jørgensen updated SPARK-45956: Summary: Upgrade ZooKeeper to 3.7.2 (was: Upgrade ZooKeeper to X.X) > Upgrade ZooKeeper

[jira] [Created] (SPARK-45956) Upgrade ZooKeeper to X.X

2023-11-16 Thread Jira
Bjørn Jørgensen created SPARK-45956: --- Summary: Upgrade ZooKeeper to X.X Key: SPARK-45956 URL: https://issues.apache.org/jira/browse/SPARK-45956 Project: Spark Issue Type: Dependency

[jira] [Updated] (SPARK-45920) group by ordinal should be idempotent

2023-11-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45920: -- Fix Version/s: 3.3.4 > group by ordinal should be idempotent >

[jira] [Updated] (SPARK-45920) group by ordinal should be idempotent

2023-11-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45920: -- Fix Version/s: 3.4.2 > group by ordinal should be idempotent >

[jira] [Updated] (SPARK-45920) group by ordinal should be idempotent

2023-11-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45920: -- Fix Version/s: 3.5.1 > group by ordinal should be idempotent >

[jira] [Updated] (SPARK-43980) Add support for EXCEPT in select clause, similar to what databricks provides

2023-11-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-43980: --- Labels: pull-request-available (was: ) > Add support for EXCEPT in select clause, similar

[jira] [Updated] (SPARK-45954) Avoid generating redundant ShuffleExchangeExec node

2023-11-16 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-45954: Summary: Avoid generating redundant ShuffleExchangeExec node (was: Remove redundant shuffles) >

[jira] [Resolved] (SPARK-45951) Upgrade buf to v1.28.1

2023-11-16 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie resolved SPARK-45951. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43835

[jira] [Assigned] (SPARK-45951) Upgrade buf to v1.28.1

2023-11-16 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie reassigned SPARK-45951: Assignee: Ruifeng Zheng > Upgrade buf to v1.28.1 > -- > >

[jira] [Updated] (SPARK-45955) Collapse Support for Flamegraph and thread dump details

2023-11-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45955: --- Labels: pull-request-available (was: ) > Collapse Support for Flamegraph and thread dump

[jira] [Created] (SPARK-45955) Collapse Support for Flamegraph and thread dump details

2023-11-16 Thread Kent Yao (Jira)
Kent Yao created SPARK-45955: Summary: Collapse Support for Flamegraph and thread dump details Key: SPARK-45955 URL: https://issues.apache.org/jira/browse/SPARK-45955 Project: Spark Issue Type:

[jira] [Updated] (SPARK-45954) Remove redundant shuffles

2023-11-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45954: --- Labels: pull-request-available (was: ) > Remove redundant shuffles >

[jira] [Created] (SPARK-45954) Remove redundant shuffles

2023-11-16 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-45954: --- Summary: Remove redundant shuffles Key: SPARK-45954 URL: https://issues.apache.org/jira/browse/SPARK-45954 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-45950) Fix IvyTestUtils#createIvyDescriptor function and make common-utils module can run tests on GitHub Action

2023-11-16 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-45950: - Summary: Fix IvyTestUtils#createIvyDescriptor function and make common-utils module can run tests on

[jira] [Updated] (SPARK-45950) Fix IvyTestUtils#createIvyDescriptor function and make common-utils module can run tests on GitHub Action

2023-11-16 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-45950: - Component/s: Spark Core > Fix IvyTestUtils#createIvyDescriptor function and make common-utils module >

[jira] [Updated] (SPARK-45950) Fix IvyTestUtils#createIvyDescriptor function and make common-utils module can run tests on GitHub Action

2023-11-16 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-45950: - Issue Type: Bug (was: Improvement) > Fix IvyTestUtils#createIvyDescriptor function and make

[jira] [Commented] (SPARK-45414) spark-xml misplaces string tag content

2023-11-16 Thread Giuseppe Ceravolo (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17786700#comment-17786700 ] Giuseppe Ceravolo commented on SPARK-45414: --- [~ritikam] I appreciate your support, but I do

[jira] [Resolved] (SPARK-45851) (Scala) Support different retry policies for connect client

2023-11-16 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45851. -- Fix Version/s: 4.0.0 Assignee: Alice Sayutina Resolution: Fixed Fixed in

[jira] [Resolved] (SPARK-45945) Add a helper function for `parser`

2023-11-16 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-45945. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43826

[jira] [Assigned] (SPARK-45953) Add Python 3.10 to Infra docker image

2023-11-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45953: -- Assignee: Apache Spark > Add Python 3.10 to Infra docker image >

[jira] [Assigned] (SPARK-45953) Add Python 3.10 to Infra docker image

2023-11-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45953: -- Assignee: (was: Apache Spark) > Add Python 3.10 to Infra docker image >

[jira] [Assigned] (SPARK-45953) Add Python 3.10 to Infra docker image

2023-11-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45953: -- Assignee: (was: Apache Spark) > Add Python 3.10 to Infra docker image >

[jira] [Assigned] (SPARK-45953) Add Python 3.10 to Infra docker image

2023-11-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45953: -- Assignee: Apache Spark > Add Python 3.10 to Infra docker image >

[jira] [Updated] (SPARK-45953) Add Python 3.10 to Infra docker image

2023-11-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45953: --- Labels: pull-request-available (was: ) > Add Python 3.10 to Infra docker image >

[jira] [Created] (SPARK-45953) Add Python 3.10 to Infra docker image

2023-11-16 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-45953: - Summary: Add Python 3.10 to Infra docker image Key: SPARK-45953 URL: https://issues.apache.org/jira/browse/SPARK-45953 Project: Spark Issue Type: Task

[jira] [Assigned] (SPARK-45929) support grouping set operation in dataframe api

2023-11-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45929: -- Assignee: (was: Apache Spark) > support grouping set operation in dataframe api

[jira] [Assigned] (SPARK-45929) support grouping set operation in dataframe api

2023-11-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45929: -- Assignee: Apache Spark > support grouping set operation in dataframe api >

[jira] [Assigned] (SPARK-45952) Use built-in math constant in math functions

2023-11-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45952: -- Assignee: (was: Apache Spark) > Use built-in math constant in math functions >

[jira] [Assigned] (SPARK-45952) Use built-in math constant in math functions

2023-11-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45952: -- Assignee: Apache Spark > Use built-in math constant in math functions >

[jira] [Updated] (SPARK-45952) Use built-in math constant in math functions

2023-11-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45952: --- Labels: pull-request-available (was: ) > Use built-in math constant in math functions >

[jira] [Assigned] (SPARK-45929) support grouping set operation in dataframe api

2023-11-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45929: -- Assignee: (was: Apache Spark) > support grouping set operation in dataframe api

[jira] [Assigned] (SPARK-45929) support grouping set operation in dataframe api

2023-11-16 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-45929: -- Assignee: Apache Spark > support grouping set operation in dataframe api >

  1   2   >