[jira] [Commented] (SPARK-47497) Make `to_csv` support some types of data

2024-03-20 Thread BingKun Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17829378#comment-17829378 ] BingKun Pan commented on SPARK-47497: - I work on it > Make `to_csv` support some types of data >

[jira] [Updated] (SPARK-47497) Make `to_csv` support some types of data

2024-03-20 Thread BingKun Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BingKun Pan updated SPARK-47497: Summary: Make `to_csv` support some types of data (was: Upgrade github action cache to v4) >

[jira] [Updated] (SPARK-47008) Spark to support S3 Express One Zone Storage

2024-03-20 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47008: -- Affects Version/s: 4.0.0 (was: 3.5.1) > Spark to support S3

[jira] [Updated] (SPARK-47008) Spark to support S3 Express One Zone Storage

2024-03-20 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47008: -- Parent Issue: SPARK-44111 (was: SPARK-44124) > Spark to support S3 Express One Zone Storage

[jira] [Closed] (SPARK-46793) Revert S3A endpoint fixup logic of SPARK-35878

2024-03-20 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-46793. - > Revert S3A endpoint fixup logic of SPARK-35878 > -- >

[jira] [Resolved] (SPARK-46793) Revert S3A endpoint fixup logic of SPARK-35878

2024-03-20 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-46793. --- Resolution: Duplicate > Revert S3A endpoint fixup logic of SPARK-35878 >

[jira] [Resolved] (SPARK-45721) Upgrade AWS SDK to v2 for Hadoop dependency

2024-03-20 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-45721. --- Resolution: Duplicate > Upgrade AWS SDK to v2 for Hadoop dependency >

[jira] [Closed] (SPARK-45721) Upgrade AWS SDK to v2 for Hadoop dependency

2024-03-20 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-45721. - > Upgrade AWS SDK to v2 for Hadoop dependency > --- > >

[jira] [Assigned] (SPARK-47491) Re-enable `driver log links` test in YarnClusterSuite

2024-03-20 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47491: - Assignee: Dongjoon Hyun > Re-enable `driver log links` test in YarnClusterSuite >

[jira] [Updated] (SPARK-47054) Remove pinned version of torch for Python 3.12 support

2024-03-20 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47054: -- Parent: SPARK-44111 Issue Type: Sub-task (was: Bug) > Remove pinned version of torch

[jira] [Assigned] (SPARK-46920) Improve executor exit error message on YARN

2024-03-20 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie reassigned SPARK-46920: Assignee: Cheng Pan > Improve executor exit error message on YARN >

[jira] [Resolved] (SPARK-46920) Improve executor exit error message on YARN

2024-03-20 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie resolved SPARK-46920. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44951

[jira] [Updated] (SPARK-41888) Support StreamingQueryListener for DataFrame.observe

2024-03-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-41888: --- Labels: pull-request-available (was: ) > Support StreamingQueryListener for

[jira] [Updated] (SPARK-47496) Java SPI Support for JDBC dialect registering

2024-03-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47496: --- Labels: pull-request-available (was: ) > Java SPI Support for JDBC dialect registering >

[jira] [Created] (SPARK-47496) Java SPI Support for JDBC dialect registering

2024-03-20 Thread Kent Yao (Jira)
Kent Yao created SPARK-47496: Summary: Java SPI Support for JDBC dialect registering Key: SPARK-47496 URL: https://issues.apache.org/jira/browse/SPARK-47496 Project: Spark Issue Type: Sub-task

[jira] [Comment Edited] (SPARK-47086) Upgrade Jetty to 12

2024-03-20 Thread HiuFung Kwok (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17829372#comment-17829372 ] HiuFung Kwok edited comment on SPARK-47086 at 3/21/24 4:10 AM: --- Due to the

[jira] [Commented] (SPARK-47086) Upgrade Jetty to 12

2024-03-20 Thread HiuFung Kwok (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17829372#comment-17829372 ] HiuFung Kwok commented on SPARK-47086: -- Due to the Jetty handler structure change, Jetty 11 to 12

[jira] [Commented] (SPARK-47495) Primary resource jar added to spark.jars twice under k8s cluster mode

2024-03-20 Thread Jiale Tan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17829354#comment-17829354 ] Jiale Tan commented on SPARK-47495: --- Proposing a fix in https://github.com/apache/spark/pull/45607 >

[jira] [Updated] (SPARK-47495) Primary resource jar added to spark.jars twice under k8s cluster mode

2024-03-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47495: --- Labels: pull-request-available (was: ) > Primary resource jar added to spark.jars twice

[jira] [Updated] (SPARK-47495) Primary resource jar added to spark.jars twice under k8s cluster mode

2024-03-20 Thread Jiale Tan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiale Tan updated SPARK-47495: -- Description: {*}Context{*}: To submit spark jobs to Kubernetes under cluster mode, the

[jira] [Updated] (SPARK-47495) Primary resource jar added to spark.jars twice under k8s cluster mode

2024-03-20 Thread Jiale Tan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiale Tan updated SPARK-47495: -- Shepherd: Dongjoon Hyun > Primary resource jar added to spark.jars twice under k8s cluster mode >

[jira] [Updated] (SPARK-47475) Jar Download Under K8s Cluster Mode Causes Executors Scaling Issues

2024-03-20 Thread Jiale Tan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiale Tan updated SPARK-47475: -- Description: {*}Issue{*}: When jars are big and the application requests a lot of executors, the

[jira] [Updated] (SPARK-47475) Jar Download Under K8s Cluster Mode Causes Executors Scaling Issues

2024-03-20 Thread Jiale Tan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiale Tan updated SPARK-47475: -- Description: {*}Context{*}: To submit spark jobs to Kubernetes under cluster mode, the

[jira] [Created] (SPARK-47495) Primary resource jar added to spark.jars twice under k8s cluster mode

2024-03-20 Thread Jiale Tan (Jira)
Jiale Tan created SPARK-47495: - Summary: Primary resource jar added to spark.jars twice under k8s cluster mode Key: SPARK-47495 URL: https://issues.apache.org/jira/browse/SPARK-47495 Project: Spark

[jira] [Resolved] (SPARK-47494) Add migration doc for the behavior change of Parquet timestamp inference since Spark 3.3

2024-03-20 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47494. --- Fix Version/s: 3.4.3 3.5.2 4.0.0 Resolution:

[jira] [Updated] (SPARK-47494) Add migration doc for the behavior change of Parquet timestamp inference since Spark 3.3

2024-03-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47494: --- Labels: pull-request-available (was: ) > Add migration doc for the behavior change of

[jira] [Resolved] (SPARK-47490) Fix RocksDB Logger deprecation warning

2024-03-20 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47490. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45616

[jira] [Assigned] (SPARK-47490) Fix RocksDB Logger deprecation warning

2024-03-20 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47490: - Assignee: Anish Shrigondekar > Fix RocksDB Logger deprecation warning >

[jira] [Updated] (SPARK-47486) Remove unused private `ArrowDeserializers.getString` method

2024-03-20 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47486: -- Summary: Remove unused private `ArrowDeserializers.getString` method (was: Remove unused

[jira] [Resolved] (SPARK-47486) Remove unused private method `getString` from `ArrowDeserializers`

2024-03-20 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47486. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45610

[jira] [Updated] (SPARK-47256) Assign error classes to FILTER expression errors

2024-03-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47256: --- Labels: pull-request-available starter (was: starter) > Assign error classes to FILTER

[jira] [Created] (SPARK-47494) Add migration doc for the behavior change of Parquet timestamp inference since Spark 3.3

2024-03-20 Thread Gengliang Wang (Jira)
Gengliang Wang created SPARK-47494: -- Summary: Add migration doc for the behavior change of Parquet timestamp inference since Spark 3.3 Key: SPARK-47494 URL: https://issues.apache.org/jira/browse/SPARK-47494

[jira] [Updated] (SPARK-47475) Jar Download Under K8s Cluster Mode Causes Executors Scaling Issues

2024-03-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47475: --- Labels: pull-request-available (was: ) > Jar Download Under K8s Cluster Mode Causes

[jira] [Updated] (SPARK-47493) Disable spark.sql.parquet.inferTimestampNTZ.enabled by default

2024-03-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47493: --- Labels: pull-request-available (was: ) > Disable

[jira] [Created] (SPARK-47493) Disable spark.sql.parquet.inferTimestampNTZ.enabled by default

2024-03-20 Thread Gengliang Wang (Jira)
Gengliang Wang created SPARK-47493: -- Summary: Disable spark.sql.parquet.inferTimestampNTZ.enabled by default Key: SPARK-47493 URL: https://issues.apache.org/jira/browse/SPARK-47493 Project: Spark

[jira] [Updated] (SPARK-47490) Fix RocksDB Logger deprecation warning

2024-03-20 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47490: -- Issue Type: Improvement (was: Task) > Fix RocksDB Logger deprecation warning >

[jira] [Updated] (SPARK-47475) Jar Download Under K8s Cluster Mode Causes Executors Scaling Issues

2024-03-20 Thread Jiale Tan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiale Tan updated SPARK-47475: -- Description: {*}Context{*}: To submit spark jobs to Kubernetes under cluster mode, the

[jira] [Updated] (SPARK-47475) Jar Download Under K8s Cluster Mode Causes Executors Scaling Issues

2024-03-20 Thread Jiale Tan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiale Tan updated SPARK-47475: -- Description: {*}Context{*}: To submit spark jobs to Kubernetes under cluster mode, the

[jira] [Updated] (SPARK-47492) Relax definition of whitespace in lexer

2024-03-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47492: --- Labels: pull-request-available (was: ) > Relax definition of whitespace in lexer >

[jira] [Updated] (SPARK-47492) Relax definition of whitespace in lexer

2024-03-20 Thread Serge Rielau (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Serge Rielau updated SPARK-47492: - Summary: Relax definition of whitespace in lexer (was: Wide definition of whitespace in lexer)

[jira] [Created] (SPARK-47492) Wide definition of whitespace in lexer

2024-03-20 Thread Serge Rielau (Jira)
Serge Rielau created SPARK-47492: Summary: Wide definition of whitespace in lexer Key: SPARK-47492 URL: https://issues.apache.org/jira/browse/SPARK-47492 Project: Spark Issue Type:

[jira] [Updated] (SPARK-47491) Re-enable `driver log links` test in YarnClusterSuite

2024-03-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47491: --- Labels: pull-request-available (was: ) > Re-enable `driver log links` test in

[jira] [Resolved] (SPARK-45393) Upgrade Hadoop to 3.4.0

2024-03-20 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-45393. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45583

[jira] [Commented] (SPARK-46762) Spark Connect 3.5 Classloading issue with external jar

2024-03-20 Thread nirav patel (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17829258#comment-17829258 ] nirav patel commented on SPARK-46762: - [~zhenli] any updates on this? > Spark Connect 3.5

[jira] [Created] (SPARK-47491) Re-enable `driver log links` test in YarnClusterSuite

2024-03-20 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-47491: - Summary: Re-enable `driver log links` test in YarnClusterSuite Key: SPARK-47491 URL: https://issues.apache.org/jira/browse/SPARK-47491 Project: Spark

[jira] [Updated] (SPARK-47490) Fix RocksDB Logger deprecation warning

2024-03-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47490: --- Labels: pull-request-available (was: ) > Fix RocksDB Logger deprecation warning >

[jira] [Created] (SPARK-47490) Fix RocksDB Logger deprecation warning

2024-03-20 Thread Anish Shrigondekar (Jira)
Anish Shrigondekar created SPARK-47490: -- Summary: Fix RocksDB Logger deprecation warning Key: SPARK-47490 URL: https://issues.apache.org/jira/browse/SPARK-47490 Project: Spark Issue

[jira] [Updated] (SPARK-47489) Action performed on Dataset is triggering an error as "The Spark SQL phase planning failed with an internal error."

2024-03-20 Thread Jagadeesh Marada (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jagadeesh Marada updated SPARK-47489: - Description: Hi team,  We are upgrading to spark 3.4.2 from 3.3.2 and facing issue in

[jira] [Updated] (SPARK-47489) Action performed on Dataset is triggering an error as "The Spark SQL phase planning failed with an internal error."

2024-03-20 Thread Jagadeesh Marada (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jagadeesh Marada updated SPARK-47489: - Attachment: code snippet.jpeg jars added to spark_submit.jpeg

[jira] [Created] (SPARK-47489) Action performed on Dataset is triggering an error as "The Spark SQL phase planning failed with an internal error."

2024-03-20 Thread Jagadeesh Marada (Jira)
Jagadeesh Marada created SPARK-47489: Summary: Action performed on Dataset is triggering an error as "The Spark SQL phase planning failed with an internal error." Key: SPARK-47489 URL:

[jira] [Resolved] (SPARK-47462) Align the mapping ways of other unsigned numeric types with TINYINT UNSGINED in MySQLDialect

2024-03-20 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47462. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45588

[jira] [Updated] (SPARK-47358) StringRepeat (all collations)

2024-03-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47358: --- Labels: pull-request-available (was: ) > StringRepeat (all collations) >

[jira] [Resolved] (SPARK-47443) Window aggregate support

2024-03-20 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-47443. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45568

[jira] [Assigned] (SPARK-47443) Window aggregate support

2024-03-20 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-47443: Assignee: Aleksandar Tomic > Window aggregate support > > >

[jira] [Assigned] (SPARK-46990) Regression: Unable to load empty avro files emitted by event-hubs

2024-03-20 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-46990: --- Assignee: Ivan Sadikov > Regression: Unable to load empty avro files emitted by event-hubs

[jira] [Resolved] (SPARK-46990) Regression: Unable to load empty avro files emitted by event-hubs

2024-03-20 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-46990. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45578

[jira] [Resolved] (SPARK-47296) fail all unsupported functions

2024-03-20 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-47296. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45422

[jira] [Updated] (SPARK-47488) [k8s]Driver stuck when thread pool is not shut down

2024-03-20 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47488: -- Affects Version/s: 4.0.0 (was: 3.2.0) > [k8s]Driver stuck when

[jira] [Updated] (SPARK-47488) [k8s]Driver stuck when thread pool is not shut down

2024-03-20 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47488: -- Component/s: Kubernetes (was: k8s) > [k8s]Driver stuck when thread pool

[jira] [Updated] (SPARK-47488) [k8s]Driver stuck when thread pool is not shut down

2024-03-20 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47488: -- Priority: Major (was: Critical) > [k8s]Driver stuck when thread pool is not shut down >

[jira] [Updated] (SPARK-47488) Driver stuck when thread pool is not shut down

2024-03-20 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47488: -- Summary: Driver stuck when thread pool is not shut down (was: [k8s]Driver stuck when thread

[jira] [Updated] (SPARK-47488) [k8s]Driver stuck when thread pool is not shut down

2024-03-20 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47488: -- Issue Type: Improvement (was: Bug) > [k8s]Driver stuck when thread pool is not shut down >

[jira] [Updated] (SPARK-47481) Fix Python linter

2024-03-20 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47481: -- Issue Type: Task (was: Improvement) > Fix Python linter > - > >

[jira] [Updated] (SPARK-47481) Fix Python linter

2024-03-20 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47481: -- Fix Version/s: 3.5.2 > Fix Python linter > - > > Key:

[jira] [Resolved] (SPARK-47481) Fix Python linter

2024-03-20 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47481. --- Fix Version/s: 3.4.3 Resolution: Fixed Issue resolved by pull request 45608

[jira] [Updated] (SPARK-47481) Fix Python linter

2024-03-20 Thread BingKun Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BingKun Pan updated SPARK-47481: Summary: Fix Python linter (was: pin `matplotlib<3.3.0`) > Fix Python linter > -

[jira] [Updated] (SPARK-47481) Fix Python linter

2024-03-20 Thread BingKun Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BingKun Pan updated SPARK-47481: Affects Version/s: 3.5.2 > Fix Python linter > - > > Key:

[jira] [Updated] (SPARK-47488) [k8s]Driver stuck when thread pool is not shut down

2024-03-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47488: --- Labels: pull-request-available (was: ) > [k8s]Driver stuck when thread pool is not shut

[jira] [Updated] (SPARK-47488) [k8s]Driver stuck when thread pool is not shut down

2024-03-20 Thread Zhou Tong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhou Tong updated SPARK-47488: -- Description: The app example:   {code:java} object SparkTest { def main(args: Array[String]): Unit

[jira] [Created] (SPARK-47488) [k8s]Driver stuck when thread pool is not shut down

2024-03-20 Thread Zhou Tong (Jira)
Zhou Tong created SPARK-47488: - Summary: [k8s]Driver stuck when thread pool is not shut down Key: SPARK-47488 URL: https://issues.apache.org/jira/browse/SPARK-47488 Project: Spark Issue Type:

[jira] [Commented] (SPARK-47358) StringRepeat (all collations)

2024-03-20 Thread Milan Dankovic (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17828917#comment-17828917 ] Milan Dankovic commented on SPARK-47358: I am working on this. > StringRepeat (all collations)

[jira] [Commented] (SPARK-47467) Error message regressed when creating hive table with illegal column name

2024-03-20 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17828915#comment-17828915 ] Kent Yao commented on SPARK-47467: -- {code:java} spark-sql (default)> select version(); 4.0.0

[jira] [Updated] (SPARK-47444) Validate alter table statements and check the numeric table stats

2024-03-20 Thread Miklos Szurap (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szurap updated SPARK-47444: -- Affects Version/s: 3.0.0 > Validate alter table statements and check the numeric table stats

[jira] [Updated] (SPARK-47487) simplify code in AnsiTypeCoercion

2024-03-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47487: --- Labels: pull-request-available (was: ) > simplify code in AnsiTypeCoercion >

[jira] [Resolved] (SPARK-47478) Improve test coverage for mysql bool synonyms

2024-03-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47478. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45604

[jira] [Updated] (SPARK-47444) Validate alter table statements and check the numeric table stats

2024-03-20 Thread Miklos Szurap (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szurap updated SPARK-47444: -- Summary: Validate alter table statements and check the numeric table stats (was: Empty

[jira] [Updated] (SPARK-47483) Add support for aggregation and join operations on arrays of collated strings

2024-03-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47483: --- Labels: pull-request-available (was: ) > Add support for aggregation and join operations

[jira] [Created] (SPARK-47486) Remove unused private method `getString` from `ArrowDeserializers`

2024-03-20 Thread Yang Jie (Jira)
Yang Jie created SPARK-47486: Summary: Remove unused private method `getString` from `ArrowDeserializers` Key: SPARK-47486 URL: https://issues.apache.org/jira/browse/SPARK-47486 Project: Spark

[jira] [Updated] (SPARK-47444) Empty numRows table stats should not break Hive tables

2024-03-20 Thread Miklos Szurap (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szurap updated SPARK-47444: -- Description: SPARK-30262 resolved/avoided the NumberFormatException in Spark when the

[jira] [Updated] (SPARK-47444) Empty numRows table stats should not break Hive tables

2024-03-20 Thread Miklos Szurap (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Szurap updated SPARK-47444: -- Attachment: (was: reproduction_steps_SPARK-47444.txt) > Empty numRows table stats should

[jira] [Updated] (SPARK-46840) Collation benchmarking

2024-03-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46840: --- Labels: pull-request-available (was: ) > Collation benchmarking > -- >

[jira] [Assigned] (SPARK-47478) Improve test coverage for mysql bool synonyms

2024-03-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-47478: Assignee: Kent Yao > Improve test coverage for mysql bool synonyms >

[jira] [Resolved] (SPARK-47480) Enable doctest for `createDataFrame`

2024-03-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47480. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45606

[jira] [Assigned] (SPARK-47481) pin `matplotlib<3.3.0`

2024-03-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-47481: -- Assignee: Apache Spark > pin `matplotlib<3.3.0` > -- > >

[jira] [Created] (SPARK-47487) simplify code in AnsiTypeCoercion

2024-03-20 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-47487: --- Summary: simplify code in AnsiTypeCoercion Key: SPARK-47487 URL: https://issues.apache.org/jira/browse/SPARK-47487 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-47486) Remove unused private method `getString` from `ArrowDeserializers`

2024-03-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47486: --- Labels: pull-request-available (was: ) > Remove unused private method `getString` from

[jira] [Resolved] (SPARK-41625) Feature parity: Streaming support

2024-03-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-41625. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45603

[jira] [Assigned] (SPARK-47481) pin `matplotlib<3.3.0`

2024-03-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-47481: -- Assignee: (was: Apache Spark) > pin `matplotlib<3.3.0` > --

[jira] [Assigned] (SPARK-47440) SQLServer does not support LIKE operator in binary comparison

2024-03-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-47440: -- Assignee: (was: Apache Spark) > SQLServer does not support LIKE operator in

[jira] [Assigned] (SPARK-47440) SQLServer does not support LIKE operator in binary comparison

2024-03-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-47440: -- Assignee: Apache Spark > SQLServer does not support LIKE operator in binary

[jira] [Created] (SPARK-47485) Create column with collations in dataframe API

2024-03-20 Thread Stefan Kandic (Jira)
Stefan Kandic created SPARK-47485: - Summary: Create column with collations in dataframe API Key: SPARK-47485 URL: https://issues.apache.org/jira/browse/SPARK-47485 Project: Spark Issue Type:

[jira] [Created] (SPARK-47484) Allow trailing comma in column definition list

2024-03-20 Thread Stefan Kandic (Jira)
Stefan Kandic created SPARK-47484: - Summary: Allow trailing comma in column definition list Key: SPARK-47484 URL: https://issues.apache.org/jira/browse/SPARK-47484 Project: Spark Issue Type:

[jira] [Updated] (SPARK-47467) Error message regressed when creating hive table with illegal column name

2024-03-20 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-47467: - Priority: Minor (was: Major) > Error message regressed when creating hive table with illegal column

[jira] [Updated] (SPARK-47483) Add support for aggregation and join operations on arrays of collated strings

2024-03-20 Thread Nikola Mandic (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikola Mandic updated SPARK-47483: -- Epic Link: SPARK-46830 > Add support for aggregation and join operations on arrays of

[jira] [Created] (SPARK-47483) Add support for aggregation and join operations on arrays of collated strings

2024-03-20 Thread Nikola Mandic (Jira)
Nikola Mandic created SPARK-47483: - Summary: Add support for aggregation and join operations on arrays of collated strings Key: SPARK-47483 URL: https://issues.apache.org/jira/browse/SPARK-47483

[jira] [Updated] (SPARK-47482) Add HiveDialect to sql module

2024-03-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47482: --- Labels: pull-request-available (was: ) > Add HiveDialect to sql module >

[jira] [Created] (SPARK-47482) Add HiveDialect to sql module

2024-03-20 Thread xleoken (Jira)
xleoken created SPARK-47482: --- Summary: Add HiveDialect to sql module Key: SPARK-47482 URL: https://issues.apache.org/jira/browse/SPARK-47482 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-47481) pin `matplotlib<3.3.0`

2024-03-20 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47481: --- Labels: pull-request-available (was: ) > pin `matplotlib<3.3.0` > -- >

[jira] [Created] (SPARK-47481) pin `matplotlib<3.3.0`

2024-03-20 Thread BingKun Pan (Jira)
BingKun Pan created SPARK-47481: --- Summary: pin `matplotlib<3.3.0` Key: SPARK-47481 URL: https://issues.apache.org/jira/browse/SPARK-47481 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-47473) Correctness issue of converting postgres INFINITIES timestamps

2024-03-20 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-47473. -- Fix Version/s: 3.5.2 4.0.0 Resolution: Fixed Issue resolved by pull request

  1   2   >