[jira] [Updated] (SPARK-46718) Upgrade Arrow to 15.0.0

[ https://issues.apache.org/jira/browse/SPARK-46718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-46718: -- Parent: SPARK-44111 Issue Type: Sub-task (was: Improvement) > Upgrade Ar

[jira] [Updated] (SPARK-47529) Use hadoop 3.4.0 in some docs

[ https://issues.apache.org/jira/browse/SPARK-47529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47529: -- Parent: SPARK-44111 Issue Type: Sub-task (was: Improvement) > Use hadoop 3.

[jira] [Assigned] (SPARK-47529) Use hadoop 3.4.0 in some docs

[ https://issues.apache.org/jira/browse/SPARK-47529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47529: - Assignee: BingKun Pan > Use hadoop 3.4.0 in some d

[jira] [Resolved] (SPARK-47529) Use hadoop 3.4.0 in some docs

[ https://issues.apache.org/jira/browse/SPARK-47529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47529. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45679 [https

[jira] [Assigned] (SPARK-47495) Primary resource jar added to spark.jars twice under k8s cluster mode

[ https://issues.apache.org/jira/browse/SPARK-47495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47495: - Assignee: Jiale Tan > Primary resource jar added to spark.jars twice under k8s clus

[jira] [Resolved] (SPARK-47495) Primary resource jar added to spark.jars twice under k8s cluster mode

[ https://issues.apache.org/jira/browse/SPARK-47495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47495. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45607 [https

[jira] [Assigned] (SPARK-47510) Fix `DSTREAM` label pattern in `labeler.yml`

[ https://issues.apache.org/jira/browse/SPARK-47510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47510: - Assignee: Dongjoon Hyun > Fix `DSTREAM` label pattern in `labeler.

[jira] [Resolved] (SPARK-47510) Fix `DSTREAM` label pattern in `labeler.yml`

[ https://issues.apache.org/jira/browse/SPARK-47510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47510. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45648 [https

[jira] [Resolved] (SPARK-47522) Read MySQL FLOAT as FloatType to keep consistent with the write side

[ https://issues.apache.org/jira/browse/SPARK-47522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47522. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45666 [https

[jira] [Assigned] (SPARK-47522) Read MySQL FLOAT as FloatType to keep consistent with the write side

[ https://issues.apache.org/jira/browse/SPARK-47522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47522: - Assignee: Kent Yao > Read MySQL FLOAT as FloatType to keep consistent with the wr

[jira] [Commented] (SPARK-47521) Use `Utils.tryWithResource` during reading shuffle data from external storage

[ https://issues.apache.org/jira/browse/SPARK-47521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17829949#comment-17829949 ] Dongjoon Hyun commented on SPARK-47521: --- FYI, [~maheshk114], `Target Version` field is reserved

[jira] [Updated] (SPARK-47521) Use `Utils.tryWithResource` during reading shuffle data from external storage

[ https://issues.apache.org/jira/browse/SPARK-47521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47521: -- Target Version/s: (was: 4.0.0) > Use `Utils.tryWithResource` during reading shuffle d

[jira] [Updated] (SPARK-47521) Use `Utils.tryWithResource` during reading shuffle data from external storage

[ https://issues.apache.org/jira/browse/SPARK-47521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47521: -- Summary: Use `Utils.tryWithResource` during reading shuffle data from external storage

[jira] [Updated] (SPARK-47521) Fix file handle leakage during shuffle data read from external storage.

[ https://issues.apache.org/jira/browse/SPARK-47521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47521: -- Issue Type: Bug (was: Improvement) > Fix file handle leakage during shuffle data read f

[jira] [Resolved] (SPARK-47521) Fix file handle leakage during shuffle data read from external storage.

[ https://issues.apache.org/jira/browse/SPARK-47521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47521. --- Fix Version/s: 3.4.3 3.5.2 4.0.0 Resolution

[jira] [Assigned] (SPARK-47521) Fix file handle leakage during shuffle data read from external storage.

[ https://issues.apache.org/jira/browse/SPARK-47521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47521: - Assignee: mahesh kumar behera > Fix file handle leakage during shuffle data read f

[jira] [Resolved] (SPARK-47523) Replace Deprecated `JsonParser#getCurrentName` with `JsonParser#currentName`

[ https://issues.apache.org/jira/browse/SPARK-47523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47523. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45668 [https

[jira] [Assigned] (SPARK-47523) Replace Deprecated `JsonParser#getCurrentName` with `JsonParser#currentName`

[ https://issues.apache.org/jira/browse/SPARK-47523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47523: - Assignee: Yang Jie > Replace Deprecated `JsonParser#getCurrentName` with `JsonPar

[jira] [Resolved] (ORC-1663) [C++] Enable TestTimezone.testMissingTZDB on Windows

[ https://issues.apache.org/jira/browse/ORC-1663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved ORC-1663. Fix Version/s: 2.1.0 Resolution: Fixed Issue resolved by pull request 1856 [https

[jira] [Assigned] (SPARK-47499) Reuse `test_help_command` in Connect

[ https://issues.apache.org/jira/browse/SPARK-47499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47499: - Assignee: Ruifeng Zheng > Reuse `test_help_command` in Conn

[jira] [Resolved] (SPARK-47499) Reuse `test_help_command` in Connect

[ https://issues.apache.org/jira/browse/SPARK-47499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47499. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45634 [https

[jira] [Updated] (SPARK-47513) Regenerate benchmark results

[ https://issues.apache.org/jira/browse/SPARK-47513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47513: -- Parent: SPARK-44111 Issue Type: Sub-task (was: Task) > Regenerate benchmark resu

[jira] [Resolved] (SPARK-47514) Add a test coverage for createTable method (partitioned-table) in CatalogSuite

[ https://issues.apache.org/jira/browse/SPARK-47514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47514. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45637 [https

[jira] [Assigned] (SPARK-47514) Add a test coverage for createTable method (partitioned-table) in CatalogSuite

[ https://issues.apache.org/jira/browse/SPARK-47514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47514: - Assignee: BingKun Pan > Add a test coverage for createTable method (partitioned-ta

[jira] [Resolved] (SPARK-47513) Regenerate benchmark results

[ https://issues.apache.org/jira/browse/SPARK-47513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47513. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45654 [https

[jira] [Assigned] (SPARK-47513) Regenerate benchmark results

[ https://issues.apache.org/jira/browse/SPARK-47513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47513: - Assignee: Dongjoon Hyun > Regenerate benchmark resu

[jira] [Created] (SPARK-47513) Regenerate benchmark results

Dongjoon Hyun created SPARK-47513: - Summary: Regenerate benchmark results Key: SPARK-47513 URL: https://issues.apache.org/jira/browse/SPARK-47513 Project: Spark Issue Type: Task

[jira] [Resolved] (SPARK-47502) Make the `size` of the installed packages output in `descending` order, and add a `header`

[ https://issues.apache.org/jira/browse/SPARK-47502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47502. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45630 [https

[jira] [Assigned] (SPARK-47502) Make the `size` of the installed packages output in `descending` order, and add a `header`

[ https://issues.apache.org/jira/browse/SPARK-47502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47502: - Assignee: BingKun Pan > Make the `size` of the installed packages out

[jira] [Resolved] (SPARK-47507) Upgrade ORC to 1.9.3

[ https://issues.apache.org/jira/browse/SPARK-47507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47507. --- Fix Version/s: 3.5.2 Resolution: Fixed Issue resolved by pull request 45646 [https

[jira] [Created] (SPARK-47510) Fix `DSTREAM` label pattern in `labeler.yml`

Dongjoon Hyun created SPARK-47510: - Summary: Fix `DSTREAM` label pattern in `labeler.yml` Key: SPARK-47510 URL: https://issues.apache.org/jira/browse/SPARK-47510 Project: Spark Issue Type

[jira] [Assigned] (SPARK-47505) Fix `pyspark-errors` test jobs for branch-3.4

[ https://issues.apache.org/jira/browse/SPARK-47505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47505: - Assignee: BingKun Pan > Fix `pyspark-errors` test jobs for branch-

[jira] [Resolved] (SPARK-47505) Fix `pyspark-errors` test jobs for branch-3.4

[ https://issues.apache.org/jira/browse/SPARK-47505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47505. --- Fix Version/s: 3.4.3 Resolution: Fixed Issue resolved by pull request 45624 [https

[jira] [Commented] (SPARK-47046) Apache Spark 4.0.0 Dependency Audit and Cleanup

[ https://issues.apache.org/jira/browse/SPARK-47046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17829639#comment-17829639 ] Dongjoon Hyun commented on SPARK-47046: --- All known issues except SPARK-47018 are resolved

[jira] [Resolved] (SPARK-47046) Apache Spark 4.0.0 Dependency Audit and Cleanup

[ https://issues.apache.org/jira/browse/SPARK-47046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47046. --- Fix Version/s: 4.0.0 Resolution: Fixed > Apache Spark 4.0.0 Dependency Au

[jira] [Assigned] (SPARK-47046) Apache Spark 4.0.0 Dependency Audit and Cleanup

[ https://issues.apache.org/jira/browse/SPARK-47046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47046: - Assignee: Dongjoon Hyun > Apache Spark 4.0.0 Dependency Audit and Clea

Re: [ANNOUNCE] Announcing Apache ORC 1.9.3

Thank you! Dongjoon. On Wed, Mar 20, 2024 at 10:15 PM Gang Wu wrote: > Hi All. > > We are happy to announce the availability of Apache ORC 1.9.3! > > https://orc.apache.org/news/2024/03/20/ORC-1.9.3/ > > 1.9.3 is a maintenance release containing important fixes. > It's available in Apache

Re: [ANNOUNCE] Announcing Apache ORC 1.9.3

Thank you! Dongjoon. On Wed, Mar 20, 2024 at 10:15 PM Gang Wu wrote: > Hi All. > > We are happy to announce the availability of Apache ORC 1.9.3! > > https://orc.apache.org/news/2024/03/20/ORC-1.9.3/ > > 1.9.3 is a maintenance release containing important fixes. > It's available in Apache

[jira] [Assigned] (SPARK-47487) simplify code in AnsiTypeCoercion

[ https://issues.apache.org/jira/browse/SPARK-47487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47487: - Assignee: Wenchen Fan > simplify code in AnsiTypeCoerc

[jira] [Resolved] (SPARK-47487) simplify code in AnsiTypeCoercion

[ https://issues.apache.org/jira/browse/SPARK-47487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47487. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45612 [https

[jira] [Resolved] (SPARK-47501) Add convertDateToDate like the existing convertTimestampToTimestamp for JdbcDialects

[ https://issues.apache.org/jira/browse/SPARK-47501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47501. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45638 [https

[jira] [Assigned] (SPARK-47501) Add convertDateToDate like the existing convertTimestampToTimestamp for JdbcDialects

[ https://issues.apache.org/jira/browse/SPARK-47501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47501: - Assignee: Kent Yao > Add convertDateToDate like the exist

[jira] [Resolved] (SPARK-41888) Support StreamingQueryListener for DataFrame.observe

[ https://issues.apache.org/jira/browse/SPARK-41888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-41888. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45627 [https

[jira] [Assigned] (SPARK-41888) Support StreamingQueryListener for DataFrame.observe

[ https://issues.apache.org/jira/browse/SPARK-41888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-41888: - Assignee: Jiaan Geng > Support StreamingQueryListener for DataFrame.obse

[jira] [Updated] (SPARK-47008) Spark to support S3 Express One Zone Storage

[ https://issues.apache.org/jira/browse/SPARK-47008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47008: -- Affects Version/s: 4.0.0 (was: 3.5.1) > Spark to support

[jira] [Updated] (SPARK-47008) Spark to support S3 Express One Zone Storage

[ https://issues.apache.org/jira/browse/SPARK-47008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47008: -- Parent Issue: SPARK-44111 (was: SPARK-44124) > Spark to support S3 Express One Zone Stor

[jira] [Closed] (SPARK-46793) Revert S3A endpoint fixup logic of SPARK-35878

[ https://issues.apache.org/jira/browse/SPARK-46793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-46793. - > Revert S3A endpoint fixup logic of SPARK-35

[jira] [Resolved] (SPARK-46793) Revert S3A endpoint fixup logic of SPARK-35878

[ https://issues.apache.org/jira/browse/SPARK-46793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-46793. --- Resolution: Duplicate > Revert S3A endpoint fixup logic of SPARK-35

[jira] [Resolved] (SPARK-45721) Upgrade AWS SDK to v2 for Hadoop dependency

[ https://issues.apache.org/jira/browse/SPARK-45721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-45721. --- Resolution: Duplicate > Upgrade AWS SDK to v2 for Hadoop depende

[jira] [Closed] (SPARK-45721) Upgrade AWS SDK to v2 for Hadoop dependency

[ https://issues.apache.org/jira/browse/SPARK-45721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-45721. - > Upgrade AWS SDK to v2 for Hadoop depende

[jira] [Assigned] (SPARK-47491) Re-enable `driver log links` test in YarnClusterSuite

[ https://issues.apache.org/jira/browse/SPARK-47491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47491: - Assignee: Dongjoon Hyun > Re-enable `driver log links` test in YarnClusterSu

[jira] [Updated] (SPARK-47054) Remove pinned version of torch for Python 3.12 support

[ https://issues.apache.org/jira/browse/SPARK-47054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47054: -- Parent: SPARK-44111 Issue Type: Sub-task (was: Bug) > Remove pinned version of to

[jira] [Resolved] (SPARK-47494) Add migration doc for the behavior change of Parquet timestamp inference since Spark 3.3

[ https://issues.apache.org/jira/browse/SPARK-47494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47494. --- Fix Version/s: 3.4.3 3.5.2 4.0.0 Resolution

[jira] [Resolved] (SPARK-47490) Fix RocksDB Logger deprecation warning

[ https://issues.apache.org/jira/browse/SPARK-47490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47490. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45616 [https

[jira] [Assigned] (SPARK-47490) Fix RocksDB Logger deprecation warning

[ https://issues.apache.org/jira/browse/SPARK-47490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47490: - Assignee: Anish Shrigondekar > Fix RocksDB Logger deprecation warn

[jira] [Updated] (SPARK-47486) Remove unused private `ArrowDeserializers.getString` method

[ https://issues.apache.org/jira/browse/SPARK-47486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47486: -- Summary: Remove unused private `ArrowDeserializers.getString` method (was: Remove unused

[jira] [Resolved] (SPARK-47486) Remove unused private method `getString` from `ArrowDeserializers`

[ https://issues.apache.org/jira/browse/SPARK-47486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47486. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45610 [https

[jira] [Updated] (SPARK-47490) Fix RocksDB Logger deprecation warning

[ https://issues.apache.org/jira/browse/SPARK-47490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47490: -- Issue Type: Improvement (was: Task) > Fix RocksDB Logger deprecation warn

Re: [ANNOUNCE] Apache Hadoop 3.4.0 release

Thank you all! Apache ORC and Spark starts to use Apache Hadoop 3.4.0 successfully. https://github.com/apache/orc/commit/d2f9d8d97a99bffb9e9adfa5a53c8419e5540f44 https://github.com/apache/spark/commit/49b4c3bc9c09325de941dfaf41e4fd3a4a4c345f Thanks, Dongjoon. On 2024/03/19 13:07:37 slfan1989

[jira] [Resolved] (SPARK-45393) Upgrade Hadoop to 3.4.0

[ https://issues.apache.org/jira/browse/SPARK-45393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-45393. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45583 [https

[jira] [Created] (SPARK-47491) Re-enable `driver log links` test in YarnClusterSuite

Dongjoon Hyun created SPARK-47491: - Summary: Re-enable `driver log links` test in YarnClusterSuite Key: SPARK-47491 URL: https://issues.apache.org/jira/browse/SPARK-47491 Project: Spark

[jira] [Resolved] (SPARK-47462) Align the mapping ways of other unsigned numeric types with TINYINT UNSGINED in MySQLDialect

[ https://issues.apache.org/jira/browse/SPARK-47462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47462. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45588 [https

[jira] [Updated] (SPARK-47488) [k8s]Driver stuck when thread pool is not shut down

[ https://issues.apache.org/jira/browse/SPARK-47488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47488: -- Affects Version/s: 4.0.0 (was: 3.2.0) > [k8s]Driver stuck w

[jira] [Updated] (SPARK-47488) [k8s]Driver stuck when thread pool is not shut down

[ https://issues.apache.org/jira/browse/SPARK-47488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47488: -- Component/s: Kubernetes (was: k8s) > [k8s]Driver stuck when thread p

[jira] [Updated] (SPARK-47488) [k8s]Driver stuck when thread pool is not shut down

[ https://issues.apache.org/jira/browse/SPARK-47488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47488: -- Priority: Major (was: Critical) > [k8s]Driver stuck when thread pool is not shut d

[jira] [Updated] (SPARK-47488) Driver stuck when thread pool is not shut down

[ https://issues.apache.org/jira/browse/SPARK-47488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47488: -- Summary: Driver stuck when thread pool is not shut down (was: [k8s]Driver stuck when thread

[jira] [Updated] (SPARK-47488) [k8s]Driver stuck when thread pool is not shut down

[ https://issues.apache.org/jira/browse/SPARK-47488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47488: -- Issue Type: Improvement (was: Bug) > [k8s]Driver stuck when thread pool is not shut d

[jira] [Updated] (SPARK-47481) Fix Python linter

[ https://issues.apache.org/jira/browse/SPARK-47481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47481: -- Issue Type: Task (was: Improvement) > Fix Python lin

[jira] [Updated] (SPARK-47481) Fix Python linter

[ https://issues.apache.org/jira/browse/SPARK-47481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47481: -- Fix Version/s: 3.5.2 > Fix Python linter > - > >

[jira] [Resolved] (SPARK-47481) Fix Python linter

[ https://issues.apache.org/jira/browse/SPARK-47481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47481. --- Fix Version/s: 3.4.3 Resolution: Fixed Issue resolved by pull request 45608 [https

[jira] [Assigned] (SPARK-47472) Pin `numpy` to 1.23.5 in `dev/infra/Dockerfile`

[ https://issues.apache.org/jira/browse/SPARK-47472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47472: - Assignee: Dongjoon Hyun > Pin `numpy` to 1.23.5 in `dev/infra/Dockerf

[jira] [Resolved] (SPARK-47472) Pin `numpy` to 1.23.5 in `dev/infra/Dockerfile`

[ https://issues.apache.org/jira/browse/SPARK-47472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47472. --- Fix Version/s: 3.4.3 Resolution: Fixed Issue resolved by pull request 45595 [https

[jira] [Created] (SPARK-47472) Pin `numpy` to 1.23.5 in `dev/infra/Dockerfile`

Dongjoon Hyun created SPARK-47472: - Summary: Pin `numpy` to 1.23.5 in `dev/infra/Dockerfile` Key: SPARK-47472 URL: https://issues.apache.org/jira/browse/SPARK-47472 Project: Spark Issue Type

[jira] [Resolved] (SPARK-47470) Ignore `IntentionallyFaultyConnectionProvider` error in `CliSuite`

[ https://issues.apache.org/jira/browse/SPARK-47470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47470. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45597 [https

[jira] [Assigned] (SPARK-47470) Ignore `IntentionallyFaultyConnectionProvider` error in `CliSuite`

[ https://issues.apache.org/jira/browse/SPARK-47470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47470: - Assignee: Dongjoon Hyun > Ignore `IntentionallyFaultyConnectionProvider` er

[jira] [Updated] (SPARK-47470) Ignore `IntentionallyFaultyConnectionProvider` error in `CliSuite`

[ https://issues.apache.org/jira/browse/SPARK-47470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47470: -- Summary: Ignore `IntentionallyFaultyConnectionProvider` error in `CliSuite` (was: Make

[jira] [Created] (SPARK-47470) Make `CliSuite` ignores `IntentionallyFaultyConnectionProvider` error

Dongjoon Hyun created SPARK-47470: - Summary: Make `CliSuite` ignores `IntentionallyFaultyConnectionProvider` error Key: SPARK-47470 URL: https://issues.apache.org/jira/browse/SPARK-47470 Project

[jira] [Comment Edited] (SPARK-44101) Support pandas 2

[ https://issues.apache.org/jira/browse/SPARK-44101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17828556#comment-17828556 ] Dongjoon Hyun edited comment on SPARK-44101 at 3/20/24 1:00 AM: I added

[jira] [Assigned] (SPARK-44101) Support pandas 2

[ https://issues.apache.org/jira/browse/SPARK-44101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-44101: - Assignee: Haejoon Lee > Support pandas 2 > > >

[jira] [Updated] (SPARK-44101) Support pandas 2

[ https://issues.apache.org/jira/browse/SPARK-44101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-44101: -- Labels: releasenotes (was: ) > Support pandas 2 > > >

[jira] [Commented] (SPARK-44101) Support pandas 2

[ https://issues.apache.org/jira/browse/SPARK-44101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17828556#comment-17828556 ] Dongjoon Hyun commented on SPARK-44101: --- I added `releasenotes` label to this JIRA. > Supp

[jira] [Closed] (SPARK-44149) Support DataFrame Merge API

[ https://issues.apache.org/jira/browse/SPARK-44149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-44149. - > Support DataFrame Merge API > --- > > Key:

[jira] [Updated] (SPARK-46207) Support MergeInto in DataFrameWriterV2

[ https://issues.apache.org/jira/browse/SPARK-46207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-46207: -- Parent: SPARK-44111 Issue Type: Sub-task (was: New Feature) > Support MergeI

[jira] [Resolved] (SPARK-47468) Exclude `logback` dependency from SBT like Maven

[ https://issues.apache.org/jira/browse/SPARK-47468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47468. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45594 [https

[jira] [Updated] (SPARK-47432) Add `pyarrow` upper bound requirement, `<13.0.0`

[ https://issues.apache.org/jira/browse/SPARK-47432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47432: -- Fix Version/s: 3.4.3 > Add `pyarrow` upper bound requirement, `<

[jira] [Resolved] (SPARK-47290) Extend CustomTaskMetric to allow metric values from multiple sources

[ https://issues.apache.org/jira/browse/SPARK-47290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47290. --- Resolution: Invalid > Extend CustomTaskMetric to allow metric values from multiple sour

[jira] [Closed] (SPARK-47290) Extend CustomTaskMetric to allow metric values from multiple sources

[ https://issues.apache.org/jira/browse/SPARK-47290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-47290. - Sure. I closed this. > Extend CustomTaskMetric to allow metric values from multiple sour

[jira] [Resolved] (SPARK-47464) Update labeler.yml

[ https://issues.apache.org/jira/browse/SPARK-47464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47464. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45590 [https

[jira] [Updated] (SPARK-47464) Update labeler.yml for module common/sketch and common/variant

[ https://issues.apache.org/jira/browse/SPARK-47464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47464: -- Summary: Update labeler.yml for module common/sketch and common/variant (was: Update

[jira] [Updated] (SPARK-47468) Exclude `logback` dependency from SBT like Maven

[ https://issues.apache.org/jira/browse/SPARK-47468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47468: -- Summary: Exclude `logback` dependency from SBT like Maven (was: Exclude `logback` from SBT

[jira] [Assigned] (SPARK-47468) Exclude `logback` from SBT dependency like Maven

[ https://issues.apache.org/jira/browse/SPARK-47468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47468: - Assignee: Dongjoon Hyun > Exclude `logback` from SBT dependency like Ma

[jira] [Created] (SPARK-47468) Exclude `logback` from SBT dependency like Maven

Dongjoon Hyun created SPARK-47468: - Summary: Exclude `logback` from SBT dependency like Maven Key: SPARK-47468 URL: https://issues.apache.org/jira/browse/SPARK-47468 Project: Spark Issue

[jira] [Resolved] (SPARK-47461) Remove unused private function `totalRunningTasksPerResourceProfile` from `ExecutorAllocationManager`

[ https://issues.apache.org/jira/browse/SPARK-47461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47461. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45587 [https

[jira] [Assigned] (SPARK-47461) Remove unused private function `totalRunningTasksPerResourceProfile` from `ExecutorAllocationManager`

[ https://issues.apache.org/jira/browse/SPARK-47461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47461: - Assignee: Yang Jie > Remove unused private funct

[jira] [Resolved] (ORC-1608) Upgrade Hadoop to 3.4.0

[ https://issues.apache.org/jira/browse/ORC-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved ORC-1608. Fix Version/s: 2.1.0 Resolution: Fixed Issue resolved by pull request 1783 [https

[jira] [Assigned] (ORC-1608) Upgrade Hadoop to 3.4.0

[ https://issues.apache.org/jira/browse/ORC-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned ORC-1608: -- Assignee: Dongjoon Hyun > Upgrade Hadoop to 3.

[jira] [Resolved] (SPARK-47457) Fix `IsolatedClientLoader.supportsHadoopShadedClient` to handle Hadoop 3.4+

[ https://issues.apache.org/jira/browse/SPARK-47457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47457. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45585 [https

[jira] [Updated] (SPARK-47457) Fix `IsolatedClientLoader.supportsHadoopShadedClient` to handle Hadoop 3.4+

[ https://issues.apache.org/jira/browse/SPARK-47457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47457: -- Summary: Fix `IsolatedClientLoader.supportsHadoopShadedClient` to handle Hadoop 3.4

[jira] [Assigned] (SPARK-47457) Fix `IsolatedClientLoader.supportsHadoopShadedClient` to handle Hadoop 3.4

[ https://issues.apache.org/jira/browse/SPARK-47457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47457: - Assignee: Dongjoon Hyun > Fix `IsolatedClientLoader.supportsHadoopShadedCli

[jira] [Updated] (SPARK-47457) Fix `IsolatedClientLoader.supportsHadoopShadedClient` to handle Hadoop 3.4

[ https://issues.apache.org/jira/browse/SPARK-47457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47457: -- Component/s: SQL (was: Spark Core) >

<    4   5   6   7   8   9   10   11   12   13   >