[jira] [Created] (SPARK-47676) Clean up the removed `VersionsSuite` references

2024-04-01 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-47676: - Summary: Clean up the removed `VersionsSuite` references Key: SPARK-47676 URL: https://issues.apache.org/jira/browse/SPARK-47676 Project: Spark Issue Type

[jira] [Resolved] (SPARK-47675) Use AWS SDK v2 2.23.19 in K8s IT

2024-04-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47675. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45799 [https

[jira] [Updated] (SPARK-47556) [K8] Spark App ID collision resulting in deleting wrong resources

2024-04-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47556: -- Fix Version/s: (was: SPARK-36014) > [K8] Spark App ID collision resulting in delet

[jira] [Closed] (SPARK-47556) [K8] Spark App ID collision resulting in deleting wrong resources

2024-04-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-47556. - > [K8] Spark App ID collision resulting in deleting wrong resour

[jira] [Resolved] (SPARK-47556) [K8] Spark App ID collision resulting in deleting wrong resources

2024-04-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47556. --- Fix Version/s: SPARK-36014 Resolution: Duplicate > [K8] Spark App ID collis

[jira] [Updated] (SPARK-47556) [K8] Spark App ID collision resulting in deleting wrong resources

2024-04-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47556: -- Fix Version/s: (was: 3.3.0) > [K8] Spark App ID collision resulting in deleting wr

[jira] [Reopened] (SPARK-47556) [K8] Spark App ID collision resulting in deleting wrong resources

2024-04-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reopened SPARK-47556: --- > [K8] Spark App ID collision resulting in deleting wrong resour

[jira] [Updated] (SPARK-47675) Use AWS SDK v2 2.23.19 in K8s IT

2024-04-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47675: -- Summary: Use AWS SDK v2 2.23.19 in K8s IT (was: Remove `aws.java.sdk.v2.version` property

[jira] [Updated] (SPARK-47675) Remove `aws.java.sdk.v2.version` property in pom.xml

2024-04-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47675: -- Component/s: Kubernetes Tests > Remove `aws.java.sdk.v2.version` prope

[jira] [Assigned] (SPARK-47675) Remove `aws.java.sdk.v2.version` property in pom.xml

2024-04-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47675: - Assignee: Dongjoon Hyun > Remove `aws.java.sdk.v2.version` property in pom.

[jira] [Created] (SPARK-47675) Remove `aws.java.sdk.v2.version` property in pom.xml

2024-04-01 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-47675: - Summary: Remove `aws.java.sdk.v2.version` property in pom.xml Key: SPARK-47675 URL: https://issues.apache.org/jira/browse/SPARK-47675 Project: Spark Issue

[jira] [Resolved] (SPARK-47674) Enable `spark.metrics.appStatusSource.enabled` by default

2024-04-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47674. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45798 [https

[jira] [Assigned] (SPARK-47674) Enable `spark.metrics.appStatusSource.enabled` by default

2024-04-01 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47674: - Assignee: Dongjoon Hyun > Enable `spark.metrics.appStatusSource.enabled` by defa

[jira] [Created] (SPARK-47674) Enable `spark.metrics.appStatusSource.enabled` by default

2024-04-01 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-47674: - Summary: Enable `spark.metrics.appStatusSource.enabled` by default Key: SPARK-47674 URL: https://issues.apache.org/jira/browse/SPARK-47674 Project: Spark

Re: [DRAFT] [REPORT] ORC - April 2024

2024-04-01 Thread Dongjoon Hyun
Thank you always, William. Looks good to me. I have only two minor comments about typos 1. - According to our release cadence, we released three maintenance + According to our release cadence, we released two maintenance 2. - the traffic of both dev, issues, and user mailing lists + the traffic

[jira] [Assigned] (ORC-1672) Modify the package name of TestCheckTool

2024-03-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/ORC-1672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned ORC-1672: -- Assignee: dzcxzl > Modify the package name of TestCheckT

[jira] [Resolved] (ORC-1672) Modify the package name of TestCheckTool

2024-03-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/ORC-1672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved ORC-1672. Fix Version/s: 2.0.1 2.1.0 Resolution: Fixed Issue resolved by pull

[jira] [Updated] (SPARK-47645) Make Spark build with -release instead of -target

2024-03-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47645: -- Parent: SPARK-44111 Issue Type: Sub-task (was: Improvement) > Make Spark bu

[jira] [Assigned] (SPARK-47645) Make Spark build with -release instead of -target

2024-03-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47645: - Assignee: Yang Jie > Make Spark build with -release instead of -tar

[jira] [Resolved] (SPARK-47645) Make Spark build with -release instead of -target

2024-03-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47645. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45716 [https

[jira] [Resolved] (SPARK-47662) Document Mapping Spark SQL Data Types to MySQL

2024-03-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47662. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45787 [https

[jira] [Assigned] (SPARK-47662) Document Mapping Spark SQL Data Types to MySQL

2024-03-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47662: - Assignee: Kent Yao > Document Mapping Spark SQL Data Types to My

Re: [VOTE] SPIP: Pure Python Package in PyPI (Spark Connect)

2024-03-31 Thread Dongjoon Hyun
+1 Thank you, Hyukjin. Dongjoon On Sun, Mar 31, 2024 at 19:07 Haejoon Lee wrote: > +1 > > On Mon, Apr 1, 2024 at 10:15 AM Hyukjin Kwon wrote: > >> Hi all, >> >> I'd like to start the vote for SPIP: Pure Python Package in PyPI (Spark >> Connect) >> >> JIRA

[jira] [Resolved] (ORC-1673) Remove test packages `o.a.o.tools.[count|merge|sizes]`

2024-03-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/ORC-1673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved ORC-1673. Fix Version/s: 2.0.1 2.1.0 Resolution: Fixed Issue resolved by pull

[jira] [Resolved] (SPARK-47658) Upgrade `tink` to 1.12.0

2024-03-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47658. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45783 [https

[jira] [Resolved] (SPARK-47656) Upgrade commons-io to 2.16.0

2024-03-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47656. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45781 [https

[jira] [Assigned] (ORC-1673) Remove test packages `o.a.o.tools.[count|merge|sizes]`

2024-03-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/ORC-1673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned ORC-1673: -- Assignee: Dongjoon Hyun > Remove test packages `o.a.o.tools.[count|merge|si

[jira] [Created] (ORC-1673) Remove test packages `org/apache/orc/tools/[count|merge|sizes]`

2024-03-31 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created ORC-1673: -- Summary: Remove test packages `org/apache/orc/tools/[count|merge|sizes]` Key: ORC-1673 URL: https://issues.apache.org/jira/browse/ORC-1673 Project: ORC Issue

[jira] [Updated] (ORC-1673) Remove test packages `o.a.o.tools.[count|merge|sizes]`

2024-03-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/ORC-1673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated ORC-1673: --- Summary: Remove test packages `o.a.o.tools.[count|merge|sizes]` (was: Remove test packages `org

Re: Regarding the Apache Committee Report Helper Website

2024-03-31 Thread Dongjoon Hyun
Ya, it seems to happen at multiple projects. Like Apache ORC community, Apache Spark community report page is also missing JIRA/Commit/GitHub PR activities like the following. - https://reporter.apache.org/wizard/statistics?spark JIRA activity: 0 issues opened in JIRA, past quarter (-100%

[jira] [Updated] (ORC-1667) Add `check` tool to check the index of the specified column

2024-03-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/ORC-1667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated ORC-1667: --- Summary: Add `check` tool to check the index of the specified column (was: Add check tool to check

[jira] [Assigned] (ORC-1667) Add `bloom-filter` tool to check the bloom filter of the specified column

2024-03-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/ORC-1667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned ORC-1667: -- Assignee: dzcxzl > Add `bloom-filter` tool to check the bloom filter of the specified col

[jira] [Updated] (ORC-1667) Add check tool to check the index of the specified column

2024-03-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/ORC-1667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated ORC-1667: --- Summary: Add check tool to check the index of the specified column (was: Add `bloom-filter` tool

[jira] [Resolved] (ORC-1667) Add `bloom-filter` tool to check the bloom filter of the specified column

2024-03-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/ORC-1667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved ORC-1667. Fix Version/s: 2.0.1 2.1.0 Resolution: Fixed Issue resolved by pull

[jira] [Resolved] (SPARK-47648) Use `checkError()` to check Exception in `[CSV|Json|Xml]Suite` and `[Csv|Json]FunctionsSuite`

2024-03-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47648. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45774 [https

[jira] [Assigned] (SPARK-47648) Use `checkError()` to check Exception in `[CSV|Json|Xml]Suite` and `[Csv|Json]FunctionsSuite`

2024-03-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47648: - Assignee: BingKun Pan > Use `checkError()` to check Exception in `[CSV|Json|Xml]Su

[jira] [Assigned] (SPARK-47647) Make MySQL data source be able to read bit(n>1) as BinaryType like Postgres

2024-03-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47647: - Assignee: Kent Yao > Make MySQL data source be able to read bit(n>1) as Bina

[jira] [Resolved] (SPARK-47647) Make MySQL data source be able to read bit(n>1) as BinaryType like Postgres

2024-03-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47647. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45773 [https

[jira] [Resolved] (SPARK-47642) Exclude `org.junit.jupiter` and `org.junit.platform` from `jmock-junit5`

2024-03-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47642. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45767 [https

[jira] [Resolved] (SPARK-47636) Use Java 17 instead of 17-jre image in K8s Dockerfile

2024-03-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47636. --- Fix Version/s: 3.5.2 Resolution: Fixed Issue resolved by pull request 45762 [https

[jira] [Resolved] (SPARK-47635) Use Java 21 instead of 21-jre in K8s Dockerfile

2024-03-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47635. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45761 [https

[jira] [Created] (SPARK-47636) Use Java 17 instead of 17-jre image in K8s Dockerfile

2024-03-28 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-47636: - Summary: Use Java 17 instead of 17-jre image in K8s Dockerfile Key: SPARK-47636 URL: https://issues.apache.org/jira/browse/SPARK-47636 Project: Spark

[jira] [Updated] (SPARK-47635) Use Java 21 instead of 21-jre in K8s Dockerfile

2024-03-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47635: -- Affects Version/s: (was: 3.5.0) (was: 3.5.1) > Use Java

[jira] [Assigned] (SPARK-47635) Use Java 21 instead of 21-jre in K8s Dockerfile

2024-03-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47635: - Assignee: Dongjoon Hyun > Use Java 21 instead of 21-jre in K8s Dockerf

[jira] [Updated] (SPARK-47635) Use Java 21 instead of 21-jre in K8s Dockerfile

2024-03-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47635: -- Affects Version/s: 3.5.1 3.5.0 > Use Java 21 instead of 21-jre in

[jira] [Created] (SPARK-47635) Use Java 21 instead of 21-jre in K8s Dockerfile

2024-03-28 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-47635: - Summary: Use Java 21 instead of 21-jre in K8s Dockerfile Key: SPARK-47635 URL: https://issues.apache.org/jira/browse/SPARK-47635 Project: Spark Issue Type

[jira] [Resolved] (SPARK-47632) Ban `com.amazonaws:aws-java-sdk-bundle` dependency

2024-03-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47632. --- Fix Version/s: 4.0.0 Resolution: Fixed This is resolved via https://github.com

[jira] [Assigned] (SPARK-47632) Ban `com.amazonaws:aws-java-sdk-bundle` dependency

2024-03-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47632: - Assignee: Dongjoon Hyun > Ban `com.amazonaws:aws-java-sdk-bundle` depende

[jira] [Created] (SPARK-47632) Ban `com.amazonaws:aws-java-sdk-bundle` dependency

2024-03-28 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-47632: - Summary: Ban `com.amazonaws:aws-java-sdk-bundle` dependency Key: SPARK-47632 URL: https://issues.apache.org/jira/browse/SPARK-47632 Project: Spark Issue

[jira] [Created] (SPARK-47631) Remove unused `SQLConf.parquetOutputCommitterClass` method

2024-03-28 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-47631: - Summary: Remove unused `SQLConf.parquetOutputCommitterClass` method Key: SPARK-47631 URL: https://issues.apache.org/jira/browse/SPARK-47631 Project: Spark

[jira] [Resolved] (SPARK-47630) Upgrade `zstd-jni` to 1.5.6-1

2024-03-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47630. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45756 [https

[jira] [Resolved] (ORC-1670) Upgrade `zstd-jni` to 1.5.6-1

2024-03-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/ORC-1670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved ORC-1670. Fix Version/s: 2.0.1 2.1.0 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (ORC-1670) Upgrade `zstd-jni` to 1.5.6-1

2024-03-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/ORC-1670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned ORC-1670: -- Assignee: Dongjoon Hyun > Upgrade `zstd-jni` to 1.5.

[jira] [Created] (ORC-1670) Upgrade `zstd-jni` to 1.5.6-1

2024-03-28 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created ORC-1670: -- Summary: Upgrade `zstd-jni` to 1.5.6-1 Key: ORC-1670 URL: https://issues.apache.org/jira/browse/ORC-1670 Project: ORC Issue Type: Bug Components: Java

[jira] [Updated] (SPARK-47475) Support `spark.kubernetes.jars.avoidDownloadSchemes` for K8s Cluster Mode

2024-03-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47475: -- Summary: Support `spark.kubernetes.jars.avoidDownloadSchemes` for K8s Cluster Mode

Re: The dedicated repository for Kubernetes Operator for Apache Spark

2024-03-28 Thread Dongjoon Hyun
Thank you, Liang-Chi! Dongjoon. On Wed, Mar 27, 2024 at 10:56 PM L. C. Hsieh wrote: > Hi all, > > For the passed SPIP: An Official Kubernetes Operator for Apache Spark, > the developers have been working on code cleaning and refactoring for > open source in the last few months. They are ready

[jira] [Assigned] (SPARK-47628) Fix Postgres bit array issue 'Cannot cast to boolean'

2024-03-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47628: - Assignee: Kent Yao > Fix Postgres bit array issue 'Cannot cast to bool

[jira] [Resolved] (SPARK-47628) Fix Postgres bit array issue 'Cannot cast to boolean'

2024-03-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47628. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45751 [https

[jira] [Updated] (SPARK-47618) Use Magic Committer for all S3 buckets by default

2024-03-27 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47618: -- Description: This issue aims to use Apache Hadoop `Magic Committer` for all S3 buckets

[jira] [Assigned] (SPARK-47618) Use Magic Committer for all S3 buckets by default

2024-03-27 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47618: - Assignee: Dongjoon Hyun > Use Magic Committer for all S3 buckets by defa

[jira] [Created] (SPARK-47618) Use Magic Committer for all S3 buckets by default

2024-03-27 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-47618: - Summary: Use Magic Committer for all S3 buckets by default Key: SPARK-47618 URL: https://issues.apache.org/jira/browse/SPARK-47618 Project: Spark Issue

[jira] [Resolved] (ORC-1668) Add merge command to Java tools documentation

2024-03-27 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/ORC-1668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved ORC-1668. Fix Version/s: 2.0.1 2.1.0 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (ORC-1668) Add merge command to Java tools documentation

2024-03-27 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/ORC-1668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned ORC-1668: -- Assignee: dzcxzl > Add merge command to Java tools documentat

[jira] [Resolved] (SPARK-47616) Document Mapping Spark SQL Data Types from MySQL

2024-03-27 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47616. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45736 [https

[jira] [Assigned] (SPARK-47616) Document Mapping Spark SQL Data Types from MySQL

2024-03-27 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47616: - Assignee: Kent Yao > Document Mapping Spark SQL Data Types from My

[jira] [Resolved] (SPARK-47611) Cleanup dead code in MySQLDialect.getCatalystType

2024-03-27 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47611. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45734 [https

[jira] [Resolved] (SPARK-47491) Re-enable `driver log links` test in YarnClusterSuite

2024-03-27 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47491. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45618 [https

[jira] [Resolved] (ORC-1665) Enable the `importOrder` function in `spotless-maven-plugin`

2024-03-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/ORC-1665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved ORC-1665. Fix Version/s: 2.1.0 Resolution: Fixed Issue resolved by pull request 1860 [https

[jira] [Assigned] (ORC-1665) Enable the `importOrder` function in `spotless-maven-plugin`

2024-03-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/ORC-1665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned ORC-1665: -- Assignee: dzcxzl > Enable the `importOrder` function in `spotless-maven-plu

[jira] [Resolved] (ORC-1666) Remove extra newlines at the end of Java files

2024-03-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/ORC-1666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved ORC-1666. Fix Version/s: 2.1.0 Resolution: Fixed Issue resolved by pull request 1861 [https

[jira] [Assigned] (ORC-1666) Remove extra newlines at the end of Java files

2024-03-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/ORC-1666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned ORC-1666: -- Assignee: dzcxzl > Remove extra newlines at the end of Java fi

[jira] [Assigned] (ORC-1664) Enable the removeUnusedImports function in spotless-maven-plugin

2024-03-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/ORC-1664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned ORC-1664: -- Assignee: dzcxzl > Enable the removeUnusedImports function in spotless-maven-plu

[jira] [Resolved] (ORC-1664) Enable the removeUnusedImports function in spotless-maven-plugin

2024-03-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/ORC-1664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved ORC-1664. Fix Version/s: 2.1.0 Resolution: Fixed Issue resolved by pull request 1859 [https

[jira] [Updated] (SPARK-47555) Show a warning message about SQLException if `JDBCTableCatalog.loadTable` fails

2024-03-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47555: -- Summary: Show a warning message about SQLException if `JDBCTableCatalog.loadTable` fails

[jira] [Assigned] (SPARK-47555) Record necessary raw exception log when loadTable

2024-03-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47555: - Assignee: xleoken > Record necessary raw exception log when loadTa

[jira] [Updated] (SPARK-47555) Record necessary raw exception log when loadTable

2024-03-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47555: -- Parent: SPARK-47361 Issue Type: Sub-task (was: Improvement) > Record necessary

[jira] [Resolved] (SPARK-47555) Record necessary raw exception log when loadTable

2024-03-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47555. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45711 [https

[jira] [Updated] (SPARK-47555) Record necessary raw exception log when loadTable

2024-03-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47555: -- Affects Version/s: 4.0.0 (was: 3.5.1) > Record necessary

[jira] [Assigned] (SPARK-47561) fix analyzer rule order issues about Alias

2024-03-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47561: - Assignee: Wenchen Fan > fix analyzer rule order issues about Al

[jira] [Resolved] (SPARK-47561) fix analyzer rule order issues about Alias

2024-03-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47561. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45718 [https

[jira] [Resolved] (SPARK-47544) [Pyspark] SparkSession builder method is incompatible with vs code intellisense

2024-03-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47544. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45700 [https

[jira] [Assigned] (SPARK-47544) [Pyspark] SparkSession builder method is incompatible with vs code intellisense

2024-03-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47544: - Assignee: Niranjan Jayakar > [Pyspark] SparkSession builder method is incompati

[jira] [Assigned] (SPARK-47557) Audit MySQL ENUM/SET Types

2024-03-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47557: - Assignee: Kent Yao > Audit MySQL ENUM/SET Ty

[jira] [Resolved] (SPARK-47557) Audit MySQL ENUM/SET Types

2024-03-26 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47557. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45713 [https

[jira] [Updated] (SPARK-47549) Remove Spark 3.0~3.2 pyspark/version.py workaround from release scripts

2024-03-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47549: -- Parent: SPARK-44111 Issue Type: Sub-task (was: Improvement) > Remove Spark 3.0~

[jira] [Assigned] (SPARK-47552) Set spark.hadoop.fs.s3a.connection.establish.timeout to 30s

2024-03-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47552: - Assignee: Dongjoon Hyun > Set spark.hadoop.fs.s3a.connection.establish.timeout to

[jira] [Resolved] (SPARK-47552) Set spark.hadoop.fs.s3a.connection.establish.timeout to 30s

2024-03-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47552. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45710 [https

[jira] [Created] (SPARK-47552) Set spark.hadoop.fs.s3a.connection.establish.timeout to 30s

2024-03-25 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-47552: - Summary: Set spark.hadoop.fs.s3a.connection.establish.timeout to 30s Key: SPARK-47552 URL: https://issues.apache.org/jira/browse/SPARK-47552 Project: Spark

[jira] [Updated] (SPARK-47550) Update kubernetes-client to 6.11.0

2024-03-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47550: -- Parent: SPARK-44111 Issue Type: Sub-task (was: Dependency upgrade) > Upd

[jira] [Updated] (SPARK-47550) Upgrade kubernetes-client to 6.11.0

2024-03-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47550: -- Summary: Upgrade kubernetes-client to 6.11.0 (was: Update kubernetes-client to 6.11.0

[jira] [Resolved] (SPARK-47550) Update kubernetes-client to 6.11.0

2024-03-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47550. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45707 [https

[jira] [Assigned] (SPARK-47550) Update kubernetes-client to 6.11.0

2024-03-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47550: - Assignee: Bjørn Jørgensen > Update kubernetes-client to 6.1

[jira] [Assigned] (SPARK-47548) Remove unused `commons-beanutils` dependency

2024-03-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47548: - Assignee: Dongjoon Hyun > Remove unused `commons-beanutils` depende

[jira] [Resolved] (SPARK-47548) Remove unused `commons-beanutils` dependency

2024-03-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47548. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45705 [https

[jira] [Created] (SPARK-47549) Remove Spark 3.0~3.2 pyspark/version.py workaround from release scripts

2024-03-25 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-47549: - Summary: Remove Spark 3.0~3.2 pyspark/version.py workaround from release scripts Key: SPARK-47549 URL: https://issues.apache.org/jira/browse/SPARK-47549 Project

[jira] [Updated] (SPARK-46743) Count bug introduced for scalar subquery when using TEMPORARY VIEW, as compared to using table

2024-03-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-46743: -- Labels: correctness pull-request-available (was: pull-request-available) > Count

[jira] [Updated] (SPARK-46743) Count bug introduced for scalar subquery when using TEMPORARY VIEW, as compared to using table

2024-03-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-46743: -- Component/s: SQL (was: Optimizer) > Count bug introduced for sca

[jira] [Created] (SPARK-47548) Remove unused `commons-beanutils` dependency

2024-03-25 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-47548: - Summary: Remove unused `commons-beanutils` dependency Key: SPARK-47548 URL: https://issues.apache.org/jira/browse/SPARK-47548 Project: Spark Issue Type

[jira] [Commented] (SPARK-42452) Remove hadoop-2 profile from Apache Spark

2024-03-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17830566#comment-17830566 ] Dongjoon Hyun commented on SPARK-42452: --- This was resolved via https://github.com/apache/spark

[jira] [Updated] (SPARK-47503) Spark history sever fails to display query for cached JDBC relation named in quotes

2024-03-25 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47503: -- Fix Version/s: 3.4.3 > Spark history sever fails to display query for cached JDBC relat

<    3   4   5   6   7   8   9   10   11   12   >