[jira] [Resolved] (SPARK-47898) Port HIVE-12270: Add DBTokenStore support to HS2 delegation token

2024-04-18 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47898. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46115 [https

[ANNOUNCE] Apache Spark 3.4.3 released

2024-04-18 Thread Dongjoon Hyun
not have been possible without you. Dongjoon Hyun

[ANNOUNCE] Apache Spark 3.4.3 released

2024-04-18 Thread Dongjoon Hyun
not have been possible without you. Dongjoon Hyun

[jira] [Resolved] (SPARK-47889) Setup gradle as build tool for operator repository

2024-04-18 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47889. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 4 [https

[jira] [Assigned] (SPARK-47889) Setup gradle as build tool for operator repository

2024-04-18 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47889: - Assignee: Zhou JIANG > Setup gradle as build tool for operator reposit

[jira] [Updated] (SPARK-47889) Setup gradle as build tool for operator repository

2024-04-18 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47889: -- Fix Version/s: kubernetes-operator-0.1.0 (was: 4.0.0) > Setup gra

[jira] [Assigned] (SPARK-47887) Remove unused import `spark/connect/common.proto` from `spark/connect/relations.proto`

2024-04-18 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47887: - Assignee: Yang Jie > Remove unused import `spark/connect/common.proto` from >

[jira] [Resolved] (SPARK-47887) Remove unused import `spark/connect/common.proto` from `spark/connect/relations.proto`

2024-04-18 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47887. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46106 [https

[jira] [Resolved] (SPARK-47893) Upgrade ASM to 9.7

2024-04-18 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47893. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46110 [https

[VOTE][RESULT] Release Spark 3.4.3 (RC2)

2024-04-18 Thread Dongjoon Hyun
The vote passes with 10 +1s (8 binding +1s). Thanks to all who helped with the release! (* = binding) +1: - Dongjoon Hyun * - Mridul Muralidharan * - Wenchen Fan * - Liang-Chi Hsieh * - Gengliang Wang * - Hyukjin Kwon * - Bo Yang - DB Tsai * - Kent Yao - Huaxin Gao * +0: None -1: None

Re: [VOTE] Release Spark 3.4.3 (RC2)

2024-04-18 Thread Dongjoon Hyun
> > > > > > +1 > > > > > > On Tue, Apr 16, 2024 at 1:38 PM Hyukjin Kwon > > wrote: > > >> > > >> +1 > > >> > > >> On Wed, Apr 17, 2024 at 3:57 AM L. C. Hsieh wrote: > > >>> > > >>&

[jira] [Updated] (SPARK-47896) Upgrade netty to `4.1.109.Final`

2024-04-18 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47896: -- Parent: SPARK-47046 Issue Type: Sub-task (was: Improvement) > Upgrade ne

[jira] [Assigned] (SPARK-47896) Upgrade netty to `4.1.109.Final`

2024-04-18 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47896: - Assignee: BingKun Pan > Upgrade netty to `4.1.109.Fi

[jira] [Resolved] (SPARK-47896) Upgrade netty to `4.1.109.Final`

2024-04-18 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47896. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46112 [https

[jira] [Assigned] (SPARK-47882) createTableColumnTypes need to be mapped to database types instead of using directly

2024-04-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47882: - Assignee: Kent Yao > createTableColumnTypes need to be mapped to database ty

[jira] [Resolved] (SPARK-47882) createTableColumnTypes need to be mapped to database types instead of using directly

2024-04-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47882. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46093 [https

[jira] [Resolved] (SPARK-47894) Add `Environment` page to Master UI

2024-04-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47894. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46111 [https

[jira] [Assigned] (SPARK-47894) Add `Environment` page to Master UI

2024-04-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47894: - Assignee: Dongjoon Hyun > Add `Environment` page to Master

[jira] [Created] (SPARK-47894) Add `Environment` page to Master UI

2024-04-17 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-47894: - Summary: Add `Environment` page to Master UI Key: SPARK-47894 URL: https://issues.apache.org/jira/browse/SPARK-47894 Project: Spark Issue Type: Sub-task

[jira] [Resolved] (SPARK-47726) Document push-based shuffle metrics

2024-04-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47726. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45872 [https

[jira] [Assigned] (SPARK-47726) Document push-based shuffle metrics

2024-04-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47726: - Assignee: Luca Canali > Document push-based shuffle metr

[jira] [Resolved] (ORC-1693) Bump `maven-jar-plugin` to 3.4.0

2024-04-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/ORC-1693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved ORC-1693. Fix Version/s: 2.1.0 Resolution: Fixed Issue resolved by pull request 1890 [https

[jira] [Assigned] (ORC-1693) Bump `maven-jar-plugin` to 3.4.0

2024-04-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/ORC-1693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned ORC-1693: -- Assignee: Dongjoon Hyun > Bump `maven-jar-plugin` to 3.

[jira] [Created] (ORC-1693) Bump `maven-jar-plugin` to 3.4.0

2024-04-17 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created ORC-1693: -- Summary: Bump `maven-jar-plugin` to 3.4.0 Key: ORC-1693 URL: https://issues.apache.org/jira/browse/ORC-1693 Project: ORC Issue Type: Task Components

[jira] [Resolved] (ORC-1692) Bump `slf4j` to 2.0.13

2024-04-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/ORC-1692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved ORC-1692. Fix Version/s: 2.1.0 Resolution: Fixed Issue resolved by pull request 1891 [https

[jira] [Assigned] (ORC-1692) Bump `slf4j` to 2.0.13

2024-04-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/ORC-1692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned ORC-1692: -- Assignee: Dongjoon Hyun > Bump `slf4j` to 2.0

[jira] [Created] (ORC-1692) Bump `slf4j` to 2.0.13

2024-04-17 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created ORC-1692: -- Summary: Bump `slf4j` to 2.0.13 Key: ORC-1692 URL: https://issues.apache.org/jira/browse/ORC-1692 Project: ORC Issue Type: Task Components: Java

[jira] [Assigned] (ORC-1691) Bump `spotbugs-maven-plugin` to 4.8.4.0

2024-04-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/ORC-1691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned ORC-1691: -- Assignee: Dongjoon Hyun > Bump `spotbugs-maven-plugin` to 4.8.

[jira] [Created] (ORC-1691) Bump `spotbugs-maven-plugin` to 4.8.4.0

2024-04-17 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created ORC-1691: -- Summary: Bump `spotbugs-maven-plugin` to 4.8.4.0 Key: ORC-1691 URL: https://issues.apache.org/jira/browse/ORC-1691 Project: ORC Issue Type: Test

[jira] [Resolved] (ORC-1691) Bump `spotbugs-maven-plugin` to 4.8.4.0

2024-04-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/ORC-1691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved ORC-1691. Fix Version/s: 2.1.0 Resolution: Fixed Issue resolved by pull request 1892 [https

[jira] [Resolved] (SPARK-47886) Postgres: Add test and doc for Postgres special numeric values

2024-04-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47886. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46102 [https

[jira] [Updated] (SPARK-47884) Switch ANSI SQL CI job to NON-ANSI SQL CI job

2024-04-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47884: -- Summary: Switch ANSI SQL CI job to NON-ANSI SQL CI job (was: Switch ANSI SQL CI to NON-ANSI

[jira] [Assigned] (SPARK-47884) Switch ANSI SQL CI job to NON-ANSI SQL CI job

2024-04-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47884: - Assignee: Dongjoon Hyun > Switch ANSI SQL CI job to NON-ANSI SQL CI

[jira] [Created] (SPARK-47884) Switch ANSI SQL CI to NON-ANSI SQL CI

2024-04-17 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-47884: - Summary: Switch ANSI SQL CI to NON-ANSI SQL CI Key: SPARK-47884 URL: https://issues.apache.org/jira/browse/SPARK-47884 Project: Spark Issue Type: Sub-task

[jira] [Resolved] (SPARK-44444) Use ANSI mode by default

2024-04-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-4?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-4. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46013 [https

[jira] [Updated] (SPARK-44444) Use ANSI SQL mode by default

2024-04-17 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-4?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-4: -- Summary: Use ANSI SQL mode by default (was: Use ANSI mode by default) > Use ANSI SQL m

[VOTE][RESULT] SPARK-44444: Use ANSI SQL mode by default

2024-04-17 Thread Dongjoon Hyun
The vote passes with 24 +1s (13 binding +1s). Thanks to all who helped with the vote! (* = binding) +1: - Dongjoon Hyun * - Gengliang Wang * - Chao Sun * - Hyukjin Kwon * - Liang-Chi Hsieh * - Holden Karau * - Huaxin Gao * - Denny Lee - Xiao Li * - Mich Talebzadeh - Christiano Anderson - Yang Jie

Re: [VOTE] SPARK-44444: Use ANSI SQL mode by default

2024-04-17 Thread Dongjoon Hyun
t; Thanks Dongjoon to drive this! > >> > >> > >> -Rui > >> > >> On Mon, Apr 15, 2024 at 10:10 AM Xinrong Meng wrote: > >> > >>> +1 > >>> > >>> Thank you @Dongjoon Hyun ! > >>> > >>> On

[jira] [Resolved] (SPARK-47875) Remove `spark.deploy.recoverySerializer`

2024-04-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47875. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46087 [https

[jira] [Created] (SPARK-47875) Remove `spark.deploy.recoverySerializer`

2024-04-16 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-47875: - Summary: Remove `spark.deploy.recoverySerializer` Key: SPARK-47875 URL: https://issues.apache.org/jira/browse/SPARK-47875 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-47871) Oracle: Map TimestampType to TIMESTAMP WITH LOCAL TIME ZONE

2024-04-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47871. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46080 [https

[jira] [Assigned] (SPARK-47871) Oracle: Map TimestampType to TIMESTAMP WITH LOCAL TIME ZONE

2024-04-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47871: - Assignee: Kent Yao > Oracle: Map TimestampType to TIMESTAMP WITH LOCAL TIME Z

[jira] [Updated] (SPARK-47745) Add License to Spark Operator

2024-04-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47745: -- Affects Version/s: kubernetes-operator-0.1.0 (was: 4.0.0) >

[jira] [Updated] (SPARK-47745) Add License to Spark Operator

2024-04-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47745: -- Fix Version/s: kubernetes-operator-0.1.0 (was: 4.0.0) > Add Lice

[jira] [Resolved] (SPARK-47745) Add License to Spark Operator

2024-04-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47745. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 3 [https

[jira] [Assigned] (SPARK-47745) Add License to Spark Operator

2024-04-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47745: - Assignee: Zhou JIANG > Add License to Spark Opera

[jira] [Resolved] (SPARK-47739) Avro does not register custom logical types on spark startup

2024-04-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47739. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45895 [https

[jira] [Resolved] (SPARK-46574) Upgrade maven plugin to latest version

2024-04-16 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-46574. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46043 [https

[jira] [Created] (SPARK-47865) Deflaky PythonForeachWriterSuite

2024-04-15 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-47865: - Summary: Deflaky PythonForeachWriterSuite Key: SPARK-47865 URL: https://issues.apache.org/jira/browse/SPARK-47865 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-47866) Deflaky PythonForeachWriterSuite

2024-04-15 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-47866: - Summary: Deflaky PythonForeachWriterSuite Key: SPARK-47866 URL: https://issues.apache.org/jira/browse/SPARK-47866 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-43394) Upgrade maven to 3.8.8

2024-04-15 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-43394: -- Fix Version/s: 3.4.4 > Upgrade maven to 3.

[jira] [Updated] (SPARK-47828) DataFrameWriterV2.overwrite fails with invalid plan

2024-04-15 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47828: -- Fix Version/s: 3.5.2 > DataFrameWriterV2.overwrite fails with invalid p

[jira] [Assigned] (SPARK-47855) Warn `spark.sql.execution.arrow.pyspark.fallback.enabled` in Connect

2024-04-15 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47855: - Assignee: Ruifeng Zheng > Warn `spark.sql.execution.arrow.pyspark.fallback.enab

[jira] [Resolved] (SPARK-47855) Warn `spark.sql.execution.arrow.pyspark.fallback.enabled` in Connect

2024-04-15 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47855. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46056 [https

[jira] [Resolved] (SPARK-47860) Upgrade `kubernetes-client` to 6.12.0

2024-04-15 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47860. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46066 [https

[jira] [Created] (SPARK-47860) Upgrade `kubernetes-client` to 6.12.0

2024-04-15 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-47860: - Summary: Upgrade `kubernetes-client` to 6.12.0 Key: SPARK-47860 URL: https://issues.apache.org/jira/browse/SPARK-47860 Project: Spark Issue Type: Sub-task

[jira] [Resolved] (SPARK-47856) Oracle: Document Mapping Spark SQL Data Types from Oracle

2024-04-15 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47856. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46059 [https

[jira] [Assigned] (SPARK-47856) Oracle: Document Mapping Spark SQL Data Types from Oracle

2024-04-15 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47856: - Assignee: Kent Yao > Oracle: Document Mapping Spark SQL Data Types from Ora

[jira] [Updated] (SPARK-47856) Oracle: Document Mapping Spark SQL Data Types from Oracle and add tests

2024-04-15 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47856: -- Summary: Oracle: Document Mapping Spark SQL Data Types from Oracle and add tests

Re: [VOTE] Release Spark 3.4.3 (RC2)

2024-04-14 Thread Dongjoon Hyun
I'll start with my +1. - Checked checksum and signature - Checked Scala/Java/R/Python/SQL Document's Spark version - Checked published Maven artifacts - All CIs passed. Thanks, Dongjoon. On 2024/04/15 04:22:26 Dongjoon Hyun wrote: > Please vote on releasing the following candidate as Apa

[VOTE] Release Spark 3.4.3 (RC2)

2024-04-14 Thread Dongjoon Hyun
Please vote on releasing the following candidate as Apache Spark version 3.4.3. The vote is open until April 18th 1AM (PDT) and passes if a majority +1 PMC votes are cast, with a minimum of 3 +1 votes. [ ] +1 Release this package as Apache Spark 3.4.3 [ ] -1 Do not release this package because

[jira] [Updated] (SPARK-47694) Make max message size configurable on client side

2024-04-14 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47694: -- Fix Version/s: (was: 3.4.3) > Make max message size configurable on client s

[jira] [Updated] (SPARK-47694) Make max message size configurable on client side

2024-04-14 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47694: -- Affects Version/s: 4.0.0 (was: 3.4.1) > Make max message s

[ANNOUNCE] Announcing Apache ORC 1.8.7

2024-04-14 Thread Dongjoon Hyun
Hi All. We are happy to announce the availability of Apache ORC 1.8.7! https://orc.apache.org/news/2024/04/14/ORC-1.8.7/ 1.8.7 is a maintenance release containing important fixes. It's available in Apache Downloads and Maven Central. https://downloads.apache.org/orc/orc-1.8.7/

[ANNOUNCE] Announcing Apache ORC 1.8.7

2024-04-14 Thread Dongjoon Hyun
Hi All. We are happy to announce the availability of Apache ORC 1.8.7! https://orc.apache.org/news/2024/04/14/ORC-1.8.7/ 1.8.7 is a maintenance release containing important fixes. It's available in Apache Downloads and Maven Central. https://downloads.apache.org/orc/orc-1.8.7/

[VOTE][RESULT] Release Apache ORC 1.8.7 (RC0)

2024-04-14 Thread Dongjoon Hyun
The vote passed. Thanks to all who helped with the release! (* = binding) +1: - Dongjoon Hyun * - Gang Wu * - Shaoyun Chen - William Hyun * +0: None -1: None

Re: [VOTE] Release Apache ORC 1.8.7 (RC0)

2024-04-14 Thread Dongjoon Hyun
Thank you all. This vote passed. Dongjoon. On 2024/04/13 21:56:05 Dongjoon Hyun wrote: > Thank you, all. > > The vote will end on April 14th 1AM (PST). > > Dongjoon. > > On 2024/04/11 18:27:18 "William H." wrote: > > +1 > > > > Thank you

[jira] [Resolved] (SPARK-47841) Upgrade `postgresql` to 42.7.3

2024-04-13 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47841. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46038 [https

[jira] [Assigned] (ORC-1669) [C++] Deprecate HDFS support

2024-04-13 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/ORC-1669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned ORC-1669: -- Assignee: Gang Wu (was: Yuanping Wu) > [C++] Deprecate HDFS supp

[jira] [Assigned] (ORC-1669) [C++] Deprecate HDFS support

2024-04-13 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/ORC-1669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned ORC-1669: -- Assignee: Yuanping Wu > [C++] Deprecate HDFS supp

[jira] [Resolved] (ORC-1669) [C++] Deprecate HDFS support

2024-04-13 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/ORC-1669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved ORC-1669. Fix Version/s: 2.0.1 2.1.0 Resolution: Fixed Issue resolved by pull

[jira] [Updated] (SPARK-47318) AuthEngine key exchange needs additional KDF round

2024-04-13 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47318: -- Fix Version/s: 3.4.3 > AuthEngine key exchange needs additional KDF ro

[jira] [Updated] (SPARK-47318) AuthEngine key exchange needs additional KDF round

2024-04-13 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47318: -- Fix Version/s: 3.5.2 > AuthEngine key exchange needs additional KDF ro

[jira] [Commented] (SPARK-44444) Use ANSI mode by default

2024-04-13 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-4?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17836869#comment-17836869 ] Dongjoon Hyun commented on SPARK-4: --- Thank you all. The vote is started here. - https

Re: [VOTE] SPARK-44444: Use ANSI SQL mode by default

2024-04-13 Thread Dongjoon Hyun
I'll start from my +1. Dongjoon. On 2024/04/13 22:22:05 Dongjoon Hyun wrote: > Please vote on SPARK-4 to use ANSI SQL mode by default. > The technical scope is defined in the following PR which is > one line of code change and one line of migration guide. > > - DISC

[jira] [Updated] (SPARK-44444) Use ANSI mode by default

2024-04-13 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-4?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-4: -- Summary: Use ANSI mode by default (was: Enabled ANSI mode by default) > Use ANSI m

[VOTE] SPARK-44444: Use ANSI SQL mode by default

2024-04-13 Thread Dongjoon Hyun
Please vote on SPARK-4 to use ANSI SQL mode by default. The technical scope is defined in the following PR which is one line of code change and one line of migration guide. - DISCUSSION: https://lists.apache.org/thread/ztlwoz1v1sn81ssks12tb19x37zozxlz - JIRA:

Re: [DISCUSS] SPARK-44444: Use ANSI SQL mode by default

2024-04-13 Thread Dongjoon Hyun
ror Attribution Framework > > <https://issues.apache.org/jira/browse/SPARK-38615> will also be beneficial > > in migrating to ANSI SQL mode. > > > > > > Gengliang > > > > > > On Thu, Apr 11, 2024 at 7:56 PM Dongjoon Hyun > <mailto:dongjoon.h

Re: [VOTE] Release Apache ORC 1.8.7 (RC0)

2024-04-13 Thread Dongjoon Hyun
-test > > > 7/8 Test #7: java-bench-spark-test Passed 215.06 sec > > > Start 8: tool-test > > > 8/8 Test #8: tool-test Passed 17.01 sec > > > > > > 100% tests passed, 0 tests failed out of 8 &

[jira] [Commented] (SPARK-46122) Disable spark.sql.legacy.createHiveTableByDefault by default

2024-04-12 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17836708#comment-17836708 ] Dongjoon Hyun commented on SPARK-46122: --- Gentle ping, [~yumwang] . You can throw a discussion

Re: [DISCUSS] Spark 4.0.0 release

2024-04-12 Thread Dongjoon Hyun
Thank you for volunteering, Wenchen. Dongjoon. On 2024/04/12 15:11:04 Wenchen Fan wrote: > Hi all, > > It's close to the previously proposed 4.0.0 release date (June 2024), and I > think it's time to prepare for it and discuss the ongoing projects: > >- ANSI by default >- Spark Connect

[jira] [Updated] (SPARK-44518) Completely make hive as a data source

2024-04-12 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-44518: -- Parent: (was: SPARK-44111) Issue Type: Improvement (was: Sub-task) > Complet

[jira] [Updated] (SPARK-44518) Completely make hive as a data source

2024-04-12 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-44518: -- Target Version/s: (was: 4.0.0) > Completely make hive as a data sou

[jira] [Updated] (SPARK-44518) Completely make hive as a data source

2024-04-12 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-44518: -- Fix Version/s: (was: 4.0.0) > Completely make hive as a data sou

[jira] [Commented] (SPARK-44518) Completely make hive as a data source

2024-04-12 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17836704#comment-17836704 ] Dongjoon Hyun commented on SPARK-44518: --- In addition, I'll convert this to an independent JIRA

[jira] [Commented] (SPARK-44518) Completely make hive as a data source

2024-04-12 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17836703#comment-17836703 ] Dongjoon Hyun commented on SPARK-44518: --- Hi, [~roryqi], this looks like a question instead of any

[jira] [Updated] (SPARK-44116) Utilize Hadoop vectorized APIs

2024-04-12 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-44116: -- Parent: (was: SPARK-44111) Issue Type: New Feature (was: Sub-task) > Util

[jira] [Commented] (HADOOP-18296) Memory fragmentation in ChecksumFileSystem Vectored IO implementation.

2024-04-12 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17836697#comment-17836697 ] Dongjoon Hyun commented on HADOOP-18296: Thank you for sharing the direction. > Mem

[jira] [Assigned] (SPARK-44444) Enabled ANSI mode by default

2024-04-12 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-4?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-4: - Assignee: Dongjoon Hyun > Enabled ANSI mode by defa

Re: [VOTE] Add new `Versions` in Apache Spark JIRA for Versioning of Spark Operator

2024-04-12 Thread Dongjoon Hyun
+1 Thank you! I hope we can customize `dev/merge_spark_pr.py` script per repository after this PR. Dongjoon. On 2024/04/12 03:28:36 "L. C. Hsieh" wrote: > Hi all, > > Thanks for all discussions in the thread of "Versioning of Spark > Operator":

[jira] [Commented] (SPARK-44444) Enabled ANSI mode by default

2024-04-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-4?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17836397#comment-17836397 ] Dongjoon Hyun commented on SPARK-4: --- Hi, All. Here is the discussion thread I sent a few

[DISCUSS] SPARK-44444: Use ANSI SQL mode by default

2024-04-11 Thread Dongjoon Hyun
Hi, All. Thanks to you, we've been achieving many things and have on-going SPIPs. I believe it's time to scope Apache Spark 4.0.0 (SPARK-44111) more narrowly by asking your opinions about Apache Spark's ANSI SQL mode. https://issues.apache.org/jira/browse/SPARK-44111 Prepare Apache Spark

[jira] [Commented] (SPARK-44444) Enabled ANSI mode by default

2024-04-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-4?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17836389#comment-17836389 ] Dongjoon Hyun commented on SPARK-4: --- I made a draft PR and will initiate the discussion thread

[jira] [Assigned] (SPARK-47814) Remove the `KinesisTestUtils` from `main` to `test`

2024-04-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47814: - Assignee: BingKun Pan > Remove the `KinesisTestUtils` from `main` to `t

[jira] [Resolved] (SPARK-47814) Remove the `KinesisTestUtils` from `main` to `test`

2024-04-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47814. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46000 [https

[jira] [Updated] (SPARK-47777) Add spark connect test for python streaming data source

2024-04-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-4?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-4: -- Component/s: Tests > Add spark connect test for python streaming data sou

[jira] [Assigned] (SPARK-47777) Add spark connect test for python streaming data source

2024-04-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-4?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-4: - Assignee: Chaoqin Li > Add spark connect test for python streaming data sou

[jira] [Resolved] (SPARK-47777) Add spark connect test for python streaming data source

2024-04-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-4?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-4. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45950 [https

[jira] [Resolved] (SPARK-47820) Run `ANSI` SQL CI twice per day

2024-04-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47820. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46010 [https

[jira] [Updated] (SPARK-47820) Run `ANSI` SQL CI twice per day

2024-04-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47820: -- Summary: Run `ANSI` SQL CI twice per day (was: Run `ANSI` SQL Daily CI twice) > Run `A

<    1   2   3   4   5   6   7   8   9   10   >