[jira] [Comment Edited] (ORC-642) ORC specification for RLEv2 is incomplete

2024-04-10 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/ORC-642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17835810#comment-17835810 ] Dongjoon Hyun edited comment on ORC-642 at 4/10/24 4:18 PM: I manually added

[jira] [Commented] (ORC-642) ORC specification for RLEv2 is incomplete

2024-04-10 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/ORC-642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17835810#comment-17835810 ] Dongjoon Hyun commented on ORC-642: --- I manually added and assigned him. > ORC specification for RL

[jira] [Commented] (ORC-642) ORC specification for RLEv2 is incomplete

2024-04-10 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/ORC-642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17835808#comment-17835808 ] Dongjoon Hyun commented on ORC-642: --- Did you try to use the official merge script, [~deshanxiao

[jira] [Assigned] (ORC-642) ORC specification for RLEv2 is incomplete

2024-04-10 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/ORC-642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned ORC-642: - Assignee: Jeffrey Vo > ORC specification for RLEv2 is incompl

[jira] [Updated] (SPARK-47790) Upgrade `commons-io` to 2.16.1

2024-04-10 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47790: -- Fix Version/s: 3.5.2 > Upgrade `commons-io` to 2.1

[jira] [Resolved] (SPARK-47790) Upgrade `commons-io` to 2.16.1

2024-04-10 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47790. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45973 [https

Re: Versioning of Spark Operator

2024-04-10 Thread Dongjoon Hyun
Ya, that would work. Inevitably, I looked at Apache Flink K8s Operator's JIRA and GitHub repo. It looks reasonable to me. Although they share the same JIRA, they choose different patterns per place. 1. In POM file and Maven Artifact, independent version number. 1.8.0 2. Tag is also based on

[jira] [Resolved] (SPARK-47706) Bump json4s 4.0.7

2024-04-10 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47706. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45838 [https

[jira] [Updated] (SPARK-45444) Upgrade `commons-io` to 2.14.0

2024-04-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45444: -- Summary: Upgrade `commons-io` to 2.14.0 (was: Upgrade `commons-io` to 1.24.0) > Upgr

[jira] [Updated] (SPARK-45444) Upgrade `commons-io` to 1.24.0

2024-04-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45444: -- Parent: SPARK-47046 Issue Type: Sub-task (was: Improvement) > Upgrade `commons

[jira] [Updated] (SPARK-45734) Upgrade commons-io to 2.15.0

2024-04-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45734: -- Parent: SPARK-47046 Issue Type: Sub-task (was: Improvement) > Upgrade commons

[jira] [Updated] (SPARK-47083) Upgrade `commons-codec` to 1.16.1

2024-04-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47083: -- Fix Version/s: 3.5.2 > Upgrade `commons-codec` to 1.1

[jira] [Updated] (SPARK-47182) Exclude `commons-(io|lang3)` transitive dependencies from `commons-compress` and `avro-*`

2024-04-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47182: -- Fix Version/s: 3.5.2 > Exclude `commons-(io|lang3)` transitive dependencies from `comm

[jira] [Resolved] (SPARK-47787) Upgrade `commons-compress` to 1.26.1`

2024-04-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47787. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45966 [https

Re: Versioning of Spark Operator

2024-04-09 Thread Dongjoon Hyun
ctor Go Client? For example, > Spark Operator 3.5.x supports Spark 3.5 and above. > > Best, > Bo > > > On Tue, Apr 9, 2024 at 10:14 AM Dongjoon Hyun wrote: > > > Ya, that's simple and possible. > > > > However, it may cause many confusions because it implie

[jira] [Updated] (SPARK-47787) Upgrade `commons-compress` to 1.26.1`

2024-04-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47787: -- Summary: Upgrade `commons-compress` to 1.26.1` (was: Upgrade `commons-compress` to 1.26

[jira] [Created] (SPARK-47787) Upgrade `commons-compress` to 1.26.`

2024-04-09 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-47787: - Summary: Upgrade `commons-compress` to 1.26.` Key: SPARK-47787 URL: https://issues.apache.org/jira/browse/SPARK-47787 Project: Spark Issue Type: Sub-task

[jira] [Assigned] (ORC-1681) Remove redundant import statement in tests to fix checkstyle failures

2024-04-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/ORC-1681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned ORC-1681: -- Assignee: Dongjoon Hyun > Remove redundant import statement in tests to fix checkstyle failu

[jira] [Updated] (ORC-1681) Remove redundant import statement in tests to fix checkstyle failures

2024-04-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/ORC-1681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated ORC-1681: --- Summary: Remove redundant import statement in tests to fix checkstyle failures (was: Remove redundant

[jira] [Created] (ORC-1681) Remove redundant import statement to fix checkstyle failures

2024-04-09 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created ORC-1681: -- Summary: Remove redundant import statement to fix checkstyle failures Key: ORC-1681 URL: https://issues.apache.org/jira/browse/ORC-1681 Project: ORC Issue Type

[jira] [Resolved] (ORC-1680) Bump `bcpkix-jdk18on` to 1.78

2024-04-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/ORC-1680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved ORC-1680. Fix Version/s: 2.1.0 Resolution: Fixed Issue resolved by pull request 1880 [https

[jira] [Assigned] (ORC-1680) Bump `bcpkix-jdk18on` to 1.78

2024-04-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/ORC-1680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned ORC-1680: -- Assignee: Dongjoon Hyun > Bump `bcpkix-jdk18on` to 1

[jira] [Created] (ORC-1680) Bump `bcpkix-jdk18on` to 1.78

2024-04-09 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created ORC-1680: -- Summary: Bump `bcpkix-jdk18on` to 1.78 Key: ORC-1680 URL: https://issues.apache.org/jira/browse/ORC-1680 Project: ORC Issue Type: Test Components

[jira] [Resolved] (ORC-1679) Bump `zstd-jni` 1.5.6-2

2024-04-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/ORC-1679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved ORC-1679. Fix Version/s: 2.0.1 2.1.0 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (ORC-1679) Bump `zstd-jni` 1.5.6-2

2024-04-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/ORC-1679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned ORC-1679: -- Assignee: Dongjoon Hyun > Bump `zstd-jni` 1.5.

[jira] [Resolved] (ORC-1678) Bump `checkstyle` to 10.15.0

2024-04-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/ORC-1678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved ORC-1678. Fix Version/s: 2.1.0 Resolution: Fixed Issue resolved by pull request 1872 [https

[jira] [Created] (ORC-1679) Bump `zstd-jni` 1.5.6-2

2024-04-09 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created ORC-1679: -- Summary: Bump `zstd-jni` 1.5.6-2 Key: ORC-1679 URL: https://issues.apache.org/jira/browse/ORC-1679 Project: ORC Issue Type: Bug Components: Java

[jira] [Assigned] (ORC-1678) Bump `checkstyle` to 10.15.0

2024-04-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/ORC-1678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned ORC-1678: -- Assignee: Dongjoon Hyun > Bump `checkstyle` to 10.1

[jira] [Created] (ORC-1678) Bump `checkstyle` to 10.15.0

2024-04-09 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created ORC-1678: -- Summary: Bump `checkstyle` to 10.15.0 Key: ORC-1678 URL: https://issues.apache.org/jira/browse/ORC-1678 Project: ORC Issue Type: Test Components: Java

Re: Versioning of Spark Operator

2024-04-09 Thread Dongjoon Hyun
elopers and > > intuitive for users. > > > > Regards, > > Mridul > > > > > > On Tue, Apr 9, 2024 at 10:09 AM Dongjoon Hyun > <mailto:dongj...@apache.org>> wrote: > >> Hi, Liang-Chi. > >> > >> Thank you for leading Ap

[jira] [Created] (SPARK-47785) Upgrade `bouncycastle` to 1.78

2024-04-09 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-47785: - Summary: Upgrade `bouncycastle` to 1.78 Key: SPARK-47785 URL: https://issues.apache.org/jira/browse/SPARK-47785 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-47748) Upgrade `zstd-jni` to 1.5.6-2

2024-04-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47748: -- Parent: SPARK-47046 Issue Type: Sub-task (was: Improvement) > Upgrade `zstd-

[jira] [Assigned] (SPARK-47782) Remove redundant json4s-jackson definition in sql/api POM

2024-04-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47782: - Assignee: Cheng Pan (was: Dongjoon Hyun) > Remove redundant json4s-jackson definit

[jira] [Assigned] (SPARK-47782) Remove redundant json4s-jackson definition in sql/api POM

2024-04-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47782: - Assignee: Dongjoon Hyun > Remove redundant json4s-jackson definition in sql/api

[jira] [Resolved] (SPARK-47782) Remove redundant json4s-jackson definition in sql/api POM

2024-04-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47782. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45943 [https

[jira] [Updated] (SPARK-47782) Remove redundant json4s-jackson definition in sql/api POM

2024-04-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47782: -- Parent: SPARK-47046 Issue Type: Sub-task (was: Task) > Remove redundant jso

[jira] [Created] (SPARK-47782) Remove redundant json4s-jackson definition in sql/api POM

2024-04-09 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-47782: - Summary: Remove redundant json4s-jackson definition in sql/api POM Key: SPARK-47782 URL: https://issues.apache.org/jira/browse/SPARK-47782 Project: Spark

Re: Versioning of Spark Operator

2024-04-09 Thread Dongjoon Hyun
Hi, Liang-Chi. Thank you for leading Apache Spark K8s operator as a shepherd. I took a look at `Apache Spark Connect Go` repo mentioned in the thread. Sadly, there is no release at all and no activity since last 6 months. It seems to be the first time for Apache Spark community to consider

[jira] [Updated] (SPARK-47774) Remove redundant rules from `MimaExcludes`

2024-04-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47774: -- Fix Version/s: 3.4.3 > Remove redundant rules from `MimaExclu

Re: SPIP: Enhancing the Flexibility of Spark's Physical Plan to Enable Execution on Various Native Engines

2024-04-09 Thread Dongjoon Hyun
Thank you for sharing, Jia. I have the same questions like the previous Weiting's thread. Do you think you can share the future milestone of Apache Gluten? I'm wondering when the first stable release will come and how we can coordinate across the ASF communities. > This project is still under

[jira] [Updated] (SPARK-47774) Remove redundant rules from `MimaExcludes`

2024-04-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47774: -- Fix Version/s: 3.5.2 > Remove redundant rules from `MimaExclu

Re: Introducing Apache Gluten(incubating), a middle layer to offload Spark to native engine

2024-04-09 Thread Dongjoon Hyun
Thank you for sharing, Weiting. Do you think you can share the future milestone of Apache Gluten? I'm wondering when the first stable release will come and how we can coordinate across the ASF communities. > This project is still under active development now, and doesn't have a stable release. >

[jira] [Resolved] (SPARK-47774) Remove redundant rules from `MimaExcludes`

2024-04-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47774. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45944 [https

[jira] [Updated] (SPARK-47773) Enhancing the Flexibility of Spark's Physical Plan to Enable Execution on Various Native Engines

2024-04-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47773: -- Affects Version/s: 4.0.0 (was: 3.5.1) > Enhancing the Flexibil

[jira] [Commented] (SPARK-47773) Enhancing the Flexibility of Spark's Physical Plan to Enable Execution on Various Native Engines

2024-04-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17835181#comment-17835181 ] Dongjoon Hyun commented on SPARK-47773: --- Since this is a new feature which cannot affect old

[jira] [Resolved] (SPARK-47754) Postgres: Support reading multidimensional arrays

2024-04-08 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47754. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45917 [https

[jira] [Created] (SPARK-47774) Remove redundant rules from `MimaExcludes`

2024-04-08 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-47774: - Summary: Remove redundant rules from `MimaExcludes` Key: SPARK-47774 URL: https://issues.apache.org/jira/browse/SPARK-47774 Project: Spark Issue Type: Sub

[jira] [Updated] (SPARK-47770) Fix `GenerateMIMAIgnore.isPackagePrivateModule` to return false instead of failing

2024-04-08 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47770: -- Fix Version/s: 3.5.2 3.4.3 >

[jira] [Updated] (SPARK-47770) Fix `GenerateMIMAIgnore.isPackagePrivateModule` to return false instead of failing

2024-04-08 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47770: -- Parent: SPARK-44111 Issue Type: Sub-task (was: Bug) >

[jira] [Assigned] (SPARK-47770) Fix `GenerateMIMAIgnore.isPackagePrivateModule` to return false instead of failing

2024-04-08 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47770: - Assignee: Dongjoon Hyun > Fix `GenerateMIMAIgnore.isPackagePrivateModule` to ret

[jira] [Created] (SPARK-47770) Fix `GenerateMIMAIgnore.isPackagePrivateModule` to return false instead of failing

2024-04-08 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-47770: - Summary: Fix `GenerateMIMAIgnore.isPackagePrivateModule` to return false instead of failing Key: SPARK-47770 URL: https://issues.apache.org/jira/browse/SPARK-47770

[jira] [Resolved] (SPARK-47737) Bump PyArrow to 10.0.0

2024-04-08 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47737. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45892 [https

[jira] [Updated] (SPARK-47737) Bump PyArrow to 10.0.0

2024-04-08 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47737: -- Parent: SPARK-44111 Issue Type: Sub-task (was: Bug) > Bump PyArrow to 10.

[jira] [Assigned] (SPARK-47737) Bump PyArrow to 10.0.0

2024-04-08 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47737: - Assignee: Haejoon Lee > Bump PyArrow to 10.

[jira] [Assigned] (SPARK-47725) Set up the CI for pyspark-connect package

2024-04-08 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47725: - Assignee: Hyukjin Kwon > Set up the CI for pyspark-connect pack

[jira] [Resolved] (SPARK-47725) Set up the CI for pyspark-connect package

2024-04-08 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47725. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45870 [https

Re: Apache Spark 3.4.3 (?)

2024-04-08 Thread Dongjoon Hyun
Thank you, Holden, Mridul, Kent, Liang-Chi, Mich, Jungtaek. I added `Target Version: 3.4.3` to SPARK-47318 and am going to continue to prepare for RC1 (April 15th). Dongjoon. - To unsubscribe e-mail:

[jira] [Commented] (SPARK-47318) AuthEngine key exchange needs additional KDF round

2024-04-08 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17834987#comment-17834987 ] Dongjoon Hyun commented on SPARK-47318: --- I added a target version (3.4.3) based on the dev mailing

[jira] [Updated] (SPARK-47318) AuthEngine key exchange needs additional KDF round

2024-04-08 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47318: -- Target Version/s: 3.4.3 > AuthEngine key exchange needs additional KDF ro

Apache Spark 3.4.3 (?)

2024-04-06 Thread Dongjoon Hyun
Hi, All. Apache Spark 3.4.2 tag was created on Nov 24th and `branch-3.4` has 85 commits including important security and correctness patches like SPARK-45580, SPARK-46092, SPARK-46466, SPARK-46794, and SPARK-46862. https://github.com/apache/spark/releases/tag/v3.4.2 $ git log --oneline

[jira] [Assigned] (SPARK-47709) Upgrade tink to 1.13.0

2024-04-06 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47709: - Assignee: Yang Jie > Upgrade tink to 1.1

[jira] [Resolved] (SPARK-47709) Upgrade tink to 1.13.0

2024-04-06 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47709. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45843 [https

[jira] [Assigned] (SPARK-47727) Make SparkConf to root level to for both SparkSession and SparkContext

2024-04-06 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47727: - Assignee: Hyukjin Kwon > Make SparkConf to root level to for both SparkSess

[jira] [Resolved] (SPARK-47727) Make SparkConf to root level to for both SparkSession and SparkContext

2024-04-06 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47727. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45873 [https

[jira] [Updated] (SPARK-47719) Change default of spark.sql.legacy.timeParserPolicy from EXCEPTION to CORRECTED

2024-04-06 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47719: -- Parent: SPARK-44111 Issue Type: Sub-task (was: Improvement) > Change defa

[jira] [Updated] (SPARK-45445) Upgrade snappy to 1.1.10.5

2024-04-05 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45445: -- Fix Version/s: 3.4.3 > Upgrade snappy to 1.1.1

[jira] [Updated] (SPARK-45445) Upgrade snappy to 1.1.10.5

2024-04-05 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45445: -- Fix Version/s: 3.5.2 > Upgrade snappy to 1.1.1

[jira] [Updated] (SPARK-46411) Change to use bcprov/bcpkix-jdk18on for test

2024-04-05 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-46411: -- Fix Version/s: 3.4.3 > Change to use bcprov/bcpkix-jdk18on for t

[jira] [Updated] (SPARK-46411) Change to use bcprov/bcpkix-jdk18on for test

2024-04-05 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-46411: -- Fix Version/s: 3.5.2 > Change to use bcprov/bcpkix-jdk18on for t

[jira] [Updated] (SPARK-45445) Upgrade snappy to 1.1.10.5

2024-04-05 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45445: -- Parent: SPARK-47046 Issue Type: Sub-task (was: Improvement) > Upgrade sna

[jira] [Updated] (SPARK-45323) Upgrade snappy to 1.1.10.4

2024-04-05 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45323: -- Parent: SPARK-47046 Issue Type: Sub-task (was: Dependency upgrade) > Upgrade sna

[jira] [Updated] (SPARK-45172) Upgrade commons-compress.version to 1.24.0

2024-04-05 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45172: -- Parent: SPARK-47046 Issue Type: Sub-task (was: Improvement) > Upgrade comm

[jira] [Updated] (SPARK-44393) Upgrade H2 from 2.1.214 to 2.2.220

2024-04-05 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-44393: -- Fix Version/s: 3.4.3 > Upgrade H2 from 2.1.214 to 2.2.

[jira] [Updated] (SPARK-44279) Upgrade optionator to ^0.9.3

2024-04-05 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-44279: -- Fix Version/s: 3.5.0 > Upgrade optionator to ^0.

[jira] [Resolved] (SPARK-47735) Make pyspark.testing.connectutils compatible with pyspark-connect

2024-04-05 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47735. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45887 [https

[jira] [Assigned] (SPARK-47735) Make pyspark.testing.connectutils compatible with pyspark-connect

2024-04-05 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47735: - Assignee: Hyukjin Kwon > Make pyspark.testing.connectutils compatible with pysp

[jira] [Resolved] (HADOOP-19141) Update VectorIO default values consistently

2024-04-04 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved HADOOP-19141. Fix Version/s: 3.3.7 3.4.1 Resolution: Fixed > Update Vecto

[jira] [Assigned] (HADOOP-19141) Update VectorIO default values consistently

2024-04-04 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned HADOOP-19141: -- Assignee: Dongjoon Hyun > Update VectorIO default values consisten

[jira] [Resolved] (HADOOP-19141) Update VectorIO default values consistently

2024-04-04 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-19141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved HADOOP-19141. Fix Version/s: 3.3.7 3.4.1 Resolution: Fixed > Update Vecto

[jira] [Resolved] (SPARK-47728) Document G1 Concurrent GC metrics

2024-04-04 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47728. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45874 [https

[jira] [Assigned] (SPARK-47728) Document G1 Concurrent GC metrics

2024-04-04 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47728: - Assignee: Luca Canali > Document G1 Concurrent GC metr

[jira] [Resolved] (SPARK-47729) Get the proper default port for pyspark-connect testcases

2024-04-04 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47729. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45875 [https

[jira] [Assigned] (SPARK-47720) Update `spark.speculation.multiplier` to 3 and `spark.speculation.quantile` to 0.9

2024-04-03 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47720: - Assignee: Dongjoon Hyun > Update `spark.speculation.multiplier` t

[jira] [Resolved] (SPARK-47720) Update `spark.speculation.multiplier` to 3 and `spark.speculation.quantile` to 0.9

2024-04-03 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47720. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45858 [https

[jira] [Assigned] (SPARK-47710) Postgres: Document Mapping Spark SQL Data Types from PostgreSQL

2024-04-03 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47710: - Assignee: Kent Yao > Postgres: Document Mapping Spark SQL Data Types from Postgre

[jira] [Resolved] (SPARK-47710) Postgres: Document Mapping Spark SQL Data Types from PostgreSQL

2024-04-03 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47710. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45845 [https

[jira] [Updated] (SPARK-47714) Set spark.sql.legacy.timeParserPolicy to CORRECTED by default

2024-04-03 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47714: -- Summary: Set spark.sql.legacy.timeParserPolicy to CORRECTED by default (was: set

[jira] [Resolved] (SPARK-47711) Parameterize JDBC Drivers for docker integration tests

2024-04-03 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47711. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45847 [https

[jira] [Created] (HADOOP-19141) Update VectorIO default values consistently

2024-04-03 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created HADOOP-19141: -- Summary: Update VectorIO default values consistently Key: HADOOP-19141 URL: https://issues.apache.org/jira/browse/HADOOP-19141 Project: Hadoop Common

[jira] [Created] (HADOOP-19141) Update VectorIO default values consistently

2024-04-03 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created HADOOP-19141: -- Summary: Update VectorIO default values consistently Key: HADOOP-19141 URL: https://issues.apache.org/jira/browse/HADOOP-19141 Project: Hadoop Common

[jira] [Resolved] (SPARK-47691) Postgres: Support multi dimensional array on the write side

2024-04-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47691. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45815 [https

[jira] [Assigned] (SPARK-47691) Postgres: Support multi dimensional array on the write side

2024-04-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47691: - Assignee: Kent Yao > Postgres: Support multi dimensional array on the write s

[jira] [Resolved] (ORC-1676) Use Hive 4.0.0 in benchmark

2024-04-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/ORC-1676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved ORC-1676. Fix Version/s: 2.0.1 2.1.0 Resolution: Fixed Issue resolved by pull

[jira] [Resolved] (SPARK-47697) Add scala style check for invalid MDC usage

2024-04-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47697. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45823 [https

[jira] [Assigned] (ORC-1676) Use Hive 4.0.0 in benchmark

2024-04-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/ORC-1676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned ORC-1676: -- Assignee: Dongjoon Hyun > Use Hive 4.0.0 in benchm

[jira] [Created] (ORC-1676) Use Hive 4.0.0 in benchmark

2024-04-02 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created ORC-1676: -- Summary: Use Hive 4.0.0 in benchmark Key: ORC-1676 URL: https://issues.apache.org/jira/browse/ORC-1676 Project: ORC Issue Type: Test Components: Java

[jira] [Commented] (HADOOP-18830) S3A: Cut S3 Select

2024-04-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/HADOOP-18830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17833281#comment-17833281 ] Dongjoon Hyun commented on HADOOP-18830: Hi, can we remove `3.4.1` from the `Fixed Version

[jira] [Updated] (SPARK-46793) Revert S3A endpoint fixup logic of SPARK-35878

2024-04-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-46793: -- Parent: (was: SPARK-44124) Issue Type: Bug (was: Sub-task) > Revert S3A endpo

[jira] [Created] (SPARK-47695) Update AWS SDK v2 to 2.24.6

2024-04-02 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-47695: - Summary: Update AWS SDK v2 to 2.24.6 Key: SPARK-47695 URL: https://issues.apache.org/jira/browse/SPARK-47695 Project: Spark Issue Type: Sub-task

[jira] [Commented] (ORC-1674) Remove patch file from vcpkg

2024-04-02 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/ORC-1674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17833222#comment-17833222 ] Dongjoon Hyun commented on ORC-1674: Thank you! > Remove patch file from vc

<    2   3   4   5   6   7   8   9   10   11   >