[jira] [Commented] (SPARK-49442) Complete Metadata requests on each micro batch causing Kafka issues

2024-08-28 Thread vipin Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17877266#comment-17877266 ] vipin Kumar commented on SPARK-49442: - [~kabhwan]  we applied *metadata.max.age.ms*

[jira] [Comment Edited] (SPARK-49442) Complete Metadata requests on each micro batch causing Kafka issues

2024-08-28 Thread vipin Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17877266#comment-17877266 ] vipin Kumar edited comment on SPARK-49442 at 8/28/24 7:29 AM:

[jira] [Comment Edited] (SPARK-49442) Complete Metadata requests on each micro batch causing Kafka issues

2024-08-28 Thread vipin Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17877266#comment-17877266 ] vipin Kumar edited comment on SPARK-49442 at 8/28/24 7:29 AM:

[jira] [Assigned] (SPARK-49439) Fix the pretty name of the `FromProtobuf` & `ToProtobuf` expression

2024-08-28 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-49439: Assignee: BingKun Pan > Fix the pretty name of the `FromProtobuf` & `ToProtobuf` expression > --

[jira] [Resolved] (SPARK-49439) Fix the pretty name of the `FromProtobuf` & `ToProtobuf` expression

2024-08-28 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-49439. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47901 [https://github.com

[jira] [Resolved] (SPARK-49438) Fix the pretty name of the `FromAvro` & `ToAvro` expression

2024-08-28 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-49438. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47900 [https://github.com

[jira] [Assigned] (SPARK-49438) Fix the pretty name of the `FromAvro` & `ToAvro` expression

2024-08-28 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-49438: Assignee: BingKun Pan > Fix the pretty name of the `FromAvro` & `ToAvro` expression > --

[jira] [Assigned] (SPARK-49383) Support Transpose DataFrame API

2024-08-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-49383: -- Assignee: Apache Spark > Support Transpose DataFrame API > --

[jira] [Created] (SPARK-49443) Implement to_variant_object expression and make schema_of_variant expressions print OBJECT for for Variant Objects

2024-08-28 Thread Harsh Motwani (Jira)
Harsh Motwani created SPARK-49443: - Summary: Implement to_variant_object expression and make schema_of_variant expressions print OBJECT for for Variant Objects Key: SPARK-49443 URL: https://issues.apache.org/jira/

[jira] [Updated] (SPARK-49443) Implement to_variant_object expression and make schema_of_variant expressions print OBJECT for for Variant Objects

2024-08-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-49443: --- Labels: pull-request-available (was: ) > Implement to_variant_object expression and make sc

[jira] [Assigned] (SPARK-49443) Implement to_variant_object expression and make schema_of_variant expressions print OBJECT for for Variant Objects

2024-08-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-49443: -- Assignee: Apache Spark > Implement to_variant_object expression and make schema_of_va

[jira] [Assigned] (SPARK-49443) Implement to_variant_object expression and make schema_of_variant expressions print OBJECT for for Variant Objects

2024-08-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-49443: -- Assignee: (was: Apache Spark) > Implement to_variant_object expression and make s

[jira] [Assigned] (SPARK-49443) Implement to_variant_object expression and make schema_of_variant expressions print OBJECT for for Variant Objects

2024-08-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-49443: -- Assignee: (was: Apache Spark) > Implement to_variant_object expression and make s

[jira] [Assigned] (SPARK-49443) Implement to_variant_object expression and make schema_of_variant expressions print OBJECT for for Variant Objects

2024-08-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-49443: -- Assignee: Apache Spark > Implement to_variant_object expression and make schema_of_va

[jira] [Assigned] (SPARK-49410) Update collation benchmarks

2024-08-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-49410: -- Assignee: Apache Spark > Update collation benchmarks > --- >

[jira] [Assigned] (SPARK-49410) Update collation benchmarks

2024-08-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-49410: -- Assignee: (was: Apache Spark) > Update collation benchmarks > ---

[jira] [Assigned] (SPARK-49443) Implement to_variant_object expression and make schema_of_variant expressions print OBJECT for for Variant Objects

2024-08-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-49443: -- Assignee: Apache Spark > Implement to_variant_object expression and make schema_of_va

[jira] [Assigned] (SPARK-49443) Implement to_variant_object expression and make schema_of_variant expressions print OBJECT for for Variant Objects

2024-08-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-49443: -- Assignee: (was: Apache Spark) > Implement to_variant_object expression and make s

[jira] [Resolved] (SPARK-43242) diagnoseCorruption should not throw Unexpected type of BlockId for ShuffleBlockBatchId

2024-08-28 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-43242. - Fix Version/s: 4.0.0 Assignee: Zhang Liang Resolution: Fixed > diagnoseCorruptio

[jira] [Created] (SPARK-49444) Univocity parser handles ArrayIndexOutOfBounds exception

2024-08-28 Thread Jira
Vladan Vasić created SPARK-49444: Summary: Univocity parser handles ArrayIndexOutOfBounds exception Key: SPARK-49444 URL: https://issues.apache.org/jira/browse/SPARK-49444 Project: Spark Issu

[jira] [Assigned] (SPARK-49119) Fix the inconsistency of syntax `show columns` between v1 and v2

2024-08-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-49119: -- Assignee: Apache Spark > Fix the inconsistency of syntax `show columns` between v1 an

[jira] [Updated] (SPARK-49444) Univocity parser handles ArrayIndexOutOfBounds exception

2024-08-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-49444: --- Labels: pull-request-available (was: ) > Univocity parser handles ArrayIndexOutOfBounds exc

[jira] [Assigned] (SPARK-49444) Univocity parser handles ArrayIndexOutOfBounds exception

2024-08-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-49444: -- Assignee: Apache Spark > Univocity parser handles ArrayIndexOutOfBounds exception > -

[jira] [Assigned] (SPARK-49444) Univocity parser handles ArrayIndexOutOfBounds exception

2024-08-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-49444: -- Assignee: (was: Apache Spark) > Univocity parser handles ArrayIndexOutOfBounds ex

[jira] [Commented] (SPARK-49444) Univocity parser handles ArrayIndexOutOfBounds exception

2024-08-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17877301#comment-17877301 ] ASF GitHub Bot commented on SPARK-49444: User 'vladanvasi-db' has created a pull

[jira] [Commented] (SPARK-49444) Univocity parser handles ArrayIndexOutOfBounds exception

2024-08-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17877302#comment-17877302 ] ASF GitHub Bot commented on SPARK-49444: User 'vladanvasi-db' has created a pull

[jira] [Assigned] (SPARK-49444) Univocity parser handles ArrayIndexOutOfBounds exception

2024-08-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-49444: -- Assignee: (was: Apache Spark) > Univocity parser handles ArrayIndexOutOfBounds ex

[jira] [Assigned] (SPARK-49444) Univocity parser handles ArrayIndexOutOfBounds exception

2024-08-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-49444: -- Assignee: Apache Spark > Univocity parser handles ArrayIndexOutOfBounds exception > -

[jira] [Commented] (SPARK-49442) Complete Metadata requests on each micro batch causing Kafka issues

2024-08-28 Thread vipin Kumar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17877314#comment-17877314 ] vipin Kumar commented on SPARK-49442: - Hi [~kabhwan]   *I don't know whether the ma

[jira] [Updated] (SPARK-46037) When Left Join build Left, ShuffledHashJoinExec may result in incorrect results

2024-08-28 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-46037: - Priority: Blocker (was: Minor) > When Left Join build Left, ShuffledHashJoinExec may result in incorrec

[jira] [Created] (SPARK-49445) Support show tooltip in the progress bar of UI

2024-08-28 Thread dzcxzl (Jira)
dzcxzl created SPARK-49445: -- Summary: Support show tooltip in the progress bar of UI Key: SPARK-49445 URL: https://issues.apache.org/jira/browse/SPARK-49445 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-49445) Support show tooltip in the progress bar of UI

2024-08-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-49445: --- Labels: pull-request-available (was: ) > Support show tooltip in the progress bar of UI > -

[jira] [Commented] (SPARK-49409) CONNECT_SESSION_PLAN_CACHE_SIZE is too small for certain programming patterns

2024-08-28 Thread Changgyoo Park (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17877366#comment-17877366 ] Changgyoo Park commented on SPARK-49409: Yes, because there is "a" case where 5

[jira] [Resolved] (SPARK-49029) Create a shared interface for Dataset

2024-08-28 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-49029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hövell resolved SPARK-49029. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47882 [

[jira] [Commented] (SPARK-34638) Spark SQL reads unnecessary nested fields (another type of pruning case)

2024-08-28 Thread Jiri Humpolicek (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17877372#comment-17877372 ] Jiri Humpolicek commented on SPARK-34638: - Hi, I currently tested similar exampl

[jira] [Created] (SPARK-49446) Upgrade jetty to 11.0.23

2024-08-28 Thread Yang Jie (Jira)
Yang Jie created SPARK-49446: Summary: Upgrade jetty to 11.0.23 Key: SPARK-49446 URL: https://issues.apache.org/jira/browse/SPARK-49446 Project: Spark Issue Type: Improvement Components

[jira] [Updated] (SPARK-49446) Upgrade jetty to 11.0.23

2024-08-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-49446: --- Labels: pull-request-available (was: ) > Upgrade jetty to 11.0.23 > ---

[jira] [Commented] (SPARK-49442) Complete Metadata requests on each micro batch causing Kafka issues

2024-08-28 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17877397#comment-17877397 ] Jungtaek Lim commented on SPARK-49442: -- OK, that's unrelated. We haven't got any re

[jira] [Resolved] (SPARK-45745) Extremely slow execution of sum of columns in Spark 3.4.1

2024-08-28 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins resolved SPARK-45745. --- Resolution: Duplicate > Extremely slow execution of sum of columns in Spark 3.4.1 >

[jira] [Assigned] (SPARK-49313) Upgrade `DB2` & `MySQL` & `Postgres` & `Mariadb` docker image version

2024-08-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-49313: - Assignee: BingKun Pan > Upgrade `DB2` & `MySQL` & `Postgres` & `Mariadb` docker image v

[jira] [Resolved] (SPARK-49313) Upgrade `DB2` & `MySQL` & `Postgres` & `Mariadb` docker image version

2024-08-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-49313. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47809 [https://

[jira] [Resolved] (SPARK-49399) Add examples for different Spark image types

2024-08-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-49399. --- Fix Version/s: kubernetes-operator-0.1.0 Resolution: Fixed Issue resolved by pull req

[jira] [Assigned] (SPARK-49399) Add examples for different Spark image types

2024-08-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-49399: - Assignee: Zhou JIANG > Add examples for different Spark image types > -

[jira] [Updated] (SPARK-49399) Add `pi-scala.yaml` and `pyspark-pi.yaml`

2024-08-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-49399: -- Summary: Add `pi-scala.yaml` and `pyspark-pi.yaml` (was: Add examples for different Spark ima

[jira] [Updated] (SPARK-49447) Fix `spark.kubernetes.allocation.batch.delay` to prevent small values less than 100

2024-08-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-49447: -- Description: The default value is `1s` (=1000). Usually, a small value like `1` happens when u

[jira] [Updated] (SPARK-49447) Fix `spark.kubernetes.allocation.batch.delay` to prevent small values less than 100

2024-08-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-49447: --- Labels: pull-request-available (was: ) > Fix `spark.kubernetes.allocation.batch.delay` to p

[jira] [Assigned] (SPARK-49447) Fix `spark.kubernetes.allocation.batch.delay` to prevent small values less than 100

2024-08-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-49447: - Assignee: Dongjoon Hyun > Fix `spark.kubernetes.allocation.batch.delay` to prevent smal

[jira] [Resolved] (SPARK-48781) Add Catalog APIs for loading stored procedures

2024-08-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48781. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47190 [https://

[jira] [Assigned] (SPARK-48781) Add Catalog APIs for loading stored procedures

2024-08-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-48781: - Assignee: Anton Okolnychyi > Add Catalog APIs for loading stored procedures > -

[jira] [Updated] (SPARK-41262) Enable canChangeCachedPlanOutputPartitioning by default

2024-08-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-41262: --- Labels: pull-request-available (was: ) > Enable canChangeCachedPlanOutputPartitioning by de

[jira] [Resolved] (SPARK-48400) Promote `PrometheusServlet` to `DeveloperApi`

2024-08-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-48400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-48400. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46716 [https://

[jira] [Resolved] (SPARK-45923) Spark Kubernetes Operator

2024-08-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-45923. --- Fix Version/s: 4.0.0 Resolution: Fixed > Spark Kubernetes Operator >

[jira] [Updated] (SPARK-45923) Spark Kubernetes Operator

2024-08-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45923: -- Labels: SPIP releasenotes (was: SPIP pull-request-available) > Spark Kubernetes Operator > --

[jira] [Updated] (SPARK-49448) Spark Connect ExecuteThreadRunner promise will always complete with success.

2024-08-28 Thread LIU (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] LIU updated SPARK-49448: Description: {code:java} //代码占位符 {code} private class ExecutionThread(onCompletionPromise: Promise[Unit]) extends

[jira] [Created] (SPARK-49448) Spark Connect ExecuteThreadRunner promise will always complete with success.

2024-08-28 Thread LIU (Jira)
LIU created SPARK-49448: --- Summary: Spark Connect ExecuteThreadRunner promise will always complete with success. Key: SPARK-49448 URL: https://issues.apache.org/jira/browse/SPARK-49448 Project: Spark I

[jira] [Updated] (SPARK-49448) Spark Connect ExecuteThreadRunner promise will always complete with success.

2024-08-28 Thread LIU (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] LIU updated SPARK-49448: Description: {code:java} private class ExecutionThread(onCompletionPromise: Promise[Unit]) extends Thread(s"S

[jira] [Updated] (SPARK-49449) Remove string and binary from metadata in spec

2024-08-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-49449: --- Labels: pull-request-available (was: ) > Remove string and binary from metadata in spec > -

[jira] [Updated] (SPARK-49450) Improve normalised collation names

2024-08-28 Thread Mihailo Milosevic (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mihailo Milosevic updated SPARK-49450: -- Parent: (was: SPARK-46830) Issue Type: Improvement (was: Sub-task) > Impr

[jira] [Updated] (SPARK-49421) Create a shared RelationalGroupedDataset interface

2024-08-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-49421: --- Labels: pull-request-available (was: ) > Create a shared RelationalGroupedDataset interface

[jira] [Assigned] (SPARK-49421) Create a shared RelationalGroupedDataset interface

2024-08-28 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-49421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hövell reassigned SPARK-49421: - Assignee: Herman van Hövell > Create a shared RelationalGroupedDataset interfac

[jira] [Assigned] (SPARK-49419) Create a shared DataFrameStatFunctions interface

2024-08-28 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-49419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hövell reassigned SPARK-49419: - Assignee: Herman van Hövell > Create a shared DataFrameStatFunctions interface

[jira] [Updated] (SPARK-49450) Improve normalised collation names

2024-08-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-49450: --- Labels: pull-request-available (was: ) > Improve normalised collation names > -

[jira] [Updated] (SPARK-49422) Create a shared KeyValueGroupedDataset interface

2024-08-28 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-49422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hövell updated SPARK-49422: -- Description: This dhpou > Create a shared KeyValueGroupedDataset interface > -

[jira] [Updated] (SPARK-49422) Create a shared KeyValueGroupedDataset interface

2024-08-28 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-49422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hövell updated SPARK-49422: -- Environment: (was: Not sure if we should do this. Connect and Classic have differe

[jira] [Assigned] (SPARK-49422) Create a shared KeyValueGroupedDataset interface

2024-08-28 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-49422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hövell reassigned SPARK-49422: - Assignee: Herman van Hövell > Create a shared KeyValueGroupedDataset interface

[jira] [Updated] (SPARK-49422) Create a shared KeyValueGroupedDataset interface

2024-08-28 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-49422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hövell updated SPARK-49422: -- Description: This should also implement RelationalGroupedDataset.as[K: Encoder, T: Enc

[jira] [Created] (SPARK-49451) Allow duplicate keys in parse_json.

2024-08-28 Thread Chenhao Li (Jira)
Chenhao Li created SPARK-49451: -- Summary: Allow duplicate keys in parse_json. Key: SPARK-49451 URL: https://issues.apache.org/jira/browse/SPARK-49451 Project: Spark Issue Type: Sub-task

[jira] [Assigned] (SPARK-49423) Consolidate Observation into a single class in sql/api

2024-08-28 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-49423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hövell reassigned SPARK-49423: - Assignee: Herman van Hövell > Consolidate Observation into a single class in sq

[jira] [Updated] (SPARK-49451) Allow duplicate keys in parse_json.

2024-08-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-49451: --- Labels: pull-request-available (was: ) > Allow duplicate keys in parse_json. >

[jira] [Updated] (SPARK-49423) Consolidate Observation into a single class in sql/api

2024-08-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-49423: --- Labels: pull-request-available (was: ) > Consolidate Observation into a single class in sql

[jira] [Updated] (SPARK-49425) Create a shared DataFrameWriter interface

2024-08-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-49425: --- Labels: pull-request-available (was: ) > Create a shared DataFrameWriter interface > --

[jira] [Resolved] (SPARK-49447) Fix `spark.kubernetes.allocation.batch.delay` to prevent small values less than 100

2024-08-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-49447. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47913 [https://

[jira] [Updated] (SPARK-46995) Allow AQE coalesce final stage in SQL cached plan

2024-08-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46995: --- Labels: pull-request-available (was: ) > Allow AQE coalesce final stage in SQL cached plan

[jira] [Created] (SPARK-49453) spark-kubernetes-operator-dynamic-configuration ConfigMap Data Overriding Failure

2024-08-28 Thread Qi Tan (Jira)
Qi Tan created SPARK-49453: -- Summary: spark-kubernetes-operator-dynamic-configuration ConfigMap Data Overriding Failure Key: SPARK-49453 URL: https://issues.apache.org/jira/browse/SPARK-49453 Project: Spark

[jira] [Created] (SPARK-49454) Avoid double normalization in the cache process

2024-08-28 Thread Xinyi Yu (Jira)
Xinyi Yu created SPARK-49454: Summary: Avoid double normalization in the cache process Key: SPARK-49454 URL: https://issues.apache.org/jira/browse/SPARK-49454 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-46037) When Left Join build Left, ShuffledHashJoinExec may result in incorrect results

2024-08-28 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-46037. - Fix Version/s: 4.0.0 3.5.3 Resolution: Fixed Issue resolved by pull re

[jira] [Assigned] (SPARK-46037) When Left Join build Left, ShuffledHashJoinExec may result in incorrect results

2024-08-28 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-46037: --- Assignee: mcdull_zhang > When Left Join build Left, ShuffledHashJoinExec may result in inco

[jira] [Resolved] (SPARK-49446) Upgrade jetty to 11.0.23

2024-08-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-49446. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 47912 [https://

[jira] [Updated] (SPARK-42879) Spark SQL reads unnecessary nested fields

2024-08-28 Thread Jiri Humpolicek (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiri Humpolicek updated SPARK-42879: Affects Version/s: 3.5.2 > Spark SQL reads unnecessary nested fields > ---

[jira] [Created] (SPARK-49455) Refactor `StagingInMemoryTableCatalog` to override the non-deprecated functions

2024-08-28 Thread Yang Jie (Jira)
Yang Jie created SPARK-49455: Summary: Refactor `StagingInMemoryTableCatalog` to override the non-deprecated functions Key: SPARK-49455 URL: https://issues.apache.org/jira/browse/SPARK-49455 Project: Spar

[jira] [Updated] (SPARK-49455) Refactor `StagingInMemoryTableCatalog` to override the non-deprecated functions

2024-08-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-49455: --- Labels: pull-request-available (was: ) > Refactor `StagingInMemoryTableCatalog` to override

[jira] [Created] (SPARK-49456) Spark website doesn't properly scroll to hash links

2024-08-28 Thread Neil Ramaswamy (Jira)
Neil Ramaswamy created SPARK-49456: -- Summary: Spark website doesn't properly scroll to hash links Key: SPARK-49456 URL: https://issues.apache.org/jira/browse/SPARK-49456 Project: Spark Issu

[jira] [Commented] (SPARK-34638) Spark SQL reads unnecessary nested fields (another type of pruning case)

2024-08-28 Thread Jiri Humpolicek (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17877595#comment-17877595 ] Jiri Humpolicek commented on SPARK-34638: - [~viirya] Do you think it would be po

[jira] [Comment Edited] (SPARK-34638) Spark SQL reads unnecessary nested fields (another type of pruning case)

2024-08-28 Thread Jiri Humpolicek (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17877595#comment-17877595 ] Jiri Humpolicek edited comment on SPARK-34638 at 8/29/24 6:03 AM:

[jira] [Updated] (SPARK-49456) Spark website doesn't properly scroll to hash links

2024-08-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-49456: --- Labels: pull-request-available (was: ) > Spark website doesn't properly scroll to hash link

[jira] [Created] (SPARK-49457) Remove uncommon curl option --retry-all-errors

2024-08-28 Thread Cheng Pan (Jira)
Cheng Pan created SPARK-49457: - Summary: Remove uncommon curl option --retry-all-errors Key: SPARK-49457 URL: https://issues.apache.org/jira/browse/SPARK-49457 Project: Spark Issue Type: Improvem

[jira] [Updated] (SPARK-49457) Remove uncommon curl option --retry-all-errors

2024-08-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-49457: --- Labels: pull-request-available (was: ) > Remove uncommon curl option --retry-all-errors > -

[jira] [Updated] (SPARK-49259) Size based partition creation during kafka read

2024-08-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-49259: --- Labels: pull-request-available (was: ) > Size based partition creation during kafka read >

[jira] [Updated] (SPARK-49453) spark-kubernetes-operator-dynamic-configuration ConfigMap Data Overriding Failure

2024-08-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-49453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-49453: --- Labels: pull-request-available (was: ) > spark-kubernetes-operator-dynamic-configuration Co