[jira] [Updated] (SPARK-45502) Upgrade Kafka to 3.6.0

2023-10-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45502: --- Labels: pull-request-available (was: ) > Upgrade Kafka to 3.6.0 > -- >

[jira] [Assigned] (SPARK-45442) Refine docstring of `DataFrame.show`

2023-10-11 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-45442: - Assignee: Allison Wang > Refine docstring of `DataFrame.show` >

[jira] [Resolved] (SPARK-45442) Refine docstring of `DataFrame.show`

2023-10-11 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-45442. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43252

[jira] [Updated] (SPARK-45510) Replace `scala.collection.generic.Growable` to `scala.collection.mutable.Growable`

2023-10-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45510: --- Labels: pull-request-available (was: ) > Replace `scala.collection.generic.Growable` to >

[jira] [Created] (SPARK-45510) Replace `scala.collection.generic.Growable` to `scala.collection.mutable.Growable`

2023-10-11 Thread Jia Fan (Jira)
Jia Fan created SPARK-45510: --- Summary: Replace `scala.collection.generic.Growable` to `scala.collection.mutable.Growable` Key: SPARK-45510 URL: https://issues.apache.org/jira/browse/SPARK-45510 Project:

[jira] [Resolved] (SPARK-45402) Add API for 'analyze' method to return a buffer to be consumed on each class creation

2023-10-11 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45402?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-45402. --- Fix Version/s: 4.0.0 Assignee: Daniel Resolution: Fixed Issue resolved by

[jira] [Updated] (SPARK-45508) Add "--add-opens=java.base/jdk.internal.ref=ALL-UNNAMED" so Platform can access cleaner on Java 9+

2023-10-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45508: --- Labels: pull-request-available (was: ) > Add

[jira] [Updated] (SPARK-45508) Add "--add-opens=java.base/jdk.internal.ref=ALL-UNNAMED" so Platform can access cleaner on Java 9+

2023-10-11 Thread Josh Rosen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-45508: --- Description: We need to add `--add-opens=java.base/jdk.internal.ref=ALL-UNNAMED` to our JVM options

[jira] [Updated] (SPARK-45508) Add "--add-opens=java.base/jdk.internal.ref=ALL-UNNAMED" so Platform can access cleaner on Java 9+

2023-10-11 Thread Josh Rosen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-45508: --- Summary: Add "--add-opens=java.base/jdk.internal.ref=ALL-UNNAMED" so Platform can access cleaner on

[jira] [Updated] (SPARK-45508) Add "--add-opens=java.base/jdk.internal.ref=ALL-UNNAMED" so Platform can access cleaner on Java 11+

2023-10-11 Thread Josh Rosen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-45508: --- Summary: Add "--add-opens=java.base/jdk.internal.ref=ALL-UNNAMED" so Platform can access cleaner on

[jira] [Updated] (SPARK-45508) Add "--add-opens=java.base/jdk.internal.ref=ALL-UNNAMED" so Platform can access cleaner on Java 11+

2023-10-11 Thread Josh Rosen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-45508: --- Description: We need to update the    ``` val f =

[jira] [Updated] (SPARK-45508) Add "--add-opens=java.base/jdk.internal.ref=ALL-UNNAMED" so Platform can access cleaner on Java 9+

2023-10-11 Thread Josh Rosen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-45508: --- Summary: Add "--add-opens=java.base/jdk.internal.ref=ALL-UNNAMED" so Platform can access cleaner on

[jira] [Updated] (SPARK-45509) Investigate the behavior difference in self-join

2023-10-11 Thread Allison Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allison Wang updated SPARK-45509: - Description: SPARK-45220 discovers a behavior difference for a self-join scenario between

[jira] [Updated] (SPARK-45509) Investigate the behavior difference in self-join

2023-10-11 Thread Allison Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allison Wang updated SPARK-45509: - Description: SPARK-45220 discovers a behavior difference for a self-join scenario between

[jira] [Updated] (SPARK-45509) Investigate the behavior difference in self-join

2023-10-11 Thread Allison Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allison Wang updated SPARK-45509: - Description: SPARK-45220 discovers a behavior difference for a self-join scenario between

[jira] [Updated] (SPARK-45508) org.apache.spark.unsafe.Platform uses wrong cleaner class name in JDK 9.b110+

2023-10-11 Thread Josh Rosen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-45508: --- Description: In JDK >= 9.b110, the code at

[jira] [Updated] (SPARK-45508) org.apache.spark.unsafe.Platform uses wrong cleaner class name in JDK 9.b110+

2023-10-11 Thread Josh Rosen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-45508: --- Description: In JDK >= 9.b110, the code at

[jira] [Created] (SPARK-45509) Investigate the behavior difference in self-join

2023-10-11 Thread Allison Wang (Jira)
Allison Wang created SPARK-45509: Summary: Investigate the behavior difference in self-join Key: SPARK-45509 URL: https://issues.apache.org/jira/browse/SPARK-45509 Project: Spark Issue Type:

[jira] [Created] (SPARK-45508) org.apache.spark.unsafe.Platform uses wrong cleaner class name in JDK 11+

2023-10-11 Thread Josh Rosen (Jira)
Josh Rosen created SPARK-45508: -- Summary: org.apache.spark.unsafe.Platform uses wrong cleaner class name in JDK 11+ Key: SPARK-45508 URL: https://issues.apache.org/jira/browse/SPARK-45508 Project: Spark

[jira] [Updated] (SPARK-45487) Replace: _LEGACY_ERROR_TEMP_3007

2023-10-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45487: --- Labels: pull-request-available (was: ) > Replace: _LEGACY_ERROR_TEMP_3007 >

[jira] [Updated] (SPARK-45507) Correctness bug in correlated scalar subqueries with COUNT aggregates

2023-10-11 Thread Andy Lam (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andy Lam updated SPARK-45507: - Description: {code:java}   create view if not exists t1(a1, a2) as values (0, 1), (1, 2); create view

[jira] [Updated] (SPARK-45507) Correctness bug in correlated scalar subqueries with COUNT aggregates

2023-10-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45507: --- Labels: pull-request-available (was: ) > Correctness bug in correlated scalar subqueries

[jira] [Created] (SPARK-45507) Correctness bug in correlated scalar subqueries with COUNT aggregates

2023-10-11 Thread Andy Lam (Jira)
Andy Lam created SPARK-45507: Summary: Correctness bug in correlated scalar subqueries with COUNT aggregates Key: SPARK-45507 URL: https://issues.apache.org/jira/browse/SPARK-45507 Project: Spark

[jira] [Created] (SPARK-45506) Support ivy URIs in SparkConnect addArtifact

2023-10-11 Thread Vsevolod Stepanov (Jira)
Vsevolod Stepanov created SPARK-45506: - Summary: Support ivy URIs in SparkConnect addArtifact Key: SPARK-45506 URL: https://issues.apache.org/jira/browse/SPARK-45506 Project: Spark Issue

[jira] [Resolved] (SPARK-45221) Refine docstring of `DataFrameReader.parquet`

2023-10-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-45221. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43301

[jira] [Assigned] (SPARK-45221) Refine docstring of `DataFrameReader.parquet`

2023-10-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-45221: Assignee: Allison Wang > Refine docstring of `DataFrameReader.parquet` >

[jira] [Updated] (SPARK-45505) Refactor analyzeInPython function to make it reusable

2023-10-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45505: --- Labels: pull-request-available (was: ) > Refactor analyzeInPython function to make it

[jira] [Updated] (SPARK-43396) Add config to control max ratio of decommissioning executors

2023-10-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-43396: --- Labels: pull-request-available (was: ) > Add config to control max ratio of

[jira] [Updated] (SPARK-42584) Improve output of Column.explain

2023-10-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-42584: --- Labels: pull-request-available (was: ) > Improve output of Column.explain >

[jira] [Created] (SPARK-45505) Refactor analyzeInPython function to make it reusable

2023-10-11 Thread Allison Wang (Jira)
Allison Wang created SPARK-45505: Summary: Refactor analyzeInPython function to make it reusable Key: SPARK-45505 URL: https://issues.apache.org/jira/browse/SPARK-45505 Project: Spark Issue

[jira] [Updated] (SPARK-45504) RocksDB State Store Should Lower RocksDB Background Thread CPU Priority

2023-10-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45504: --- Labels: pull-request-available (was: ) > RocksDB State Store Should Lower RocksDB

[jira] [Created] (SPARK-45504) RocksDB State Store Should Lower RocksDB Background Thread CPU Priority

2023-10-11 Thread Siying Dong (Jira)
Siying Dong created SPARK-45504: --- Summary: RocksDB State Store Should Lower RocksDB Background Thread CPU Priority Key: SPARK-45504 URL: https://issues.apache.org/jira/browse/SPARK-45504 Project: Spark

[jira] [Updated] (SPARK-45503) RocksDB State Store to Use LZ4 Compression

2023-10-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45503: --- Labels: pull-request-available (was: ) > RocksDB State Store to Use LZ4 Compression >

[jira] [Created] (SPARK-45503) RocksDB State Store to Use LZ4 Compression

2023-10-11 Thread Siying Dong (Jira)
Siying Dong created SPARK-45503: --- Summary: RocksDB State Store to Use LZ4 Compression Key: SPARK-45503 URL: https://issues.apache.org/jira/browse/SPARK-45503 Project: Spark Issue Type: Task

[jira] [Resolved] (SPARK-45490) Replace: _LEGACY_ERROR_TEMP_2151 with a proper error class

2023-10-11 Thread Serge Rielau (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Serge Rielau resolved SPARK-45490. -- Resolution: Cannot Reproduce Seems to have been implemented as: EXPRESSION_DECODING_FAILED >

[jira] [Resolved] (SPARK-44855) Small tweaks to attaching ExecuteGrpcResponseSender to ExecuteResponseObserver

2023-10-11 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-44855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hövell resolved SPARK-44855. --- Fix Version/s: 4.0.0 Assignee: Juliusz Sompolski Resolution: Fixed

[jira] [Commented] (SPARK-45502) Upgrade Kafka to 3.6.0

2023-10-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774147#comment-17774147 ] Dongjoon Hyun commented on SPARK-45502: --- Thank you for volunteering, [~dengziming] . BTW, Apache

[jira] [Resolved] (SPARK-45433) CSV/JSON schema inference when timestamps do not match specified timestampFormat with only one row on each partition report error

2023-10-11 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-45433. -- Fix Version/s: 3.5.1 4.0.0 Resolution: Fixed Issue resolved by pull request

[jira] [Assigned] (SPARK-45433) CSV/JSON schema inference when timestamps do not match specified timestampFormat with only one row on each partition report error

2023-10-11 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-45433: Assignee: Jia Fan > CSV/JSON schema inference when timestamps do not match specified >

[jira] [Resolved] (SPARK-45483) Correct the function groups in connect.functions

2023-10-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-45483. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43309

[jira] [Resolved] (SPARK-45499) Replace `Reference#isEnqueued` with `Reference#refersTo`

2023-10-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-45499. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43325

[jira] [Assigned] (SPARK-45499) Replace `Reference#isEnqueued` with `Reference#refersTo`

2023-10-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-45499: - Assignee: Yang Jie > Replace `Reference#isEnqueued` with `Reference#refersTo` >

[jira] [Resolved] (SPARK-42881) Codegen Support for get_json_object

2023-10-11 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-42881. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 40506

[jira] [Assigned] (SPARK-42881) Codegen Support for get_json_object

2023-10-11 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-42881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-42881: Assignee: BingKun Pan > Codegen Support for get_json_object >

[jira] [Resolved] (SPARK-45467) Replace `Proxy.getProxyClass()` with `Proxy.newProxyInstance().getClass`

2023-10-11 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-45467. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43291

[jira] [Assigned] (SPARK-45467) Replace `Proxy.getProxyClass()` with `Proxy.newProxyInstance().getClass`

2023-10-11 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-45467: Assignee: Yang Jie > Replace `Proxy.getProxyClass()` with

[jira] [Updated] (SPARK-45467) Replace `Proxy.getProxyClass()` with `Proxy.newProxyInstance().getClass`

2023-10-11 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-45467: - Priority: Minor (was: Major) > Replace `Proxy.getProxyClass()` with

[jira] [Resolved] (SPARK-45496) Fix the compilation warning related to other-pure-statement

2023-10-11 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-45496. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43312

[jira] [Updated] (SPARK-45496) Fix the compilation warning related to other-pure-statement

2023-10-11 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-45496: - Priority: Minor (was: Major) > Fix the compilation warning related to other-pure-statement >

[jira] [Resolved] (SPARK-43828) Add config to control whether close idle connection

2023-10-11 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-43828. -- Resolution: Won't Fix > Add config to control whether close idle connection >

[jira] [Assigned] (SPARK-45496) Fix the compilation warning related to other-pure-statement

2023-10-11 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-45496: Assignee: Yang Jie > Fix the compilation warning related to other-pure-statement >

[jira] [Updated] (SPARK-45496) Fix the compilation warning related to other-pure-statement

2023-10-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45496: --- Labels: pull-request-available (was: ) > Fix the compilation warning related to

[jira] [Resolved] (SPARK-45451) Make the default storage level of dataset cache configurable

2023-10-11 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-45451. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43259

[jira] [Assigned] (SPARK-45451) Make the default storage level of dataset cache configurable

2023-10-11 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-45451: --- Assignee: XiDuo You > Make the default storage level of dataset cache configurable >

[jira] [Resolved] (SPARK-45116) Add some comment for param of JdbcDialect createTable

2023-10-11 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-45116. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42799

[jira] [Assigned] (SPARK-45116) Add some comment for param of JdbcDialect createTable

2023-10-11 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-45116: Assignee: Jia Fan > Add some comment for param of JdbcDialect createTable >

[jira] [Resolved] (SPARK-45397) Add vector assembler feature transformer

2023-10-11 Thread Weichen Xu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weichen Xu resolved SPARK-45397. Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43199

[jira] [Assigned] (SPARK-45428) Add Matomo analytics to all released docs pages

2023-10-11 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-45428: - Assignee: BingKun Pan > Add Matomo analytics to all released docs pages >

[jira] [Assigned] (SPARK-45397) Add vector assembler feature transformer

2023-10-11 Thread Weichen Xu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weichen Xu reassigned SPARK-45397: -- Assignee: Weichen Xu > Add vector assembler feature transformer >

[jira] [Resolved] (SPARK-45466) VectorAssembler should validate the vector values

2023-10-11 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-45466. --- Resolution: Not A Problem > VectorAssembler should validate the vector values >

[jira] [Comment Edited] (SPARK-45201) NoClassDefFoundError: InternalFutureFailureAccess when compiling Spark 3.5.0

2023-10-11 Thread Sebastian Daberdaku (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17767622#comment-17767622 ] Sebastian Daberdaku edited comment on SPARK-45201 at 10/11/23 9:58 AM:

[jira] [Comment Edited] (SPARK-45201) NoClassDefFoundError: InternalFutureFailureAccess when compiling Spark 3.5.0

2023-10-11 Thread Sebastian Daberdaku (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17767622#comment-17767622 ] Sebastian Daberdaku edited comment on SPARK-45201 at 10/11/23 9:58 AM:

[jira] [Comment Edited] (SPARK-45201) NoClassDefFoundError: InternalFutureFailureAccess when compiling Spark 3.5.0

2023-10-11 Thread Sebastian Daberdaku (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17767622#comment-17767622 ] Sebastian Daberdaku edited comment on SPARK-45201 at 10/11/23 9:58 AM:

[jira] [Commented] (SPARK-45201) NoClassDefFoundError: InternalFutureFailureAccess when compiling Spark 3.5.0

2023-10-11 Thread xie shuiahu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773976#comment-17773976 ] xie shuiahu commented on SPARK-45201: - [~sdaberdaku] I alse have the same issue. I solved it by

[jira] [Resolved] (SPARK-45469) Replace `toIterator` with `iterator` for `IterableOnce`

2023-10-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-45469. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43295

[jira] [Commented] (SPARK-45502) Upgrade Kafka to 3.6.0

2023-10-11 Thread Deng Ziming (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773964#comment-17773964 ] Deng Ziming commented on SPARK-45502: - I'm trying this.  > Upgrade Kafka to 3.6.0 >

[jira] [Resolved] (SPARK-45500) Show the number of abnormally completed drivers in MasterPage

2023-10-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-45500. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43328

[jira] [Assigned] (SPARK-45500) Show the number of abnormally completed drivers in MasterPage

2023-10-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-45500: - Assignee: Dongjoon Hyun > Show the number of abnormally completed drivers in

[jira] [Comment Edited] (SPARK-44757) Vulnerabilities in Spark3.4

2023-10-11 Thread Laurenceau Julien (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773949#comment-17773949 ] Laurenceau Julien edited comment on SPARK-44757 at 10/11/23 8:14 AM: -

[jira] [Commented] (SPARK-44757) Vulnerabilities in Spark3.4

2023-10-11 Thread Laurenceau Julien (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773949#comment-17773949 ] Laurenceau Julien commented on SPARK-44757: --- Hi, In addition to this I'd like to add the

[jira] [Resolved] (SPARK-45480) Selectable SQL Plan

2023-10-11 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-45480. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43307

[jira] [Assigned] (SPARK-45480) Selectable SQL Plan

2023-10-11 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao reassigned SPARK-45480: Assignee: Kent Yao > Selectable SQL Plan > --- > > Key:

[jira] [Assigned] (SPARK-45497) Add a symbolic link file `spark-examples.jar` in K8s Docker images

2023-10-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-45497: - Assignee: Dongjoon Hyun > Add a symbolic link file `spark-examples.jar` in K8s Docker

[jira] [Resolved] (SPARK-45497) Add a symbolic link file `spark-examples.jar` in K8s Docker images

2023-10-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-45497. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43324

[jira] [Updated] (SPARK-45502) Upgrade Kafka to 3.6.0

2023-10-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45502: -- Summary: Upgrade Kafka to 3.6.0 (was: Upgrade to Kafka 3.6.0) > Upgrade Kafka to 3.6.0 >

[jira] [Updated] (SPARK-44115) Upgrade Apache ORC to 2.0

2023-10-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-44115: -- Summary: Upgrade Apache ORC to 2.0 (was: Upgrade to Apache ORC 2.0) > Upgrade Apache ORC to

[jira] [Created] (SPARK-45502) Upgrade to Kafka 3.6.0

2023-10-11 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-45502: - Summary: Upgrade to Kafka 3.6.0 Key: SPARK-45502 URL: https://issues.apache.org/jira/browse/SPARK-45502 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-45501) Use pattern matching for type checking and conversion

2023-10-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45501: --- Labels: pull-request-available (was: ) > Use pattern matching for type checking and

[jira] [Created] (SPARK-45501) Use pattern matching for type checking and conversion

2023-10-11 Thread Yang Jie (Jira)
Yang Jie created SPARK-45501: Summary: Use pattern matching for type checking and conversion Key: SPARK-45501 URL: https://issues.apache.org/jira/browse/SPARK-45501 Project: Spark Issue Type:

[jira] [Updated] (SPARK-45500) Show the number of abnormally completed drivers in MasterPage

2023-10-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45500: --- Labels: pull-request-available (was: ) > Show the number of abnormally completed drivers

[jira] [Created] (SPARK-45500) Show the number of abnormally completed drivers in MasterPage

2023-10-11 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-45500: - Summary: Show the number of abnormally completed drivers in MasterPage Key: SPARK-45500 URL: https://issues.apache.org/jira/browse/SPARK-45500 Project: Spark

[jira] [Commented] (SPARK-45478) codegen sum(decimal_column / 2) computes div twice

2023-10-11 Thread Zhizhen Hou (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773921#comment-17773921 ] Zhizhen Hou commented on SPARK-45478: - There are three children in If: predicate, trueValue and

[jira] [Updated] (SPARK-45498) Followup: Ignore task completion from old stage after retrying indeterminate stages

2023-10-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45498: --- Labels: pull-request-available (was: ) > Followup: Ignore task completion from old stage

[jira] [Updated] (SPARK-14745) CEP support in Spark Streaming

2023-10-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-14745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-14745: --- Labels: pull-request-available (was: ) > CEP support in Spark Streaming >

[jira] [Updated] (SPARK-45499) Replace `Reference#isEnqueued` with `Reference#refersTo`

2023-10-11 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-45499: --- Labels: pull-request-available (was: ) > Replace `Reference#isEnqueued` with

[jira] [Created] (SPARK-45498) Followup: Ignore task completion from old stage after retrying indeterminate stages

2023-10-11 Thread Mayur Bhosale (Jira)
Mayur Bhosale created SPARK-45498: - Summary: Followup: Ignore task completion from old stage after retrying indeterminate stages Key: SPARK-45498 URL: https://issues.apache.org/jira/browse/SPARK-45498

[jira] [Created] (SPARK-45499) Replace `Reference#isEnqueued` with `Reference#refersTo`

2023-10-11 Thread Yang Jie (Jira)
Yang Jie created SPARK-45499: Summary: Replace `Reference#isEnqueued` with `Reference#refersTo` Key: SPARK-45499 URL: https://issues.apache.org/jira/browse/SPARK-45499 Project: Spark Issue Type: