[jira] [Created] (SPARK-43838) Subquery on single table with having clause can't be optimized

Jia Fan created SPARK-43838: --- Summary: Subquery on single table with having clause can't be optimized Key: SPARK-43838 URL: https://issues.apache.org/jira/browse/SPARK-43838 Project: Spark Issue

[jira] [Created] (SPARK-43837) Assign a name to the error class _LEGACY_ERROR_TEMP_103[1-2]

BingKun Pan created SPARK-43837: --- Summary: Assign a name to the error class _LEGACY_ERROR_TEMP_103[1-2] Key: SPARK-43837 URL: https://issues.apache.org/jira/browse/SPARK-43837 Project: Spark

[jira] [Created] (SPARK-43836) Make Scala 2.13 as default Scala version in Spark 3.5

Dongjoon Hyun created SPARK-43836: - Summary: Make Scala 2.13 as default Scala version in Spark 3.5 Key: SPARK-43836 URL: https://issues.apache.org/jira/browse/SPARK-43836 Project: Spark

[jira] [Updated] (SPARK-43205) Add an IDENTIFIER(stringLiteral) clause that maps a string to an identifier

[ https://issues.apache.org/jira/browse/SPARK-43205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-43205: -- Priority: Major (was: Minor) > Add an IDENTIFIER(stringLiteral) clause that maps a string to

[jira] [Updated] (SPARK-43205) Add an IDENTIFIER(stringLiteral) clause that maps a string to an identifier

[ https://issues.apache.org/jira/browse/SPARK-43205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-43205: -- Component/s: Documentation (was: Spark Core) > Add an

[jira] [Updated] (SPARK-43205) Add an IDENTIFIER(stringLiteral) clause that maps a string to an identifier

[ https://issues.apache.org/jira/browse/SPARK-43205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-43205: -- Issue Type: New Feature (was: Documentation) > Add an IDENTIFIER(stringLiteral) clause that

[jira] [Updated] (SPARK-43205) Add an IDENTIFIER(stringLiteral) clause that maps a string to an identifier

[ https://issues.apache.org/jira/browse/SPARK-43205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-43205: -- Component/s: SQL (was: Documentation) > Add an IDENTIFIER(stringLiteral)

[jira] [Updated] (SPARK-43205) Add an IDENTIFIER(stringLiteral) clause that maps a string to an identifier

[ https://issues.apache.org/jira/browse/SPARK-43205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-43205: -- Issue Type: Documentation (was: New Feature) > Add an IDENTIFIER(stringLiteral) clause that

[jira] [Updated] (SPARK-43205) Add an IDENTIFIER(stringLiteral) clause that maps a string to an identifier

[ https://issues.apache.org/jira/browse/SPARK-43205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-43205: -- Priority: Minor (was: Major) > Add an IDENTIFIER(stringLiteral) clause that maps a string to

[jira] [Created] (SPARK-43835) Add `build_java21.yml` Github Action Daily Job

Dongjoon Hyun created SPARK-43835: - Summary: Add `build_java21.yml` Github Action Daily Job Key: SPARK-43835 URL: https://issues.apache.org/jira/browse/SPARK-43835 Project: Spark Issue Type:

[jira] [Created] (SPARK-43834) Use error classes in the compilation errors of `ResolveDefaultColumns`

BingKun Pan created SPARK-43834: --- Summary: Use error classes in the compilation errors of `ResolveDefaultColumns` Key: SPARK-43834 URL: https://issues.apache.org/jira/browse/SPARK-43834 Project: Spark

[jira] [Created] (SPARK-43833) Upgrade Scala to 2.13.11

Dongjoon Hyun created SPARK-43833: - Summary: Upgrade Scala to 2.13.11 Key: SPARK-43833 URL: https://issues.apache.org/jira/browse/SPARK-43833 Project: Spark Issue Type: Sub-task

[jira] [Assigned] (SPARK-43702) Fix pyspark.sql.pandas.types.to_arrow_type to work with Spark Connect

[ https://issues.apache.org/jira/browse/SPARK-43702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-43702: - Assignee: Takuya Ueshin > Fix pyspark.sql.pandas.types.to_arrow_type to work with

[jira] [Resolved] (SPARK-43702) Fix pyspark.sql.pandas.types.to_arrow_type to work with Spark Connect

[ https://issues.apache.org/jira/browse/SPARK-43702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-43702. --- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41333

[jira] [Resolved] (SPARK-43817) Support UserDefinedType in creaetDataFrame from pandas DataFrame and toPandas

[ https://issues.apache.org/jira/browse/SPARK-43817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-43817. --- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41333

[jira] [Assigned] (SPARK-43817) Support UserDefinedType in creaetDataFrame from pandas DataFrame and toPandas

[ https://issues.apache.org/jira/browse/SPARK-43817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-43817: - Assignee: Takuya Ueshin > Support UserDefinedType in creaetDataFrame from pandas

[jira] [Resolved] (SPARK-43671) Enable CategoricalOps.ge to work with Spark Connect.

[ https://issues.apache.org/jira/browse/SPARK-43671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-43671. --- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41326

[jira] [Assigned] (SPARK-43671) Enable CategoricalOps.ge to work with Spark Connect.

[ https://issues.apache.org/jira/browse/SPARK-43671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-43671: - Assignee: Haejoon Lee > Enable CategoricalOps.ge to work with Spark Connect. >

[jira] [Assigned] (SPARK-43693) Fix StringOps.gt to work with Spark Connect

[ https://issues.apache.org/jira/browse/SPARK-43693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-43693: - Assignee: Haejoon Lee > Fix StringOps.gt to work with Spark Connect >

[jira] [Resolved] (SPARK-43695) Fix StringOps.lt to work with Spark Connect

[ https://issues.apache.org/jira/browse/SPARK-43695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-43695. --- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41308

[jira] [Assigned] (SPARK-43694) Fix StringOps.le to work with Spark Connect

[ https://issues.apache.org/jira/browse/SPARK-43694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-43694: - Assignee: Haejoon Lee > Fix StringOps.le to work with Spark Connect >

[jira] [Resolved] (SPARK-43693) Fix StringOps.gt to work with Spark Connect

[ https://issues.apache.org/jira/browse/SPARK-43693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-43693. --- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41308

[jira] [Assigned] (SPARK-43695) Fix StringOps.lt to work with Spark Connect

[ https://issues.apache.org/jira/browse/SPARK-43695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-43695: - Assignee: Haejoon Lee > Fix StringOps.lt to work with Spark Connect >

[jira] [Resolved] (SPARK-43694) Fix StringOps.le to work with Spark Connect

[ https://issues.apache.org/jira/browse/SPARK-43694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-43694. --- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41308

[jira] [Assigned] (SPARK-43692) Fix StringOps.ge to work with Spark Connect

[ https://issues.apache.org/jira/browse/SPARK-43692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-43692: - Assignee: Haejoon Lee > Fix StringOps.ge to work with Spark Connect >

[jira] [Resolved] (SPARK-43692) Fix StringOps.ge to work with Spark Connect

[ https://issues.apache.org/jira/browse/SPARK-43692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-43692. --- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41308

[jira] [Assigned] (SPARK-43773) Implement 'levenshtein(str1, str2[, threshold])' functions in python client

[ https://issues.apache.org/jira/browse/SPARK-43773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-43773: - Assignee: BingKun Pan > Implement 'levenshtein(str1, str2[, threshold])' functions in

[jira] [Resolved] (SPARK-43773) Implement 'levenshtein(str1, str2[, threshold])' functions in python client

[ https://issues.apache.org/jira/browse/SPARK-43773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-43773. --- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41296

[jira] [Resolved] (SPARK-43603) Rebalance pyspark.pandas.DataFrame Unit Tests

[ https://issues.apache.org/jira/browse/SPARK-43603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-43603. --- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41330

[jira] [Assigned] (SPARK-43603) Rebalance pyspark.pandas.DataFrame Unit Tests

[ https://issues.apache.org/jira/browse/SPARK-43603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-43603: - Assignee: Ruifeng Zheng > Rebalance pyspark.pandas.DataFrame Unit Tests >

[jira] [Created] (SPARK-43832) Upgrade Scala to 2.12.18

Dongjoon Hyun created SPARK-43832: - Summary: Upgrade Scala to 2.12.18 Key: SPARK-43832 URL: https://issues.apache.org/jira/browse/SPARK-43832 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-43577) Upgrade cyclonedx-maven-plugin to 2.7.9

[ https://issues.apache.org/jira/browse/SPARK-43577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-43577: -- Parent: SPARK-43831 Issue Type: Sub-task (was: Improvement) > Upgrade

[jira] [Commented] (SPARK-43577) Upgrade cyclonedx-maven-plugin to 2.7.9

[ https://issues.apache.org/jira/browse/SPARK-43577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17726854#comment-17726854 ] Dongjoon Hyun commented on SPARK-43577: --- We will revisit this as your contribution while achieving

[jira] [Commented] (SPARK-42369) Fix constructor for java.nio.DirectByteBuffer for Java 21+

[ https://issues.apache.org/jira/browse/SPARK-42369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17726853#comment-17726853 ] Dongjoon Hyun commented on SPARK-42369: --- Thank you, [~luhenry]. I collected this as a subtask of

[jira] [Commented] (SPARK-43588) Upgrade ASM to 9.5

[ https://issues.apache.org/jira/browse/SPARK-43588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17726852#comment-17726852 ] Dongjoon Hyun commented on SPARK-43588: --- Thank you always, [~LuciferYang]. I collected this as a

[jira] [Updated] (SPARK-43588) Upgrade ASM to 9.5

[ https://issues.apache.org/jira/browse/SPARK-43588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-43588: -- Parent: SPARK-43831 Issue Type: Sub-task (was: Improvement) > Upgrade ASM to 9.5 >

[jira] [Created] (SPARK-43831) Build and Run Spark on Java 21

Dongjoon Hyun created SPARK-43831: - Summary: Build and Run Spark on Java 21 Key: SPARK-43831 URL: https://issues.apache.org/jira/browse/SPARK-43831 Project: Spark Issue Type: New Feature

[jira] [Updated] (SPARK-42369) Fix constructor for java.nio.DirectByteBuffer for Java 21+

[ https://issues.apache.org/jira/browse/SPARK-42369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-42369: -- Parent: SPARK-43831 Issue Type: Sub-task (was: Improvement) > Fix constructor for

[jira] [Updated] (SPARK-43831) Build and Run Spark on Java 21

[ https://issues.apache.org/jira/browse/SPARK-43831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-43831: -- Target Version/s: 3.5.0 > Build and Run Spark on Java 21 > -- > >

[jira] [Assigned] (SPARK-43830) Update scalatest and scalatestplus related dependencies to newest version

[ https://issues.apache.org/jira/browse/SPARK-43830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-43830: Assignee: BingKun Pan > Update scalatest and scalatestplus related dependencies to

[jira] [Resolved] (SPARK-43830) Update scalatest and scalatestplus related dependencies to newest version

[ https://issues.apache.org/jira/browse/SPARK-43830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-43830. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41341

[jira] [Resolved] (SPARK-43824) Assign a name to the error class _LEGACY_ERROR_TEMP_1281

[ https://issues.apache.org/jira/browse/SPARK-43824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-43824. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41338

[jira] [Assigned] (SPARK-43824) Assign a name to the error class _LEGACY_ERROR_TEMP_1281

[ https://issues.apache.org/jira/browse/SPARK-43824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-43824: Assignee: BingKun Pan > Assign a name to the error class _LEGACY_ERROR_TEMP_1281 >

[jira] [Resolved] (SPARK-43825) Assign a name to the error class _LEGACY_ERROR_TEMP_1282

[ https://issues.apache.org/jira/browse/SPARK-43825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-43825. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41338

[jira] [Assigned] (SPARK-43825) Assign a name to the error class _LEGACY_ERROR_TEMP_1282

[ https://issues.apache.org/jira/browse/SPARK-43825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-43825: Assignee: BingKun Pan > Assign a name to the error class _LEGACY_ERROR_TEMP_1282 >

[jira] [Resolved] (SPARK-43530) Protobuf: Read descriptor file only once at the compile time

[ https://issues.apache.org/jira/browse/SPARK-43530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie resolved SPARK-43530. -- Resolution: Fixed Issue resolved by pull request 41192 [https://github.com/apache/spark/pull/41192]

[jira] [Assigned] (SPARK-43530) Protobuf: Read descriptor file only once at the compile time

[ https://issues.apache.org/jira/browse/SPARK-43530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie reassigned SPARK-43530: Assignee: Raghu Angadi > Protobuf: Read descriptor file only once at the compile time >

[jira] [Updated] (SPARK-43830) Update scalatest and scalatestplus related dependencies to newest version

[ https://issues.apache.org/jira/browse/SPARK-43830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BingKun Pan updated SPARK-43830: Description: # scalatest: upgrade scalatest from 3.2.15 to 3.2.16     # mockito    -

[jira] [Updated] (SPARK-43830) Update scalatest and scalatestplus related dependencies to newest version

[ https://issues.apache.org/jira/browse/SPARK-43830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BingKun Pan updated SPARK-43830: Description: - scalatest: upgrade scalatest from 3.2.15 to 3.2.16       - mockito    -

[jira] [Created] (SPARK-43830) Update scalatest and scalatestplus related dependencies to newest version

BingKun Pan created SPARK-43830: --- Summary: Update scalatest and scalatestplus related dependencies to newest version Key: SPARK-43830 URL: https://issues.apache.org/jira/browse/SPARK-43830 Project:

[jira] [Created] (SPARK-43829) Improve SparkConnectPlanner by reuse Dataset and avoid construct new Dataset

jiaan.geng created SPARK-43829: -- Summary: Improve SparkConnectPlanner by reuse Dataset and avoid construct new Dataset Key: SPARK-43829 URL: https://issues.apache.org/jira/browse/SPARK-43829 Project:

[jira] [Updated] (SPARK-43795) Remove parameters not used for SparkConnectPlanner

[ https://issues.apache.org/jira/browse/SPARK-43795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-43795: --- Issue Type: Improvement (was: Documentation) > Remove parameters not used for SparkConnectPlanner