[jira] [Resolved] (SPARK-47743) Use milliseconds as the time unit in loggings

2024-04-05 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-47743. Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45903

[jira] [Resolved] (SPARK-47094) SPJ : Dynamically rebalance number of buckets when they are not equal

2024-04-05 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved SPARK-47094. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45267

[jira] [Updated] (SPARK-47744) Add support for negative byte types in RocksDB range scan encoder

2024-04-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47744: --- Labels: pull-request-available (was: ) > Add support for negative byte types in RocksDB

[jira] [Created] (SPARK-47746) Use column ordinals instead of prefix ordering columns in the range scan encoder

2024-04-05 Thread Neil Ramaswamy (Jira)
Neil Ramaswamy created SPARK-47746: -- Summary: Use column ordinals instead of prefix ordering columns in the range scan encoder Key: SPARK-47746 URL: https://issues.apache.org/jira/browse/SPARK-47746

[jira] [Updated] (SPARK-47581) SQL catalyst: Migrate logWarn with variables to structured logging framework

2024-04-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47581: --- Labels: pull-request-available (was: ) > SQL catalyst: Migrate logWarn with variables to

[jira] [Created] (SPARK-47745) Add License to Spark Operator

2024-04-05 Thread Zhou JIANG (Jira)
Zhou JIANG created SPARK-47745: -- Summary: Add License to Spark Operator Key: SPARK-47745 URL: https://issues.apache.org/jira/browse/SPARK-47745 Project: Spark Issue Type: Sub-task

[jira] [Resolved] (SPARK-47582) SQL catalyst: Migrate logInfo with variables to structured logging framework

2024-04-05 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-47582. Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45866

[jira] [Created] (SPARK-47744) Add support for negative byte types in RocksDB range scan encoder

2024-04-05 Thread Neil Ramaswamy (Jira)
Neil Ramaswamy created SPARK-47744: -- Summary: Add support for negative byte types in RocksDB range scan encoder Key: SPARK-47744 URL: https://issues.apache.org/jira/browse/SPARK-47744 Project: Spark

[jira] [Updated] (SPARK-45445) Upgrade snappy to 1.1.10.5

2024-04-05 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45445: -- Fix Version/s: 3.4.3 > Upgrade snappy to 1.1.10.5 > -- > >

[jira] [Updated] (SPARK-47743) Use milliseconds as the time unit in loggings

2024-04-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47743: --- Labels: pull-request-available (was: ) > Use milliseconds as the time unit in loggings >

[jira] [Created] (SPARK-47743) Use milliseconds as the time unit in loggings

2024-04-05 Thread Gengliang Wang (Jira)
Gengliang Wang created SPARK-47743: -- Summary: Use milliseconds as the time unit in loggings Key: SPARK-47743 URL: https://issues.apache.org/jira/browse/SPARK-47743 Project: Spark Issue

[jira] [Updated] (SPARK-45445) Upgrade snappy to 1.1.10.5

2024-04-05 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45445: -- Fix Version/s: 3.5.2 > Upgrade snappy to 1.1.10.5 > -- > >

[jira] [Updated] (SPARK-46411) Change to use bcprov/bcpkix-jdk18on for test

2024-04-05 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-46411: -- Fix Version/s: 3.4.3 > Change to use bcprov/bcpkix-jdk18on for test >

[jira] [Resolved] (SPARK-47719) Change default of spark.sql.legacy.timeParserPolicy from EXCEPTION to CORRECTED

2024-04-05 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-47719. Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45859

[jira] [Assigned] (SPARK-47719) Change default of spark.sql.legacy.timeParserPolicy from EXCEPTION to CORRECTED

2024-04-05 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang reassigned SPARK-47719: -- Assignee: Serge Rielau > Change default of spark.sql.legacy.timeParserPolicy from

[jira] [Updated] (SPARK-47742) Spark Transformation with Multi Case filter can improve efficiency

2024-04-05 Thread Hemant Sakharkar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hemant Sakharkar updated SPARK-47742: - Description: In Feature Engineering we need to process the input data to create feature

[jira] [Updated] (SPARK-47741) Handle stack overflow when parsing query as part of Execute immediate

2024-04-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47741: --- Labels: pull-request-available (was: ) > Handle stack overflow when parsing query as part

[jira] [Updated] (SPARK-47742) Spark Transformation with Multi Case filter can improve efficiency

2024-04-05 Thread Hemant Sakharkar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hemant Sakharkar updated SPARK-47742: - Description: In Feature Engineering we need to process the input data to create feature

[jira] [Updated] (SPARK-47742) Spark Transformation with Multi Case filter can improve efficiency

2024-04-05 Thread Hemant Sakharkar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hemant Sakharkar updated SPARK-47742: - Attachment: spark_chain_transformation.png > Spark Transformation with Multi Case

[jira] [Created] (SPARK-47742) Spark Transformation with Multi Case filter can improve efficiency

2024-04-05 Thread Hemant Sakharkar (Jira)
Hemant Sakharkar created SPARK-47742: Summary: Spark Transformation with Multi Case filter can improve efficiency Key: SPARK-47742 URL: https://issues.apache.org/jira/browse/SPARK-47742 Project:

[jira] [Updated] (SPARK-47740) Stop JVM by calling SparkSession.stop from PySpark

2024-04-05 Thread Maxim Martynov (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Martynov updated SPARK-47740: --- Description: {code:python} from pyspark.sql import SparkSession spark =

[jira] [Updated] (SPARK-47740) Stop JVM by calling SparkSession.stop from PySpark

2024-04-05 Thread Maxim Martynov (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Martynov updated SPARK-47740: --- Description: {code:python} from pyspark.sql import SparkSession spark =

[jira] [Updated] (SPARK-46411) Change to use bcprov/bcpkix-jdk18on for test

2024-04-05 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-46411: -- Fix Version/s: 3.5.2 > Change to use bcprov/bcpkix-jdk18on for test >

[jira] [Updated] (SPARK-45445) Upgrade snappy to 1.1.10.5

2024-04-05 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45445: -- Parent: SPARK-47046 Issue Type: Sub-task (was: Improvement) > Upgrade snappy to

[jira] [Updated] (SPARK-45323) Upgrade snappy to 1.1.10.4

2024-04-05 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45323: -- Parent: SPARK-47046 Issue Type: Sub-task (was: Dependency upgrade) > Upgrade snappy

[jira] [Updated] (SPARK-47740) Stop JVM by calling SparkSession.stop from PySpark

2024-04-05 Thread Maxim Martynov (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Martynov updated SPARK-47740: --- Description: {code:python} from pyspark.sql import SparkSession spark =

[jira] [Commented] (SPARK-47740) Stop JVM by calling SparkSession.stop from PySpark

2024-04-05 Thread Maxim Martynov (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17834379#comment-17834379 ] Maxim Martynov commented on SPARK-47740: Here is a PR for adding helper function

[jira] [Updated] (SPARK-45172) Upgrade commons-compress.version to 1.24.0

2024-04-05 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-45172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-45172: -- Parent: SPARK-47046 Issue Type: Sub-task (was: Improvement) > Upgrade

[jira] [Updated] (SPARK-44441) Upgrade bcprov-jdk15on and bcpkix-jdk15on to 1.70

2024-04-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-1?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-1: --- Labels: pull-request-available (was: ) > Upgrade bcprov-jdk15on and bcpkix-jdk15on to 1.70

[jira] [Updated] (SPARK-44393) Upgrade H2 from 2.1.214 to 2.2.220

2024-04-05 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-44393: -- Fix Version/s: 3.4.3 > Upgrade H2 from 2.1.214 to 2.2.220 >

[jira] [Updated] (SPARK-44393) Upgrade H2 from 2.1.214 to 2.2.220

2024-04-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-44393: --- Labels: pull-request-available (was: ) > Upgrade H2 from 2.1.214 to 2.2.220 >

[jira] [Updated] (SPARK-44279) Upgrade optionator to ^0.9.3

2024-04-05 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-44279: -- Fix Version/s: 3.5.0 > Upgrade optionator to ^0.9.3 > > >

[jira] [Created] (SPARK-47741) Handle stack overflow when parsing query as part of Execute immediate

2024-04-05 Thread Milan Stefanovic (Jira)
Milan Stefanovic created SPARK-47741: Summary: Handle stack overflow when parsing query as part of Execute immediate Key: SPARK-47741 URL: https://issues.apache.org/jira/browse/SPARK-47741

[jira] [Updated] (SPARK-47740) Stop JVM by calling SparkSession.stop from PySpark

2024-04-05 Thread Maxim Martynov (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Martynov updated SPARK-47740: --- Description: {code:python} from pyspark.sql import SparkSession spark =

[jira] [Updated] (SPARK-47740) Stop JVM by calling SparkSession.stop from PySpark

2024-04-05 Thread Maxim Martynov (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Martynov updated SPARK-47740: --- Description: {code:python} from pyspark.sql import SparkSession spark =

[jira] [Updated] (SPARK-47740) Stop JVM by calling SparkSession.stop from PySpark

2024-04-05 Thread Maxim Martynov (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Martynov updated SPARK-47740: --- Description: {code:python} from pyspark.sql import SparkSession spark =

[jira] [Created] (SPARK-47740) Stop JVM by calling SparkSession.stop from PySpark

2024-04-05 Thread Maxim Martynov (Jira)
Maxim Martynov created SPARK-47740: -- Summary: Stop JVM by calling SparkSession.stop from PySpark Key: SPARK-47740 URL: https://issues.apache.org/jira/browse/SPARK-47740 Project: Spark Issue

[jira] [Resolved] (SPARK-47735) Make pyspark.testing.connectutils compatible with pyspark-connect

2024-04-05 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-47735. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45887

[jira] [Assigned] (SPARK-47735) Make pyspark.testing.connectutils compatible with pyspark-connect

2024-04-05 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-47735: - Assignee: Hyukjin Kwon > Make pyspark.testing.connectutils compatible with

[jira] [Updated] (SPARK-46632) EquivalentExpressions throw IllegalStateException

2024-04-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46632: --- Labels: pull-request-available (was: ) > EquivalentExpressions throw IllegalStateException

[jira] [Comment Edited] (SPARK-46143) pyspark.pandas read_excel implementation at version 3.4.1

2024-04-05 Thread Christos Karras (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17834315#comment-17834315 ] Christos Karras edited comment on SPARK-46143 at 4/5/24 1:47 PM: - I have

[jira] [Comment Edited] (SPARK-46143) pyspark.pandas read_excel implementation at version 3.4.1

2024-04-05 Thread Christos Karras (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17834315#comment-17834315 ] Christos Karras edited comment on SPARK-46143 at 4/5/24 1:42 PM: - I have

[jira] [Comment Edited] (SPARK-46143) pyspark.pandas read_excel implementation at version 3.4.1

2024-04-05 Thread Christos Karras (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17834315#comment-17834315 ] Christos Karras edited comment on SPARK-46143 at 4/5/24 1:42 PM: - I have

[jira] [Comment Edited] (SPARK-46143) pyspark.pandas read_excel implementation at version 3.4.1

2024-04-05 Thread Christos Karras (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17834315#comment-17834315 ] Christos Karras edited comment on SPARK-46143 at 4/5/24 1:41 PM: - I have

[jira] [Commented] (SPARK-46143) pyspark.pandas read_excel implementation at version 3.4.1

2024-04-05 Thread Christos Karras (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17834315#comment-17834315 ] Christos Karras commented on SPARK-46143: - I have the same issue too. The problem is because the

[jira] [Updated] (SPARK-47737) Bump PyArrow to 15.0.2

2024-04-05 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47737: --- Labels: pull-request-available (was: ) > Bump PyArrow to 15.0.2 > -- >

[jira] [Updated] (SPARK-47736) Add support for AbstractArrayType(StringTypeCollated)

2024-04-05 Thread Mihailo Milosevic (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mihailo Milosevic updated SPARK-47736: -- Summary: Add support for AbstractArrayType(StringTypeCollated) (was: Add support for

[jira] [Created] (SPARK-47736) Add support for ArrayType(StringTypeAnyCollation)

2024-04-05 Thread Mihailo Milosevic (Jira)
Mihailo Milosevic created SPARK-47736: - Summary: Add support for ArrayType(StringTypeAnyCollation) Key: SPARK-47736 URL: https://issues.apache.org/jira/browse/SPARK-47736 Project: Spark