[jira] [Created] (SPARK-47852) Support DataFrameQueryContext for reverse operations

Haejoon Lee created SPARK-47852: --- Summary: Support DataFrameQueryContext for reverse operations Key: SPARK-47852 URL: https://issues.apache.org/jira/browse/SPARK-47852 Project: Spark Issue

[jira] [Created] (SPARK-47851) Document pyspark-connect package

Hyukjin Kwon created SPARK-47851: Summary: Document pyspark-connect package Key: SPARK-47851 URL: https://issues.apache.org/jira/browse/SPARK-47851 Project: Spark Issue Type: Sub-task

[jira] [Resolved] (SPARK-47803) Support cast to variant.

[ https://issues.apache.org/jira/browse/SPARK-47803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-47803. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45989

[jira] [Updated] (SPARK-47757) Reeanble MemoryProfilerParityTests for pyspark-connect

[ https://issues.apache.org/jira/browse/SPARK-47757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-47757: - Summary: Reeanble MemoryProfilerParityTests for pyspark-connect (was: Reeanble

[jira] [Updated] (SPARK-47830) Reeanble ResourceProfileTests for pyspark-connect

[ https://issues.apache.org/jira/browse/SPARK-47830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-47830: - Summary: Reeanble ResourceProfileTests for pyspark-connect (was: Reeanble

[jira] [Updated] (SPARK-47850) Support converting insert for unpartitioned Hive table

[ https://issues.apache.org/jira/browse/SPARK-47850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47850: --- Labels: pull-request-available (was: ) > Support converting insert for unpartitioned Hive

[jira] [Resolved] (SPARK-47849) Change release script to release pyspark-connect

[ https://issues.apache.org/jira/browse/SPARK-47849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47849. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46049

[jira] [Assigned] (SPARK-47849) Change release script to release pyspark-connect

[ https://issues.apache.org/jira/browse/SPARK-47849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-47849: Assignee: Hyukjin Kwon > Change release script to release pyspark-connect >

[jira] [Resolved] (SPARK-47757) Reeanble ResourceProfileTests for pyspark-connect

[ https://issues.apache.org/jira/browse/SPARK-47757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47757. -- Fix Version/s: 4.0.0 Assignee: Hyukjin Kwon Resolution: Fixed Fixed in

[jira] [Resolved] (SPARK-47756) Reeanble UDFProfilerParityTests for pyspark-connect

[ https://issues.apache.org/jira/browse/SPARK-47756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47756. -- Fix Version/s: 4.0.0 Assignee: Hyukjin Kwon Resolution: Fixed Fixed in

[jira] [Created] (SPARK-47850) Support converting insert for unpartitioned Hive table

Cheng Pan created SPARK-47850: - Summary: Support converting insert for unpartitioned Hive table Key: SPARK-47850 URL: https://issues.apache.org/jira/browse/SPARK-47850 Project: Spark Issue Type:

[jira] [Updated] (SPARK-47694) Make max message size configurable on client side

[ https://issues.apache.org/jira/browse/SPARK-47694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47694: -- Fix Version/s: (was: 3.4.3) > Make max message size configurable on client side >

[jira] [Resolved] (SPARK-47828) DataFrameWriterV2.overwrite fails with invalid plan

[ https://issues.apache.org/jira/browse/SPARK-47828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-47828. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46023

[jira] [Updated] (SPARK-47694) Make max message size configurable on client side

[ https://issues.apache.org/jira/browse/SPARK-47694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-47694: -- Affects Version/s: 4.0.0 (was: 3.4.1) > Make max message size

[jira] [Assigned] (SPARK-47828) DataFrameWriterV2.overwrite fails with invalid plan

[ https://issues.apache.org/jira/browse/SPARK-47828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-47828: - Assignee: Ruifeng Zheng > DataFrameWriterV2.overwrite fails with invalid plan >

[jira] [Resolved] (SPARK-47796) Spark Session is not created for spark-2.1.1 when log4j is migrated from 1.* to 2.17.1 to fix the security issue

[ https://issues.apache.org/jira/browse/SPARK-47796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahima Khatri resolved SPARK-47796. --- Resolution: Workaround > Spark Session is not created for spark-2.1.1 when log4j is

[jira] [Closed] (SPARK-47796) Spark Session is not created for spark-2.1.1 when log4j is migrated from 1.* to 2.17.1 to fix the security issue

[ https://issues.apache.org/jira/browse/SPARK-47796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahima Khatri closed SPARK-47796. - > Spark Session is not created for spark-2.1.1 when log4j is migrated from 1.* > to 2.17.1 to fix

[jira] [Commented] (SPARK-47796) Spark Session is not created for spark-2.1.1 when log4j is migrated from 1.* to 2.17.1 to fix the security issue

[ https://issues.apache.org/jira/browse/SPARK-47796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17837057#comment-17837057 ] Mahima Khatri commented on SPARK-47796: --- Found a workaround by removing the conflicting jar

[jira] [Updated] (SPARK-47849) Change release script to release pyspark-connect

[ https://issues.apache.org/jira/browse/SPARK-47849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47849: --- Labels: pull-request-available (was: ) > Change release script to release pyspark-connect

[jira] [Created] (SPARK-47849) Change release script to release pyspark-connect

Hyukjin Kwon created SPARK-47849: Summary: Change release script to release pyspark-connect Key: SPARK-47849 URL: https://issues.apache.org/jira/browse/SPARK-47849 Project: Spark Issue Type:

[jira] [Updated] (SPARK-47848) Fix thread safe access for loadedMaps in close

[ https://issues.apache.org/jira/browse/SPARK-47848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47848: --- Labels: pull-request-available (was: ) > Fix thread safe access for loadedMaps in close >

[jira] [Updated] (SPARK-47847) Deprecate spark.network.remoteReadNioBufferConversion

[ https://issues.apache.org/jira/browse/SPARK-47847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47847: --- Labels: pull-request-available (was: ) > Deprecate

[jira] [Created] (SPARK-47847) Deprecate spark.network.remoteReadNioBufferConversion

Cheng Pan created SPARK-47847: - Summary: Deprecate spark.network.remoteReadNioBufferConversion Key: SPARK-47847 URL: https://issues.apache.org/jira/browse/SPARK-47847 Project: Spark Issue Type:

[jira] [Commented] (SPARK-46981) Driver OOM happens in query planning phase with empty tables

[ https://issues.apache.org/jira/browse/SPARK-46981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17837039#comment-17837039 ] Jarred Li commented on SPARK-46981: --- I used default driver memory setting(1GB), OOM was thrown out. It

[jira] [Updated] (SPARK-47846) Add support for Variant schema in from_json

[ https://issues.apache.org/jira/browse/SPARK-47846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47846: --- Labels: pull-request-available (was: ) > Add support for Variant schema in from_json >

[jira] [Created] (SPARK-47846) Add support for Variant schema in from_json

Harsh Motwani created SPARK-47846: - Summary: Add support for Variant schema in from_json Key: SPARK-47846 URL: https://issues.apache.org/jira/browse/SPARK-47846 Project: Spark Issue Type:

[jira] [Commented] (SPARK-47845) Support column type in split function in scala and python

[ https://issues.apache.org/jira/browse/SPARK-47845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17837028#comment-17837028 ] Liu Cao commented on SPARK-47845: - Created PR for this https://github.com/apache/spark/pull/46045 >

[jira] [Updated] (SPARK-47845) Support column type in split function in scala and python

[ https://issues.apache.org/jira/browse/SPARK-47845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47845: --- Labels: pull-request-available (was: ) > Support column type in split function in scala

[jira] [Updated] (SPARK-47845) Support column type in split function in scala and python

[ https://issues.apache.org/jira/browse/SPARK-47845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liu Cao updated SPARK-47845: Target Version/s: (was: 4.0.0) > Support column type in split function in scala and python >

[jira] [Updated] (SPARK-47826) Add VariantVal for PySpark

[ https://issues.apache.org/jira/browse/SPARK-47826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-47826: --- Labels: pull-request-available (was: ) > Add VariantVal for PySpark >

[jira] [Updated] (SPARK-47845) Support column type in split function in scala and python

[ https://issues.apache.org/jira/browse/SPARK-47845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liu Cao updated SPARK-47845: Description: I have a use case to split a String typed column with different delimiters defined in other

[jira] [Updated] (SPARK-47845) Support column type in split function in scala and python

[ https://issues.apache.org/jira/browse/SPARK-47845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liu Cao updated SPARK-47845: Description: I have a use case to split a String typed column with different delimiters defined in other

[jira] [Created] (SPARK-47845) Support column type in split function in scala and python

Liu Cao created SPARK-47845: --- Summary: Support column type in split function in scala and python Key: SPARK-47845 URL: https://issues.apache.org/jira/browse/SPARK-47845 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-47413) Substring, Right, Left (all collations)

[ https://issues.apache.org/jira/browse/SPARK-47413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-47413: -- Assignee: (was: Apache Spark) > Substring, Right, Left (all collations) >

[jira] [Assigned] (SPARK-47413) Substring, Right, Left (all collations)

[ https://issues.apache.org/jira/browse/SPARK-47413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-47413: -- Assignee: Apache Spark > Substring, Right, Left (all collations) >