[jira] [Resolved] (SPARK-48190) Introduce a helper function to drop metadata

[ https://issues.apache.org/jira/browse/SPARK-48190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-48190. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46466

[jira] [Updated] (SPARK-48184) Always set the seed of dataframe.sample in Client side

[ https://issues.apache.org/jira/browse/SPARK-48184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-48184: -- Description: In Spark Classic: In [1]: df = spark.range(1).sample(0.1) In [2]:

[jira] [Assigned] (SPARK-48142) Enable `CogroupedApplyInPandasTests.test_wrong_args`

[ https://issues.apache.org/jira/browse/SPARK-48142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-48142: - Assignee: Ruifeng Zheng > Enable `CogroupedApplyInPandasTests.test_wrong_args` >

[jira] [Resolved] (SPARK-48142) Enable `CogroupedApplyInPandasTests.test_wrong_args`

[ https://issues.apache.org/jira/browse/SPARK-48142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-48142. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46397

[jira] [Created] (SPARK-48142) Enable `CogroupedApplyInPandasTests.test_wrong_args`

Ruifeng Zheng created SPARK-48142: - Summary: Enable `CogroupedApplyInPandasTests.test_wrong_args` Key: SPARK-48142 URL: https://issues.apache.org/jira/browse/SPARK-48142 Project: Spark Issue

[jira] [Created] (SPARK-48058) `UserDefinedFunction.returnType` parse the DDL string

Ruifeng Zheng created SPARK-48058: - Summary: `UserDefinedFunction.returnType` parse the DDL string Key: SPARK-48058 URL: https://issues.apache.org/jira/browse/SPARK-48058 Project: Spark

[jira] [Resolved] (SPARK-48055) Enable PandasUDFScalarParityTests.{test_vectorized_udf_empty_partition, test_vectorized_udf_struct_with_empty_partition}

[ https://issues.apache.org/jira/browse/SPARK-48055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-48055. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46296

[jira] [Updated] (SPARK-47129) Make ResolveRelations cache connect plan properly

[ https://issues.apache.org/jira/browse/SPARK-47129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-47129: -- Issue Type: Bug (was: Improvement) > Make ResolveRelations cache connect plan properly >

[jira] [Updated] (SPARK-47129) Make ResolveRelations cache connect plan properly

[ https://issues.apache.org/jira/browse/SPARK-47129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-47129: -- Affects Version/s: 3.4.3 3.5.1 > Make ResolveRelations cache connect

[jira] [Created] (SPARK-48020) Pin 'pandas==2.2.2'

Ruifeng Zheng created SPARK-48020: - Summary: Pin 'pandas==2.2.2' Key: SPARK-48020 URL: https://issues.apache.org/jira/browse/SPARK-48020 Project: Spark Issue Type: Bug Components:

[jira] [Created] (SPARK-48005) Enable `DefaultIndexParityTests. test_index_distributed_sequence_cleanup`

Ruifeng Zheng created SPARK-48005: - Summary: Enable `DefaultIndexParityTests. test_index_distributed_sequence_cleanup` Key: SPARK-48005 URL: https://issues.apache.org/jira/browse/SPARK-48005 Project:

[jira] [Resolved] (SPARK-47986) [CONNECT][PYTHON] Unable to create a new session when the default session is closed by the server

[ https://issues.apache.org/jira/browse/SPARK-47986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-47986. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46221

[jira] [Resolved] (SPARK-47985) Simplify functions with `lit`

[ https://issues.apache.org/jira/browse/SPARK-47985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-47985. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46219

[jira] [Assigned] (SPARK-47985) Simplify functions with `lit`

[ https://issues.apache.org/jira/browse/SPARK-47985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-47985: - Assignee: Ruifeng Zheng > Simplify functions with `lit` >

[jira] [Updated] (SPARK-47623) Enable `QuietTest` in parity tests

[ https://issues.apache.org/jira/browse/SPARK-47623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-47623: -- Summary: Enable `QuietTest` in parity tests (was: Use `QuietTest` in parity tests) > Enable

[jira] [Created] (SPARK-47987) Reenable `ArrowParityTests.test_createDataFrame_empty_partition`

Ruifeng Zheng created SPARK-47987: - Summary: Reenable `ArrowParityTests.test_createDataFrame_empty_partition` Key: SPARK-47987 URL: https://issues.apache.org/jira/browse/SPARK-47987 Project: Spark

[jira] [Updated] (SPARK-47970) Revisit skipped parity tests for PySpark Connect

[ https://issues.apache.org/jira/browse/SPARK-47970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-47970: -- Summary: Revisit skipped parity tests for PySpark Connect (was: Revisit skipped parity tests

[jira] [Created] (SPARK-47985) Simplify functions with `lit`

Ruifeng Zheng created SPARK-47985: - Summary: Simplify functions with `lit` Key: SPARK-47985 URL: https://issues.apache.org/jira/browse/SPARK-47985 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-47499) Reuse `test_help_command` in Connect

[ https://issues.apache.org/jira/browse/SPARK-47499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-47499: -- Parent: SPARK-47970 Issue Type: Sub-task (was: Test) > Reuse `test_help_command` in

[jira] [Created] (SPARK-47970) Revisit skipped parity tests for PySpark

Ruifeng Zheng created SPARK-47970: - Summary: Revisit skipped parity tests for PySpark Key: SPARK-47970 URL: https://issues.apache.org/jira/browse/SPARK-47970 Project: Spark Issue Type:

[jira] [Created] (SPARK-47937) Fix docstring of `hll_sketch_agg`

Ruifeng Zheng created SPARK-47937: - Summary: Fix docstring of `hll_sketch_agg` Key: SPARK-47937 URL: https://issues.apache.org/jira/browse/SPARK-47937 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-47935) Pin pandas==2.0.3 for pypy3.8

Ruifeng Zheng created SPARK-47935: - Summary: Pin pandas==2.0.3 for pypy3.8 Key: SPARK-47935 URL: https://issues.apache.org/jira/browse/SPARK-47935 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-47845) Support column type in split function in scala and python

[ https://issues.apache.org/jira/browse/SPARK-47845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-47845. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46045

[jira] [Assigned] (SPARK-47845) Support column type in split function in scala and python

[ https://issues.apache.org/jira/browse/SPARK-47845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-47845: - Assignee: Liu Cao > Support column type in split function in scala and python >

[jira] [Resolved] (SPARK-47906) Fix docstring and type hint of `hll_union_agg`

[ https://issues.apache.org/jira/browse/SPARK-47906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-47906. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46128

[jira] [Assigned] (SPARK-47883) Make CollectTailExec execute lazily

[ https://issues.apache.org/jira/browse/SPARK-47883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-47883: - Assignee: Ruifeng Zheng > Make CollectTailExec execute lazily >

[jira] [Resolved] (SPARK-47883) Make CollectTailExec execute lazily

[ https://issues.apache.org/jira/browse/SPARK-47883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-47883. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46101

[jira] [Updated] (SPARK-47883) Make CollectTailExec execute lazily

[ https://issues.apache.org/jira/browse/SPARK-47883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-47883: -- Summary: Make CollectTailExec execute lazily (was: Make CollectTailExec lazily execute) >

[jira] [Created] (SPARK-47883) Make CollectTailExec lazily execute

Ruifeng Zheng created SPARK-47883: - Summary: Make CollectTailExec lazily execute Key: SPARK-47883 URL: https://issues.apache.org/jira/browse/SPARK-47883 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-47868) Recursion Limit Error in SparkSession and SparkConnectPlanner

[ https://issues.apache.org/jira/browse/SPARK-47868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-47868. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46075

[jira] [Created] (SPARK-47855) Warn `spark.sql.execution.arrow.pyspark.fallback.enabled` in Connect

Ruifeng Zheng created SPARK-47855: - Summary: Warn `spark.sql.execution.arrow.pyspark.fallback.enabled` in Connect Key: SPARK-47855 URL: https://issues.apache.org/jira/browse/SPARK-47855 Project:

[jira] [Resolved] (SPARK-47828) DataFrameWriterV2.overwrite fails with invalid plan

[ https://issues.apache.org/jira/browse/SPARK-47828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-47828. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46023

[jira] [Assigned] (SPARK-47828) DataFrameWriterV2.overwrite fails with invalid plan

[ https://issues.apache.org/jira/browse/SPARK-47828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-47828: - Assignee: Ruifeng Zheng > DataFrameWriterV2.overwrite fails with invalid plan >

[jira] [Assigned] (SPARK-47816) Document the lazy evaluation of views in spark.{sql, table}

[ https://issues.apache.org/jira/browse/SPARK-47816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-47816: - Assignee: Ruifeng Zheng > Document the lazy evaluation of views in spark.{sql, table}

[jira] [Resolved] (SPARK-47816) Document the lazy evaluation of views in spark.{sql, table}

[ https://issues.apache.org/jira/browse/SPARK-47816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-47816. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 46007

[jira] [Updated] (SPARK-47828) DataFrameWriterV2.overwrite fails with invalid plan

[ https://issues.apache.org/jira/browse/SPARK-47828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-47828: -- Affects Version/s: 3.4.2 > DataFrameWriterV2.overwrite fails with invalid plan >

[jira] [Updated] (SPARK-47828) DataFrameWriterV2.overwrite fails with invalid plan

[ https://issues.apache.org/jira/browse/SPARK-47828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-47828: -- Issue Type: Bug (was: Improvement) > DataFrameWriterV2.overwrite fails with invalid plan >

[jira] [Created] (SPARK-47828) DataFrameWriterV2.overwrite fails with invalid plan

Ruifeng Zheng created SPARK-47828: - Summary: DataFrameWriterV2.overwrite fails with invalid plan Key: SPARK-47828 URL: https://issues.apache.org/jira/browse/SPARK-47828 Project: Spark Issue

[jira] [Resolved] (SPARK-47815) Unify the user agent with json

[ https://issues.apache.org/jira/browse/SPARK-47815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-47815. --- Resolution: Not A Problem > Unify the user agent with json > --

[jira] [Created] (SPARK-47816) Document the lazy evaluation of views in spark.{sql, table}

Ruifeng Zheng created SPARK-47816: - Summary: Document the lazy evaluation of views in spark.{sql, table} Key: SPARK-47816 URL: https://issues.apache.org/jira/browse/SPARK-47816 Project: Spark

[jira] [Updated] (SPARK-47815) Unify the user agent string with json

[ https://issues.apache.org/jira/browse/SPARK-47815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-47815: -- Summary: Unify the user agent string with json (was: Unify the user agent string

[jira] [Created] (SPARK-47815) Unify the user agent string representation with json

Ruifeng Zheng created SPARK-47815: - Summary: Unify the user agent string representation with json Key: SPARK-47815 URL: https://issues.apache.org/jira/browse/SPARK-47815 Project: Spark Issue

[jira] [Updated] (SPARK-47815) Unify the user agent with json

[ https://issues.apache.org/jira/browse/SPARK-47815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-47815: -- Summary: Unify the user agent with json (was: Unify the user agent string with json) >

[jira] [Assigned] (SPARK-47779) Add a helper function to sort PS Frame/Series

[ https://issues.apache.org/jira/browse/SPARK-47779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-47779: - Assignee: Ruifeng Zheng > Add a helper function to sort PS Frame/Series >

[jira] [Resolved] (SPARK-47779) Add a helper function to sort PS Frame/Series

[ https://issues.apache.org/jira/browse/SPARK-47779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-47779. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45952

[jira] [Created] (SPARK-47779) Add a helper function to sort PS Frame/Series

Ruifeng Zheng created SPARK-47779: - Summary: Add a helper function to sort PS Frame/Series Key: SPARK-47779 URL: https://issues.apache.org/jira/browse/SPARK-47779 Project: Spark Issue Type:

[jira] [Updated] (SPARK-47779) Add a helper function to sort PS Frame/Series

[ https://issues.apache.org/jira/browse/SPARK-47779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-47779: -- Component/s: PS Tests (was: PySpark) > Add a helper

[jira] [Resolved] (SPARK-47772) Fix the doctest of mode function

[ https://issues.apache.org/jira/browse/SPARK-47772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-47772. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45940

[jira] [Resolved] (SPARK-47771) Make max_by, min_by doctests deterministic

[ https://issues.apache.org/jira/browse/SPARK-47771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-47771. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45939

[jira] [Created] (SPARK-47771) Make max_by, min_by doctests deterministic

Ruifeng Zheng created SPARK-47771: - Summary: Make max_by, min_by doctests deterministic Key: SPARK-47771 URL: https://issues.apache.org/jira/browse/SPARK-47771 Project: Spark Issue Type:

[jira] [Created] (SPARK-47749) Dataframe.collect should accept duplicated column names

Ruifeng Zheng created SPARK-47749: - Summary: Dataframe.collect should accept duplicated column names Key: SPARK-47749 URL: https://issues.apache.org/jira/browse/SPARK-47749 Project: Spark

[jira] [Resolved] (SPARK-47669) Add `try_cast` function in DataFrame

[ https://issues.apache.org/jira/browse/SPARK-47669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-47669. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45796

[jira] [Created] (SPARK-47664) Validate the column name with cached schema

Ruifeng Zheng created SPARK-47664: - Summary: Validate the column name with cached schema Key: SPARK-47664 URL: https://issues.apache.org/jira/browse/SPARK-47664 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-47638) Skip column name validation in PS

[ https://issues.apache.org/jira/browse/SPARK-47638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-47638. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45752

[jira] [Assigned] (SPARK-47638) Skip column name validation in PS

[ https://issues.apache.org/jira/browse/SPARK-47638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-47638: - Assignee: Ruifeng Zheng > Skip column name validation in PS >

[jira] [Created] (SPARK-47638) Skip column name validation in PS

Ruifeng Zheng created SPARK-47638: - Summary: Skip column name validation in PS Key: SPARK-47638 URL: https://issues.apache.org/jira/browse/SPARK-47638 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-47621) Refine docstring of `try_sum`, `try_avg`, `avg`, `sum`, `mean`

[ https://issues.apache.org/jira/browse/SPARK-47621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-47621. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45745

[jira] [Resolved] (SPARK-47562) Factor literal handling out of `plan.py`

[ https://issues.apache.org/jira/browse/SPARK-47562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-47562. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45719

[jira] [Assigned] (SPARK-47562) Factor literal handling out of `plan.py`

[ https://issues.apache.org/jira/browse/SPARK-47562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-47562: - Assignee: Ruifeng Zheng > Factor literal handling out of `plan.py` >

[jira] [Created] (SPARK-47562) Factor literal handling out of `plan.py`

Ruifeng Zheng created SPARK-47562: - Summary: Factor literal handling out of `plan.py` Key: SPARK-47562 URL: https://issues.apache.org/jira/browse/SPARK-47562 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-47560) Avoid RPC to validate column name with cached schema

[ https://issues.apache.org/jira/browse/SPARK-47560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-47560. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45717

[jira] [Assigned] (SPARK-47560) Avoid RPC to validate column name with cached schema

[ https://issues.apache.org/jira/browse/SPARK-47560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-47560: - Assignee: Ruifeng Zheng > Avoid RPC to validate column name with cached schema >

[jira] [Created] (SPARK-47560) Avoid RPC to validate column name with cached schema

Ruifeng Zheng created SPARK-47560: - Summary: Avoid RPC to validate column name with cached schema Key: SPARK-47560 URL: https://issues.apache.org/jira/browse/SPARK-47560 Project: Spark Issue

[jira] [Created] (SPARK-47500) Factor column name handling out of `plan.py`

Ruifeng Zheng created SPARK-47500: - Summary: Factor column name handling out of `plan.py` Key: SPARK-47500 URL: https://issues.apache.org/jira/browse/SPARK-47500 Project: Spark Issue Type:

[jira] [Created] (SPARK-47499) Reuse `test_help_command` in Connect

Ruifeng Zheng created SPARK-47499: - Summary: Reuse `test_help_command` in Connect Key: SPARK-47499 URL: https://issues.apache.org/jira/browse/SPARK-47499 Project: Spark Issue Type: Test

[jira] [Assigned] (SPARK-47436) Fix docstring links and type hints in Python Data Source

[ https://issues.apache.org/jira/browse/SPARK-47436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-47436: - Assignee: Hyukjin Kwon > Fix docstring links and type hints in Python Data Source >

[jira] [Resolved] (SPARK-47436) Fix docstring links and type hints in Python Data Source

[ https://issues.apache.org/jira/browse/SPARK-47436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-47436. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45557

[jira] [Created] (SPARK-47437) Correct the error class for `DataFrame.sort`

Ruifeng Zheng created SPARK-47437: - Summary: Correct the error class for `DataFrame.sort` Key: SPARK-47437 URL: https://issues.apache.org/jira/browse/SPARK-47437 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-41762) Make `Column.__neg__` return the same column name as PySpark

[ https://issues.apache.org/jira/browse/SPARK-41762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-41762: - Assignee: Ruifeng Zheng > Make `Column.__neg__` return the same column name as PySpark

[jira] [Resolved] (SPARK-41762) Make `Column.__neg__` return the same column name as PySpark

[ https://issues.apache.org/jira/browse/SPARK-41762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-41762. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45493

[jira] [Created] (SPARK-47377) Factor out tests from `SparkConnectSQLTestCase`

Ruifeng Zheng created SPARK-47377: - Summary: Factor out tests from `SparkConnectSQLTestCase` Key: SPARK-47377 URL: https://issues.apache.org/jira/browse/SPARK-47377 Project: Spark Issue

[jira] [Created] (SPARK-47347) Factor session-related tests out of test_connect_basic

Ruifeng Zheng created SPARK-47347: - Summary: Factor session-related tests out of test_connect_basic Key: SPARK-47347 URL: https://issues.apache.org/jira/browse/SPARK-47347 Project: Spark

[jira] [Created] (SPARK-47334) Make `withColumnRenamed` reuse the implementation of `withColumnsRenamed`

Ruifeng Zheng created SPARK-47334: - Summary: Make `withColumnRenamed` reuse the implementation of `withColumnsRenamed` Key: SPARK-47334 URL: https://issues.apache.org/jira/browse/SPARK-47334 Project:

[jira] [Created] (SPARK-47322) Make `withColumnsRenamed` duplicated column name handling consisten with `withColumnRenamed`

Ruifeng Zheng created SPARK-47322: - Summary: Make `withColumnsRenamed` duplicated column name handling consisten with `withColumnRenamed` Key: SPARK-47322 URL: https://issues.apache.org/jira/browse/SPARK-47322

[jira] [Assigned] (SPARK-46988) proto message abbreviation should support map fields

[ https://issues.apache.org/jira/browse/SPARK-46988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-46988: - Assignee: Ruifeng Zheng > proto message abbreviation should support map fields >

[jira] [Resolved] (SPARK-46988) proto message abbreviation should support map fields

[ https://issues.apache.org/jira/browse/SPARK-46988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-46988. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45413

[jira] [Created] (SPARK-47266) Make `ProtoUtils.abbreviate` return the same type as the input

Ruifeng Zheng created SPARK-47266: - Summary: Make `ProtoUtils.abbreviate` return the same type as the input Key: SPARK-47266 URL: https://issues.apache.org/jira/browse/SPARK-47266 Project: Spark

[jira] [Created] (SPARK-47244) SparkConnectPlanner make internal functions private

Ruifeng Zheng created SPARK-47244: - Summary: SparkConnectPlanner make internal functions private Key: SPARK-47244 URL: https://issues.apache.org/jira/browse/SPARK-47244 Project: Spark Issue

[jira] [Assigned] (SPARK-47201) sameSemantics check input types

[ https://issues.apache.org/jira/browse/SPARK-47201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-47201: - Assignee: Ruifeng Zheng > sameSemantics check input types >

[jira] [Resolved] (SPARK-47201) sameSemantics check input types

[ https://issues.apache.org/jira/browse/SPARK-47201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-47201. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45300

[jira] [Created] (SPARK-47201) sameSemantics check input types

Ruifeng Zheng created SPARK-47201: - Summary: sameSemantics check input types Key: SPARK-47201 URL: https://issues.apache.org/jira/browse/SPARK-47201 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-47183) Fix the error class for sameSemantics

[ https://issues.apache.org/jira/browse/SPARK-47183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-47183: - Assignee: Ruifeng Zheng > Fix the error class for sameSemantics >

[jira] [Resolved] (SPARK-47183) Fix the error class for sameSemantics

[ https://issues.apache.org/jira/browse/SPARK-47183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-47183. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45280

[jira] [Resolved] (SPARK-41873) Implement DataFrame `pandas_api`

[ https://issues.apache.org/jira/browse/SPARK-41873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-41873. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45279

[jira] [Resolved] (SPARK-47184) Make test_repartitionByRange_dataframe reusable

[ https://issues.apache.org/jira/browse/SPARK-47184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-47184. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45281

[jira] [Created] (SPARK-47184) Make test_repartitionByRange_dataframe reusable

Ruifeng Zheng created SPARK-47184: - Summary: Make test_repartitionByRange_dataframe reusable Key: SPARK-47184 URL: https://issues.apache.org/jira/browse/SPARK-47184 Project: Spark Issue

[jira] [Created] (SPARK-47183) Fix the error class for sameSemantics

Ruifeng Zheng created SPARK-47183: - Summary: Fix the error class for sameSemantics Key: SPARK-47183 URL: https://issues.apache.org/jira/browse/SPARK-47183 Project: Spark Issue Type:

[jira] [Updated] (SPARK-46975) Support dedicated fallback methods

[ https://issues.apache.org/jira/browse/SPARK-46975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-46975: -- Summary: Support dedicated fallback methods (was: Move to_{hdf, feather, stata} to the

[jira] [Updated] (SPARK-47129) Make ResolveRelations cache connect plan properly

[ https://issues.apache.org/jira/browse/SPARK-47129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-47129: -- Summary: Make ResolveRelations cache connect plan properly (was: Make ResolveRelations

[jira] [Assigned] (SPARK-43117) proto message abbreviation should support repeated fields

[ https://issues.apache.org/jira/browse/SPARK-43117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-43117: - Assignee: Ruifeng Zheng > proto message abbreviation should support repeated fields >

[jira] [Resolved] (SPARK-43117) proto message abbreviation should support repeated fields

[ https://issues.apache.org/jira/browse/SPARK-43117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-43117. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45056

[jira] [Created] (SPARK-46988) proto message abbreviation should support map fields

Ruifeng Zheng created SPARK-46988: - Summary: proto message abbreviation should support map fields Key: SPARK-46988 URL: https://issues.apache.org/jira/browse/SPARK-46988 Project: Spark Issue

[jira] [Updated] (SPARK-43117) proto message abbreviation should support repeated fields

[ https://issues.apache.org/jira/browse/SPARK-43117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-43117: -- Summary: proto message abbreviation should support repeated fields (was: proto message

[jira] [Created] (SPARK-46987) ProtoUtils.abbreviate avoid unnecessary setField

Ruifeng Zheng created SPARK-46987: - Summary: ProtoUtils.abbreviate avoid unnecessary setField Key: SPARK-46987 URL: https://issues.apache.org/jira/browse/SPARK-46987 Project: Spark Issue

[jira] [Assigned] (SPARK-46976) Implement `DataFrameGroupBy.corr`

[ https://issues.apache.org/jira/browse/SPARK-46976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-46976: - Assignee: Ruifeng Zheng > Implement `DataFrameGroupBy.corr` >

[jira] [Resolved] (SPARK-46976) Implement `DataFrameGroupBy.corr`

[ https://issues.apache.org/jira/browse/SPARK-46976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-46976. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45028

[jira] [Created] (SPARK-46976) Implement `DataFrameGroupBy.corr`

Ruifeng Zheng created SPARK-46976: - Summary: Implement `DataFrameGroupBy.corr` Key: SPARK-46976 URL: https://issues.apache.org/jira/browse/SPARK-46976 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-46975) Move to_{hdf, feather, stata} to the fallback list

Ruifeng Zheng created SPARK-46975: - Summary: Move to_{hdf, feather, stata} to the fallback list Key: SPARK-46975 URL: https://issues.apache.org/jira/browse/SPARK-46975 Project: Spark Issue

[jira] [Resolved] (SPARK-46955) Implement `Frame.to_stata`

[ https://issues.apache.org/jira/browse/SPARK-46955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-46955. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44996

[jira] [Assigned] (SPARK-46955) Implement `Frame.to_stata`

[ https://issues.apache.org/jira/browse/SPARK-46955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-46955: - Assignee: Ruifeng Zheng > Implement `Frame.to_stata` > -- > >

  1   2   3   4   5   6   7   8   9   10   >