[jira] [Updated] (SPARK-47540) SPIP: Pure Python Package (Spark Connect)

2024-04-01 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-47540: - Description: *Q1. What are you trying to do? Articulate your objectives using absolutely no

[jira] [Updated] (SPARK-47540) SPIP: Pure Python Package (Spark Connect)

2024-04-01 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-47540: - Description: *Q1. What are you trying to do? Articulate your objectives using absolutely no

[jira] [Updated] (SPARK-47646) try_to_number fails with NPE for malformed input

2024-03-30 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-47646: - Fix Version/s: 3.4.3 > try_to_number fails with NPE for malformed input >

[jira] [Assigned] (SPARK-47646) try_to_number fails with NPE for malformed input

2024-03-29 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-47646: Assignee: Hyukjin Kwon > try_to_number fails with NPE for malformed input >

[jira] [Resolved] (SPARK-47646) try_to_number fails with NPE for malformed input

2024-03-29 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47646. -- Fix Version/s: 3.5.2 4.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-47644) Refine docstrings of try_*

2024-03-29 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-47644: Assignee: Hyukjin Kwon > Refine docstrings of try_* > -- > >

[jira] [Resolved] (SPARK-47644) Refine docstrings of try_*

2024-03-29 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47644. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45769

[jira] [Created] (SPARK-47646) try_to_number fails with NPE for malformed input

2024-03-29 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-47646: Summary: try_to_number fails with NPE for malformed input Key: SPARK-47646 URL: https://issues.apache.org/jira/browse/SPARK-47646 Project: Spark Issue Type:

[jira] [Created] (SPARK-47644) Improve docstrings of try_*

2024-03-28 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-47644: Summary: Improve docstrings of try_* Key: SPARK-47644 URL: https://issues.apache.org/jira/browse/SPARK-47644 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-47644) Refine docstrings of try_*

2024-03-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-47644: - Summary: Refine docstrings of try_* (was: Improve docstrings of try_*) > Refine docstrings of

[jira] [Resolved] (SPARK-47623) Use `QuietTest` in parity tests

2024-03-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47623. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45747

[jira] [Resolved] (SPARK-47631) Remove unused `SQLConf.parquetOutputCommitterClass` method

2024-03-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47631. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45757

[jira] [Assigned] (SPARK-47631) Remove unused `SQLConf.parquetOutputCommitterClass` method

2024-03-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-47631: Assignee: Dongjoon Hyun > Remove unused `SQLConf.parquetOutputCommitterClass` method >

[jira] [Created] (SPARK-47621) Refine docstring of `try_sum`, `try_avg`, `avg`, `sum`, `mean`

2024-03-27 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-47621: Summary: Refine docstring of `try_sum`, `try_avg`, `avg`, `sum`, `mean` Key: SPARK-47621 URL: https://issues.apache.org/jira/browse/SPARK-47621 Project: Spark

[jira] [Resolved] (SPARK-47619) Refine docstring of `to_json/from_json`

2024-03-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47619. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45742

[jira] [Created] (SPARK-47619) Refine docstring of `to_json/from_json`

2024-03-27 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-47619: Summary: Refine docstring of `to_json/from_json` Key: SPARK-47619 URL: https://issues.apache.org/jira/browse/SPARK-47619 Project: Spark Issue Type: Sub-task

[jira] [Resolved] (SPARK-47543) Inferring `dict` as `MapType` from Pandas DataFrame to allow DataFrame creation.

2024-03-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47543. -- Assignee: Haejoon Lee Resolution: Fixed Fixed in

[jira] [Updated] (SPARK-47543) Inferring `dict` as `MapType` from Pandas DataFrame to allow DataFrame creation.

2024-03-27 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-47543: - Fix Version/s: 4.0.0 > Inferring `dict` as `MapType` from Pandas DataFrame to allow DataFrame

[jira] [Assigned] (SPARK-47367) Support Python data source API with Spark Connect

2024-03-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-47367: Assignee: Allison Wang > Support Python data source API with Spark Connect >

[jira] [Resolved] (SPARK-47367) Support Python data source API with Spark Connect

2024-03-26 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47367. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45486

[jira] [Resolved] (SPARK-47450) Use R 4.3.3 in `windows` R GitHub Action job

2024-03-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47450. -- Fix Version/s: 4.0.0 Assignee: Dongjoon Hyun Resolution: Fixed Reverted the

[jira] [Updated] (SPARK-47450) Use R 4.3.3 in `windows` R GitHub Action job

2024-03-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-47450: - Fix Version/s: (was: 4.0.0) > Use R 4.3.3 in `windows` R GitHub Action job >

[jira] [Resolved] (SPARK-47554) Upgrade `sbt-assembly` to `2.2.0` and `sbt-protoc` to `1.0.7`

2024-03-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47554. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45696

[jira] [Assigned] (SPARK-47549) Remove Spark 3.0~3.2 pyspark/version.py workaround from release scripts

2024-03-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-47549: Assignee: Dongjoon Hyun > Remove Spark 3.0~3.2 pyspark/version.py workaround from

[jira] [Resolved] (SPARK-47549) Remove Spark 3.0~3.2 pyspark/version.py workaround from release scripts

2024-03-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47549. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45706

[jira] [Assigned] (SPARK-47539) Make the return value of method `castToString` be `Any => UTF8String`

2024-03-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-47539: Assignee: BingKun Pan > Make the return value of method `castToString` be `Any =>

[jira] [Resolved] (SPARK-47539) Make the return value of method `castToString` be `Any => UTF8String`

2024-03-25 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47539. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45688

[jira] [Created] (SPARK-47540) SPIP: Pure Python Package (Spark Connect)

2024-03-25 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-47540: Summary: SPIP: Pure Python Package (Spark Connect) Key: SPARK-47540 URL: https://issues.apache.org/jira/browse/SPARK-47540 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-47516) Move `remove unused installation package logic` from `each test job` to `create the docker image`

2024-03-24 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-47516: Assignee: BingKun Pan > Move `remove unused installation package logic` from `each test

[jira] [Resolved] (SPARK-47516) Move `remove unused installation package logic` from `each test job` to `create the docker image`

2024-03-24 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47516. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45659

[jira] [Assigned] (SPARK-44708) Migrate test_reset_index assert_eq to use assertDataFrameEqual

2024-03-24 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-44708: Assignee: Amanda Liu > Migrate test_reset_index assert_eq to use assertDataFrameEqual >

[jira] [Resolved] (SPARK-44708) Migrate test_reset_index assert_eq to use assertDataFrameEqual

2024-03-24 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-44708. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45629

[jira] [Assigned] (SPARK-47500) Factor column name handling out of `plan.py`

2024-03-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-47500: Assignee: Ruifeng Zheng > Factor column name handling out of `plan.py` >

[jira] [Resolved] (SPARK-47500) Factor column name handling out of `plan.py`

2024-03-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47500. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45636

[jira] [Resolved] (SPARK-47478) Improve test coverage for mysql bool synonyms

2024-03-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47478. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45604

[jira] [Assigned] (SPARK-47478) Improve test coverage for mysql bool synonyms

2024-03-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-47478: Assignee: Kent Yao > Improve test coverage for mysql bool synonyms >

[jira] [Resolved] (SPARK-47480) Enable doctest for `createDataFrame`

2024-03-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47480. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45606

[jira] [Resolved] (SPARK-41625) Feature parity: Streaming support

2024-03-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-41625. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45603

[jira] [Resolved] (SPARK-47330) XML: Add XmlExpressionsSuite

2024-03-19 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47330. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45445

[jira] [Assigned] (SPARK-47330) XML: Add XmlExpressionsSuite

2024-03-19 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-47330: Assignee: Yousof Hosny > XML: Add XmlExpressionsSuite > - >

[jira] [Resolved] (SPARK-47454) Split `pyspark.sql.tests.test_dataframe`

2024-03-19 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47454. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45580

[jira] [Assigned] (SPARK-47437) Correct the error class for `DataFrame.sort`

2024-03-18 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-47437: Assignee: Ruifeng Zheng > Correct the error class for `DataFrame.sort` >

[jira] [Resolved] (SPARK-47437) Correct the error class for `DataFrame.sort`

2024-03-18 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47437. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45559

[jira] [Resolved] (SPARK-43435) re-enable doctest `pyspark.sql.connect.dataframe.DataFrame.writeStream`

2024-03-18 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-43435. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45560

[jira] [Resolved] (SPARK-47439) Document Python Data Source API in API reference page

2024-03-18 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47439. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45561

[jira] [Assigned] (SPARK-47439) Document Python Data Source API in API reference page

2024-03-18 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-47439: Assignee: Hyukjin Kwon > Document Python Data Source API in API reference page >

[jira] [Updated] (SPARK-47439) Document Python Data Source API in API reference page

2024-03-18 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-47439: - Summary: Document Python Data Source API in API reference page (was: Document Python Data

[jira] [Created] (SPARK-47439) Document Python Data Source API

2024-03-18 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-47439: Summary: Document Python Data Source API Key: SPARK-47439 URL: https://issues.apache.org/jira/browse/SPARK-47439 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-47436) Fix docstring links and type hints in Python Data Source

2024-03-17 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-47436: Summary: Fix docstring links and type hints in Python Data Source Key: SPARK-47436 URL: https://issues.apache.org/jira/browse/SPARK-47436 Project: Spark

[jira] [Resolved] (SPARK-47374) Fix connect-repl `usage prompt` & `docs link`

2024-03-14 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47374. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45494

[jira] [Assigned] (SPARK-47374) Fix connect-repl `usage prompt` & `docs link`

2024-03-14 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-47374: Assignee: BingKun Pan > Fix connect-repl `usage prompt` & `docs link` >

[jira] [Assigned] (SPARK-47377) Factor out tests from `SparkConnectSQLTestCase`

2024-03-14 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-47377: Assignee: Ruifeng Zheng > Factor out tests from `SparkConnectSQLTestCase` >

[jira] [Resolved] (SPARK-47377) Factor out tests from `SparkConnectSQLTestCase`

2024-03-14 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47377. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45497

[jira] [Assigned] (SPARK-46761) quoted strings in a JSON path should support ? characters

2024-03-13 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-46761: Assignee: Robert Joseph Evans > quoted strings in a JSON path should support ?

[jira] [Resolved] (SPARK-46761) quoted strings in a JSON path should support ? characters

2024-03-13 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-46761. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45420

[jira] [Resolved] (SPARK-47348) Align the version of the ubuntu host machine where the GitHub workflow runs on

2024-03-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47348. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45416

[jira] [Assigned] (SPARK-47348) Align the version of the ubuntu host machine where the GitHub workflow runs on

2024-03-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-47348: Assignee: BingKun Pan > Align the version of the ubuntu host machine where the GitHub

[jira] [Resolved] (SPARK-47347) Factor session-related tests out of test_connect_basic

2024-03-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47347. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45472

[jira] [Assigned] (SPARK-47347) Factor session-related tests out of test_connect_basic

2024-03-12 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-47347: Assignee: Ruifeng Zheng > Factor session-related tests out of test_connect_basic >

[jira] [Assigned] (SPARK-47327) Fix thread safety issue in ICU Collator

2024-03-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-47327: Assignee: Stefan Kandic > Fix thread safety issue in ICU Collator >

[jira] [Resolved] (SPARK-47327) Fix thread safety issue in ICU Collator

2024-03-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47327. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45436

[jira] [Resolved] (SPARK-47324) Call convertJavaTimestampToTimeStamp in Array getter

2024-03-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47324. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45435

[jira] [Assigned] (SPARK-47324) Call convertJavaTimestampToTimeStamp in Array getter

2024-03-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-47324: Assignee: Kent Yao > Call convertJavaTimestampToTimeStamp in Array getter >

[jira] [Resolved] (SPARK-47334) Make `withColumnRenamed` reuse the implementation of `withColumnsRenamed`

2024-03-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47334. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45450

[jira] [Assigned] (SPARK-47334) Make `withColumnRenamed` reuse the implementation of `withColumnsRenamed`

2024-03-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-47334: Assignee: Ruifeng Zheng > Make `withColumnRenamed` reuse the implementation of

[jira] [Assigned] (SPARK-47325) Use the latest buf-setup-action in github workflow

2024-03-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-47325: Assignee: BingKun Pan > Use the latest buf-setup-action in github workflow >

[jira] [Resolved] (SPARK-47325) Use the latest buf-setup-action in github workflow

2024-03-10 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47325. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45433

[jira] [Resolved] (SPARK-47322) Make `withColumnsRenamed` duplicated column name handling consisten with `withColumnRenamed`

2024-03-08 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47322. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45431

[jira] [Assigned] (SPARK-47322) Make `withColumnsRenamed` duplicated column name handling consisten with `withColumnRenamed`

2024-03-08 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-47322: Assignee: Ruifeng Zheng > Make `withColumnsRenamed` duplicated column name handling

[jira] [Resolved] (SPARK-47309) [XML] Add schema inference unit tests

2024-03-07 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47309. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45426

[jira] [Assigned] (SPARK-47309) [XML] Add schema inference unit tests

2024-03-07 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-47309: Assignee: Shujing Yang > [XML] Add schema inference unit tests >

[jira] [Assigned] (SPARK-47078) Documentation for SparkSession-based Profilers

2024-03-07 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-47078: Assignee: Xinrong Meng > Documentation for SparkSession-based Profilers >

[jira] [Resolved] (SPARK-47078) Documentation for SparkSession-based Profilers

2024-03-07 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47078. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45269

[jira] [Commented] (SPARK-47288) DataType __repr__ change breaks datatype checking (anit-)pattern

2024-03-07 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17824577#comment-17824577 ] Hyukjin Kwon commented on SPARK-47288: -- I agree in principle but my concern is more about backward

[jira] [Assigned] (SPARK-47155) Fix incorrect error class in create_data_source.py

2024-03-06 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-47155: Assignee: Chaoqin Li > Fix incorrect error class in create_data_source.py >

[jira] [Resolved] (SPARK-47155) Fix incorrect error class in create_data_source.py

2024-03-06 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47155. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45306

[jira] [Resolved] (SPARK-47311) Suppress Python exceptions where PySpark is not in the Python path

2024-03-06 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47311. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45414

[jira] [Assigned] (SPARK-47311) Suppress Python exceptions where PySpark is not in the Python path

2024-03-06 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-47311: Assignee: Hyukjin Kwon > Suppress Python exceptions where PySpark is not in the Python

[jira] [Updated] (SPARK-47311) Suppress Python exceptions where PySpark is not in the Python path

2024-03-06 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-47311: - Description: {code} scala> sql("create table t(i int) using json") 24/03/06 16:09:44 WARN

[jira] [Updated] (SPARK-47311) Suppress Python exceptions where PySpark is not in the Python path

2024-03-06 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-47311: - Summary: Suppress Python exceptions where PySpark is not in the Python path (was: Surpress

[jira] [Updated] (SPARK-47311) Surpress Python exceptions where PySpark is not in the Python path

2024-03-06 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-47311: - Priority: Minor (was: Major) > Surpress Python exceptions where PySpark is not in the Python

[jira] [Created] (SPARK-47311) Surpress Python exceptions where PySpark is not in the Python path

2024-03-06 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-47311: Summary: Surpress Python exceptions where PySpark is not in the Python path Key: SPARK-47311 URL: https://issues.apache.org/jira/browse/SPARK-47311 Project: Spark

[jira] [Assigned] (SPARK-47271) Explain importance of statistics on SQL performance tuning page

2024-03-06 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-47271: Assignee: Nicholas Chammas > Explain importance of statistics on SQL performance tuning

[jira] [Resolved] (SPARK-47271) Explain importance of statistics on SQL performance tuning page

2024-03-06 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47271. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45374

[jira] [Assigned] (SPARK-47303) Restructuring MasterSuite

2024-03-06 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-47303: Assignee: Hyukjin Kwon > Restructuring MasterSuite > - > >

[jira] [Resolved] (SPARK-47303) Restructuring MasterSuite

2024-03-06 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47303. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45366

[jira] [Created] (SPARK-47303) Restructuring MasterSuite

2024-03-06 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-47303: Summary: Restructuring MasterSuite Key: SPARK-47303 URL: https://issues.apache.org/jira/browse/SPARK-47303 Project: Spark Issue Type: Test

[jira] [Resolved] (SPARK-44746) Improve the documentation for TABLE input arguments for UDTFs

2024-03-05 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-44746. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45375

[jira] [Assigned] (SPARK-44746) Improve the documentation for TABLE input arguments for UDTFs

2024-03-05 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-44746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-44746: Assignee: Daniel > Improve the documentation for TABLE input arguments for UDTFs >

[jira] [Resolved] (SPARK-47277) PySpark util function assertDataFrameEqual should not support streaming DF

2024-03-05 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47277. -- Fix Version/s: 4.0.0 Resolution: Fixed Fixed in

[jira] [Resolved] (SPARK-47252) Clarify that pivot may trigger an eager computation

2024-03-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47252. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45363

[jira] [Assigned] (SPARK-47252) Clarify that pivot may trigger an eager computation

2024-03-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-47252: Assignee: Nicholas Chammas > Clarify that pivot may trigger an eager computation >

[jira] [Resolved] (SPARK-47266) Make `ProtoUtils.abbreviate` return the same type as the input

2024-03-04 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47266. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45369

[jira] [Assigned] (SPARK-47244) SparkConnectPlanner make internal functions private

2024-03-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-47244: Assignee: Ruifeng Zheng > SparkConnectPlanner make internal functions private >

[jira] [Resolved] (SPARK-47244) SparkConnectPlanner make internal functions private

2024-03-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47244. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45354

[jira] [Resolved] (SPARK-47251) Block invalid types from the `args` argument for `sql` command

2024-03-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47251. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45361

[jira] [Assigned] (SPARK-47251) Block invalid types from the `args` argument for `sql` command

2024-03-02 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-47251: Assignee: Takuya Ueshin > Block invalid types from the `args` argument for `sql` command

[jira] [Resolved] (SPARK-47218) XML: Ignore commented Row Tags in XML tokenizer

2024-02-29 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-47218. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 45325

[jira] [Assigned] (SPARK-47218) XML: Ignore commented Row Tags in XML tokenizer

2024-02-29 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-47218: Assignee: Yousof Hosny > XML: Ignore commented Row Tags in XML tokenizer >

[jira] [Assigned] (SPARK-47211) Fix ignored PySpark Connect string collation

2024-02-29 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-47211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-47211: Assignee: Nikola Mandic > Fix ignored PySpark Connect string collation >

<    1   2   3   4   5   6   7   8   9   10   >