[jira] [Resolved] (SPARK-46154) Add a new Daily testing GitHub Action job for Maven with Java 21

2023-11-28 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie resolved SPARK-46154. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44070

[jira] [Updated] (SPARK-46170) Support inject adaptive query post planner strategy rules in SparkSessionExtensions

2023-11-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46170: --- Labels: pull-request-available (was: ) > Support inject adaptive query post planner

[jira] [Created] (SPARK-46170) Support inject adaptive query post planner strategy rules in SparkSessionExtensions

2023-11-28 Thread XiDuo You (Jira)
XiDuo You created SPARK-46170: - Summary: Support inject adaptive query post planner strategy rules in SparkSessionExtensions Key: SPARK-46170 URL: https://issues.apache.org/jira/browse/SPARK-46170

[jira] [Updated] (SPARK-46169) Assign appropriate JIRA numbers to unlabeled TODO items for DataFrame API.

2023-11-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46169: --- Labels: pull-request-available (was: ) > Assign appropriate JIRA numbers to unlabeled TODO

[jira] [Created] (SPARK-46169) Assign appropriate JIRA numbers to unlabeled TODO items for DataFrame API.

2023-11-28 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-46169: --- Summary: Assign appropriate JIRA numbers to unlabeled TODO items for DataFrame API. Key: SPARK-46169 URL: https://issues.apache.org/jira/browse/SPARK-46169 Project:

[jira] [Created] (SPARK-46168) Add axis parameter to DataFrame.idxmin & idxmax

2023-11-28 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-46168: --- Summary: Add axis parameter to DataFrame.idxmin & idxmax Key: SPARK-46168 URL: https://issues.apache.org/jira/browse/SPARK-46168 Project: Spark Issue Type:

[jira] [Created] (SPARK-46167) Add axis, pct and na_option parameter to DataFrame.rank

2023-11-28 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-46167: --- Summary: Add axis, pct and na_option parameter to DataFrame.rank Key: SPARK-46167 URL: https://issues.apache.org/jira/browse/SPARK-46167 Project: Spark Issue

[jira] [Created] (SPARK-46165) Improve axis parameter for DataFrame.all to support columns.

2023-11-28 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-46165: --- Summary: Improve axis parameter for DataFrame.all to support columns. Key: SPARK-46165 URL: https://issues.apache.org/jira/browse/SPARK-46165 Project: Spark

[jira] [Created] (SPARK-46166) Add axis and skipna parameters to DataFrame.any

2023-11-28 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-46166: --- Summary: Add axis and skipna parameters to DataFrame.any Key: SPARK-46166 URL: https://issues.apache.org/jira/browse/SPARK-46166 Project: Spark Issue Type:

[jira] [Created] (SPARK-46164) Add include and exclude parameters for DataFrame.describe

2023-11-28 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-46164: --- Summary: Add include and exclude parameters for DataFrame.describe Key: SPARK-46164 URL: https://issues.apache.org/jira/browse/SPARK-46164 Project: Spark

[jira] [Created] (SPARK-46163) Add filter_func and errors parameter for DataFrame.update

2023-11-28 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-46163: --- Summary: Add filter_func and errors parameter for DataFrame.update Key: SPARK-46163 URL: https://issues.apache.org/jira/browse/SPARK-46163 Project: Spark

[jira] [Created] (SPARK-46160) Add freq and axis parameters to DataFrame.shift

2023-11-28 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-46160: --- Summary: Add freq and axis parameters to DataFrame.shift Key: SPARK-46160 URL: https://issues.apache.org/jira/browse/SPARK-46160 Project: Spark Issue Type:

[jira] [Created] (SPARK-46162) Improve axis parameter for DataFrame.nunique to support columns.

2023-11-28 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-46162: --- Summary: Improve axis parameter for DataFrame.nunique to support columns. Key: SPARK-46162 URL: https://issues.apache.org/jira/browse/SPARK-46162 Project: Spark

[jira] [Created] (SPARK-46161) Improve axis parameter for DataFrame.diff to support columns.

2023-11-28 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-46161: --- Summary: Improve axis parameter for DataFrame.diff to support columns. Key: SPARK-46161 URL: https://issues.apache.org/jira/browse/SPARK-46161 Project: Spark

[jira] [Created] (SPARK-46159) Improve axis parameter for DataFrame.at_time to support columns.

2023-11-28 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-46159: --- Summary: Improve axis parameter for DataFrame.at_time to support columns. Key: SPARK-46159 URL: https://issues.apache.org/jira/browse/SPARK-46159 Project: Spark

[jira] [Assigned] (SPARK-46150) combine python codegen check and protobuf braking change

2023-11-28 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-46150: - Assignee: Ruifeng Zheng > combine python codegen check and protobuf braking change >

[jira] [Resolved] (SPARK-46150) combine python codegen check and protobuf braking change

2023-11-28 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-46150. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44067

[jira] [Created] (SPARK-46158) Improve axis parameter for DataFrame.xs to support columns.

2023-11-28 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-46158: --- Summary: Improve axis parameter for DataFrame.xs to support columns. Key: SPARK-46158 URL: https://issues.apache.org/jira/browse/SPARK-46158 Project: Spark

[jira] [Created] (SPARK-46157) Add `axis` parameter for DataFrame.aggregate.

2023-11-28 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-46157: --- Summary: Add `axis` parameter for DataFrame.aggregate. Key: SPARK-46157 URL: https://issues.apache.org/jira/browse/SPARK-46157 Project: Spark Issue Type:

[jira] [Updated] (SPARK-46154) Add a new Daily testing GitHub Action job for Maven with Java 21

2023-11-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46154: --- Labels: pull-request-available (was: ) > Add a new Daily testing GitHub Action job for

[jira] [Created] (SPARK-46154) Add a new Daily testing GitHub Action job for Maven with Java 21

2023-11-28 Thread Yang Jie (Jira)
Yang Jie created SPARK-46154: Summary: Add a new Daily testing GitHub Action job for Maven with Java 21 Key: SPARK-46154 URL: https://issues.apache.org/jira/browse/SPARK-46154 Project: Spark

[jira] [Updated] (SPARK-46152) XML: Add DecimalType support in schema inference

2023-11-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46152: --- Labels: pull-request-available (was: ) > XML: Add DecimalType support in schema inference

[jira] [Updated] (SPARK-46151) Hide the "More" drop-down button in the PySpark docs navigation bar

2023-11-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46151: --- Labels: pull-request-available (was: ) > Hide the "More" drop-down button in the PySpark

[jira] [Created] (SPARK-46153) XML: Add TimestampNTZType support in schema inference

2023-11-28 Thread Sandip Agarwala (Jira)
Sandip Agarwala created SPARK-46153: --- Summary: XML: Add TimestampNTZType support in schema inference Key: SPARK-46153 URL: https://issues.apache.org/jira/browse/SPARK-46153 Project: Spark

[jira] [Created] (SPARK-46152) XML: Add DecimalType support in schema inference

2023-11-28 Thread Sandip Agarwala (Jira)
Sandip Agarwala created SPARK-46152: --- Summary: XML: Add DecimalType support in schema inference Key: SPARK-46152 URL: https://issues.apache.org/jira/browse/SPARK-46152 Project: Spark Issue

[jira] [Created] (SPARK-46151) Hide the "More" drop-down button in the PySpark docs navigation bar

2023-11-28 Thread BingKun Pan (Jira)
BingKun Pan created SPARK-46151: --- Summary: Hide the "More" drop-down button in the PySpark docs navigation bar Key: SPARK-46151 URL: https://issues.apache.org/jira/browse/SPARK-46151 Project: Spark

[jira] [Updated] (SPARK-46150) combine python codegen check and protobuf braking change

2023-11-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46150: --- Labels: pull-request-available (was: ) > combine python codegen check and protobuf braking

[jira] [Created] (SPARK-46150) combine python codegen check and protobuf braking change

2023-11-28 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-46150: - Summary: combine python codegen check and protobuf braking change Key: SPARK-46150 URL: https://issues.apache.org/jira/browse/SPARK-46150 Project: Spark

[jira] [Assigned] (SPARK-46149) Skip `TorchDistributorLocalUnitTests.test_end_to_end_run_locally` with Python 3.12

2023-11-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-46149: Assignee: Hyukjin Kwon > Skip

[jira] [Resolved] (SPARK-46149) Skip `TorchDistributorLocalUnitTests.test_end_to_end_run_locally` with Python 3.12

2023-11-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-46149. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44065

[jira] [Resolved] (SPARK-46148) Fix pyspark.pandas.mlflow.load_model test (Python 3.12)

2023-11-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-46148. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44064

[jira] [Assigned] (SPARK-46148) Fix pyspark.pandas.mlflow.load_model test (Python 3.12)

2023-11-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-46148: Assignee: Hyukjin Kwon > Fix pyspark.pandas.mlflow.load_model test (Python 3.12) >

[jira] [Resolved] (SPARK-46146) Unpin `markupsafe`

2023-11-28 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-46146. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44062

[jira] [Assigned] (SPARK-46003) Create an ui-test module with Jest

2023-11-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-46003: - Assignee: Kent Yao > Create an ui-test module with Jest >

[jira] [Resolved] (SPARK-46003) Create an ui-test module with Jest

2023-11-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-46003. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43903

[jira] [Resolved] (SPARK-46147) Fix the doctest in pyspark.pandas.series.Series.to_dict (Python 3.12)

2023-11-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-46147. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44063

[jira] [Updated] (SPARK-46140) Remove comments about JVM module options from the test submission options of `HiveExternalCatalogVersionsSuite`

2023-11-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-46140: -- Summary: Remove comments about JVM module options from the test submission options of

[jira] [Assigned] (SPARK-46140) Remove no longer needed JVM module options from the test submission options of `HiveExternalCatalogVersionsSuite`

2023-11-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-46140: - Assignee: Yang Jie > Remove no longer needed JVM module options from the test

[jira] [Resolved] (SPARK-46140) Remove no longer needed JVM module options from the test submission options of `HiveExternalCatalogVersionsSuite`

2023-11-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-46140. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44057

[jira] [Updated] (SPARK-46149) Skip `TorchDistributorLocalUnitTests.test_end_to_end_run_locally` with Python 3.12

2023-11-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46149: --- Labels: pull-request-available (was: ) > Skip

[jira] [Updated] (SPARK-46149) Skip `TorchDistributorLocalUnitTests.test_end_to_end_run_locally` with Python 3.12

2023-11-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-46149: - Description: {code} ==

[jira] [Updated] (SPARK-46149) Skip `TorchDistributorLocalUnitTests.test_end_to_end_run_locally` with Python 3.12

2023-11-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-46149: - Summary: Skip `TorchDistributorLocalUnitTests.test_end_to_end_run_locally` with Python 3.12

[jira] [Updated] (SPARK-46149) Skip `TorchDistributorLocalUnitTests.test_end_to_end_run_locally`

2023-11-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-46149: - Priority: Minor (was: Major) > Skip

[jira] [Created] (SPARK-46149) Skip `TorchDistributorLocalUnitTests.test_end_to_end_run_locally`

2023-11-28 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-46149: Summary: Skip `TorchDistributorLocalUnitTests.test_end_to_end_run_locally` Key: SPARK-46149 URL: https://issues.apache.org/jira/browse/SPARK-46149 Project: Spark

[jira] [Updated] (SPARK-46148) Fix pyspark.pandas.mlflow.load_model test (Python 3.12)

2023-11-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-46148: - Description: {code} ** File

[jira] [Updated] (SPARK-46148) Fix pyspark.pandas.mlflow.load_model test (Python 3.12)

2023-11-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46148: --- Labels: pull-request-available (was: ) > Fix pyspark.pandas.mlflow.load_model test (Python

[jira] [Created] (SPARK-46148) Fix pyspark.pandas.mlflow.load_model test (Python 3.12)

2023-11-28 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-46148: Summary: Fix pyspark.pandas.mlflow.load_model test (Python 3.12) Key: SPARK-46148 URL: https://issues.apache.org/jira/browse/SPARK-46148 Project: Spark

[jira] [Updated] (SPARK-46147) Fix the doctest in pyspark.pandas.series.Series.to_dict (Python 3.12)

2023-11-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46147: --- Labels: pull-request-available (was: ) > Fix the doctest in

[jira] [Updated] (SPARK-46147) Fix the doctest in pyspark.pandas.series.Series.to_dict (Python 3.12)

2023-11-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-46147: - Fix Version/s: (was: 4.0.0) > Fix the doctest in pyspark.pandas.series.Series.to_dict

[jira] [Updated] (SPARK-46147) Fix the doctest in pyspark.pandas.series.Series.to_dict (Python 3.12)

2023-11-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-46147: - Labels: (was: pull-request-available) > Fix the doctest in

[jira] [Updated] (SPARK-46147) Fix the doctest in pyspark.pandas.series.Series.to_dict (Python 3.12)

2023-11-28 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-46147: - Description: {code} File "/__w/spark/spark/python/pyspark/pandas/series.py", line 1633, in

[jira] [Created] (SPARK-46147) Fix the doctest in pyspark.pandas.series.Series.to_dict (Python 3.12)

2023-11-28 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-46147: Summary: Fix the doctest in pyspark.pandas.series.Series.to_dict (Python 3.12) Key: SPARK-46147 URL: https://issues.apache.org/jira/browse/SPARK-46147 Project: Spark

[jira] [Updated] (SPARK-46146) Unpin `markupsafe`

2023-11-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46146: --- Labels: pull-request-available (was: ) > Unpin `markupsafe` > -- > >

[jira] [Created] (SPARK-46146) Unpin `markupsafe`

2023-11-28 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-46146: - Summary: Unpin `markupsafe` Key: SPARK-46146 URL: https://issues.apache.org/jira/browse/SPARK-46146 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-46029) Escape the single quote, _ and % for DS V2 pushdown

2023-11-28 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-46029. - Fix Version/s: 3.5.1 4.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Updated] (SPARK-46108) XML: keepInnerXmlAsRaw option

2023-11-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46108: --- Labels: pull-request-available (was: ) > XML: keepInnerXmlAsRaw option >

[jira] [Resolved] (SPARK-46055) Refactor Catalog Database APIs implementation

2023-11-28 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-46055. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 43959

[jira] [Assigned] (SPARK-46142) Remove `dev/ansible-for-test-node` directory

2023-11-28 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie reassigned SPARK-46142: Assignee: Dongjoon Hyun > Remove `dev/ansible-for-test-node` directory >

[jira] [Resolved] (SPARK-46142) Remove `dev/ansible-for-test-node` directory

2023-11-28 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie resolved SPARK-46142. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44059

[jira] [Updated] (SPARK-46145) spark.catalog.listTables does not throw exception when the table or view is not found

2023-11-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46145: --- Labels: pull-request-available (was: ) > spark.catalog.listTables does not throw exception

[jira] [Created] (SPARK-46145) spark.catalog.listTables does not throw exception when the table or view is not found

2023-11-28 Thread Rui Wang (Jira)
Rui Wang created SPARK-46145: Summary: spark.catalog.listTables does not throw exception when the table or view is not found Key: SPARK-46145 URL: https://issues.apache.org/jira/browse/SPARK-46145

[jira] [Updated] (SPARK-46144) Fail INSERT INTO ... REPLACE statement if the condition contains subquery

2023-11-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46144: --- Labels: pull-request-available (was: ) > Fail INSERT INTO ... REPLACE statement if the

[jira] [Updated] (SPARK-46144) Fail INSERT INTO ... REPLACE statement if the condition contains subquery

2023-11-28 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-46144: --- Description: For the following query: {code:java} INSERT INTO tbl REPLACE WHERE id =

[jira] [Updated] (SPARK-46144) Fail INSERT INTO ... REPLACE statement if the condition contains subquery

2023-11-28 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-46144: --- Description: For the following query: {code:java} INSERT INTO tbl REPLACE WHERE id =

[jira] [Updated] (SPARK-46144) Fail INSERT INTO ... REPLACE statement if the condition contains subquery

2023-11-28 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-46144: --- Affects Version/s: 4.0.0 (was: 3.5.0) > Fail INSERT INTO ...

[jira] [Created] (SPARK-46144) Fail INSERT INTO ... REPLACE statement if the condition contains subquery

2023-11-28 Thread Gengliang Wang (Jira)
Gengliang Wang created SPARK-46144: -- Summary: Fail INSERT INTO ... REPLACE statement if the condition contains subquery Key: SPARK-46144 URL: https://issues.apache.org/jira/browse/SPARK-46144

[jira] [Commented] (SPARK-46125) Memory leak when using createDataFrame with persist

2023-11-28 Thread Josh Rosen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17790751#comment-17790751 ] Josh Rosen commented on SPARK-46125: I think that this issue relates specifically to

[jira] [Updated] (SPARK-46125) Memory leak when using createDataFrame with persist

2023-11-28 Thread Josh Rosen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-46125: --- Attachment: image-2023-11-28-12-55-58-461.png > Memory leak when using createDataFrame with persist

[jira] [Commented] (SPARK-46105) df.emptyDataFrame shows 1 if we repartition(1) in Spark 3.3.x and above

2023-11-28 Thread Josh Rosen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17790742#comment-17790742 ] Josh Rosen commented on SPARK-46105: {quote}The reason for raising this as a bug is I have a

[jira] [Updated] (SPARK-46143) pyspark.pandas read_excel implementation at version 3.4.1

2023-11-28 Thread Matheus Pavanetti (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matheus Pavanetti updated SPARK-46143: -- Environment: pyspark 3.4.1.5.3 build 20230713. Running on Microsoft Fabric

[jira] [Updated] (SPARK-46143) pyspark.pandas read_excel implementation at version 3.4.1

2023-11-28 Thread Matheus Pavanetti (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matheus Pavanetti updated SPARK-46143: -- Description: Hello,  I would like to report an issue with pyspark.pandas

[jira] [Updated] (SPARK-46143) pyspark.pandas read_excel implementation at version 3.4.1

2023-11-28 Thread Matheus Pavanetti (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matheus Pavanetti updated SPARK-46143: -- Attachment: image-2023-11-28-13-20-51-291.png > pyspark.pandas read_excel

[jira] [Updated] (SPARK-46143) pyspark.pandas read_excel implementation at version 3.4.1

2023-11-28 Thread Matheus Pavanetti (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matheus Pavanetti updated SPARK-46143: -- Attachment: image-2023-11-28-13-20-40-275.png > pyspark.pandas read_excel

[jira] [Updated] (SPARK-46143) pyspark.pandas read_excel implementation at version 3.4.1

2023-11-28 Thread Matheus Pavanetti (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matheus Pavanetti updated SPARK-46143: -- Description: Hello,  I would like to report an issue with pyspark.pandas

[jira] [Updated] (SPARK-46143) pyspark.pandas read_excel implementation at version 3.4.1

2023-11-28 Thread Matheus Pavanetti (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matheus Pavanetti updated SPARK-46143: -- Attachment: MicrosoftTeams-image.png > pyspark.pandas read_excel implementation at

[jira] [Created] (SPARK-46143) pyspark.pandas read_excel implementation at version 3.4.1

2023-11-28 Thread Matheus Pavanetti (Jira)
Matheus Pavanetti created SPARK-46143: - Summary: pyspark.pandas read_excel implementation at version 3.4.1 Key: SPARK-46143 URL: https://issues.apache.org/jira/browse/SPARK-46143 Project: Spark

[jira] [Resolved] (SPARK-46134) Replace `df.take(1).toSeq(0)` with `df.first()` in `ProtobufFunctionsSuite`

2023-11-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-46134. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44048

[jira] [Assigned] (SPARK-46134) Replace `df.take(1).toSeq(0)` with `df.first()` in `ProtobufFunctionsSuite`

2023-11-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-46134: - Assignee: Yang Jie > Replace `df.take(1).toSeq(0)` with `df.first()` in

[jira] [Updated] (SPARK-46142) Remove `dev/ansible-for-test-node` directory

2023-11-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46142: --- Labels: pull-request-available (was: ) > Remove `dev/ansible-for-test-node` directory >

[jira] [Created] (SPARK-46142) Remove `dev/ansible-for-test-node` directory

2023-11-28 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-46142: - Summary: Remove `dev/ansible-for-test-node` directory Key: SPARK-46142 URL: https://issues.apache.org/jira/browse/SPARK-46142 Project: Spark Issue Type:

[jira] [Updated] (SPARK-46098) Reduce stack depth by replace (string|array).size with (string|array).length

2023-11-28 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-46098: - Priority: Minor (was: Major) > Reduce stack depth by replace (string|array).size with

[jira] [Resolved] (SPARK-46098) Reduce stack depth by replace (string|array).size with (string|array).length

2023-11-28 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-46098. -- Fix Version/s: 4.0.0 Resolution: Fixed > Reduce stack depth by replace

[jira] [Resolved] (SPARK-46101) Replace (string|array).size with (string|array).length in all the modules

2023-11-28 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-46101. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44016

[jira] [Updated] (SPARK-46141) Change default of spark.sql.legacy.ctePrecedencePolicy from EXCEPTION to CORRECTED

2023-11-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46141: --- Labels: pull-request-available (was: ) > Change default of

[jira] [Assigned] (SPARK-46129) Add GitHub link icon to PySpark documentation header

2023-11-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-46129: - Assignee: Haejoon Lee > Add GitHub link icon to PySpark documentation header >

[jira] [Resolved] (SPARK-46129) Add GitHub link icon to PySpark documentation header

2023-11-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-46129. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44043

[jira] [Assigned] (SPARK-46119) Override toString method for UnresolvedAlias

2023-11-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-46119: - Assignee: Yuming Wang > Override toString method for UnresolvedAlias >

[jira] [Resolved] (SPARK-46119) Override toString method for UnresolvedAlias

2023-11-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-46119. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44038

[jira] [Assigned] (SPARK-46139) Fix QueryExecutionErrorsSuite with Java 21

2023-11-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-46139: - Assignee: Yang Jie > Fix QueryExecutionErrorsSuite with Java 21 >

[jira] [Resolved] (SPARK-46139) Fix QueryExecutionErrorsSuite with Java 21

2023-11-28 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-46139. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44056

[jira] [Created] (SPARK-46141) Change default of spark.sql.legacy.ctePrecedencePolicy from EXCEPTION to CORRECTED

2023-11-28 Thread Serge Rielau (Jira)
Serge Rielau created SPARK-46141: Summary: Change default of spark.sql.legacy.ctePrecedencePolicy from EXCEPTION to CORRECTED Key: SPARK-46141 URL: https://issues.apache.org/jira/browse/SPARK-46141

[jira] (SPARK-43106) Data lost from the table if the INSERT OVERWRITE query fails

2023-11-28 Thread jeanlyn (Jira)
[ https://issues.apache.org/jira/browse/SPARK-43106 ] jeanlyn deleted comment on SPARK-43106: - was (Author: jeanlyn): I think we also encountered similar problems, we circumvent this problem by using parameters

[jira] [Updated] (SPARK-46140) Remove no longer needed JVM module options from the test submission options of `HiveExternalCatalogVersionsSuite`

2023-11-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46140: --- Labels: pull-request-available (was: ) > Remove no longer needed JVM module options from

[jira] [Created] (SPARK-46140) Remove no longer needed JVM module options from the test submission options of `HiveExternalCatalogVersionsSuite`

2023-11-28 Thread Yang Jie (Jira)
Yang Jie created SPARK-46140: Summary: Remove no longer needed JVM module options from the test submission options of `HiveExternalCatalogVersionsSuite` Key: SPARK-46140 URL:

[jira] [Updated] (SPARK-46139) Fix QueryExecutionErrorsSuite with Java 21

2023-11-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46139: --- Labels: pull-request-available (was: ) > Fix QueryExecutionErrorsSuite with Java 21 >

[jira] [Created] (SPARK-46139) Fix QueryExecutionErrorsSuite with Java 21

2023-11-28 Thread Yang Jie (Jira)
Yang Jie created SPARK-46139: Summary: Fix QueryExecutionErrorsSuite with Java 21 Key: SPARK-46139 URL: https://issues.apache.org/jira/browse/SPARK-46139 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-46138) Clean up the use of `SQLContext` in hive-thriftserver module

2023-11-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46138: --- Labels: pull-request-available (was: ) > Clean up the use of `SQLContext` in

[jira] [Created] (SPARK-46138) Clean up the use of `SQLContext` in hive-thriftserver module

2023-11-28 Thread Yang Jie (Jira)
Yang Jie created SPARK-46138: Summary: Clean up the use of `SQLContext` in hive-thriftserver module Key: SPARK-46138 URL: https://issues.apache.org/jira/browse/SPARK-46138 Project: Spark Issue

[jira] [Updated] (SPARK-46137) Janino compiler has new version that fix issue with compilation

2023-11-28 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/SPARK-46137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46137: --- Labels: pull-request-available (was: ) > Janino compiler has new version that fix issue

[jira] [Created] (SPARK-46137) Janino compiler has new version that fix issue with compilation

2023-11-28 Thread Izek Greenfield (Jira)
Izek Greenfield created SPARK-46137: --- Summary: Janino compiler has new version that fix issue with compilation Key: SPARK-46137 URL: https://issues.apache.org/jira/browse/SPARK-46137 Project: Spark

  1   2   >