[jira] [Resolved] (SPARK-46871) Clean up the imports in `pyspark.pandas.tests.computation.*`

[ https://issues.apache.org/jira/browse/SPARK-46871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-46871. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44895

[jira] [Updated] (SPARK-46873) PySpark spark.streams should not recreate new StreamingQueryManager

[ https://issues.apache.org/jira/browse/SPARK-46873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46873: --- Labels: pull-request-available (was: ) > PySpark spark.streams should not recreate new

[jira] [Updated] (SPARK-46855) Add `sketch` to the dependencies of the `catalyst` module in `module.py`

[ https://issues.apache.org/jira/browse/SPARK-46855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-46855: -- Fix Version/s: 3.4.3 > Add `sketch` to the dependencies of the `catalyst` module in

[jira] [Created] (SPARK-46873) PySpark spark.streams should not recreate new StreamingQueryManager

Wei Liu created SPARK-46873: --- Summary: PySpark spark.streams should not recreate new StreamingQueryManager Key: SPARK-46873 URL: https://issues.apache.org/jira/browse/SPARK-46873 Project: Spark

[jira] [Updated] (SPARK-46855) Add `sketch` to the dependencies of the `catalyst` module in `module.py`

[ https://issues.apache.org/jira/browse/SPARK-46855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-46855: -- Fix Version/s: 3.5.1 > Add `sketch` to the dependencies of the `catalyst` module in

[jira] [Updated] (SPARK-46872) Recover `log-view.js` to be non-module

[ https://issues.apache.org/jira/browse/SPARK-46872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-46872: -- Summary: Recover `log-view.js` to be non-module (was: Fix `log-view.js` to be non-module) >

[jira] [Resolved] (SPARK-46872) Fix `log-view.js` to be non-module

[ https://issues.apache.org/jira/browse/SPARK-46872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-46872. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44896

[jira] [Assigned] (SPARK-46872) Fix `log-view.js` to be non-module

[ https://issues.apache.org/jira/browse/SPARK-46872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-46872: - Assignee: Dongjoon Hyun > Fix `log-view.js` to be non-module >

[jira] [Updated] (SPARK-46872) Fix `log-view.js` to be non-module

[ https://issues.apache.org/jira/browse/SPARK-46872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46872: --- Labels: pull-request-available (was: ) > Fix `log-view.js` to be non-module >

[jira] [Created] (SPARK-46872) Fix `log-view.js` to be non-module

Dongjoon Hyun created SPARK-46872: - Summary: Fix `log-view.js` to be non-module Key: SPARK-46872 URL: https://issues.apache.org/jira/browse/SPARK-46872 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-46855) Add `sketch` to the dependencies of the `catalyst` module in `module.py`

[ https://issues.apache.org/jira/browse/SPARK-46855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-46855: - Issue Type: Bug (was: Improvement) > Add `sketch` to the dependencies of the `catalyst` module in

[jira] [Resolved] (SPARK-46870) Support Spark Master Log UI

[ https://issues.apache.org/jira/browse/SPARK-46870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-46870. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44890

[jira] [Updated] (SPARK-46855) Add `sketch` to the dependencies of the `catalyst` module in `module.py`

[ https://issues.apache.org/jira/browse/SPARK-46855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-46855: - Affects Version/s: 3.5.0 3.4.2 > Add `sketch` to the dependencies of the

[jira] [Assigned] (SPARK-46870) Support Spark Master Log UI

[ https://issues.apache.org/jira/browse/SPARK-46870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-46870: - Assignee: Dongjoon Hyun > Support Spark Master Log UI > --- >

[jira] [Resolved] (SPARK-46868) Support Spark Worker Log UI

[ https://issues.apache.org/jira/browse/SPARK-46868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-46868. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44888

[jira] [Resolved] (SPARK-46863) Clean up custom.css

[ https://issues.apache.org/jira/browse/SPARK-46863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-46863. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44885

[jira] [Updated] (SPARK-46432) Upgrade Netty to 4.1.106.Final

[ https://issues.apache.org/jira/browse/SPARK-46432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-46432: - Summary: Upgrade Netty to 4.1.106.Final (was: Upgrade Netty to 4.1.104.Final) > Upgrade Netty to

[jira] [Resolved] (SPARK-46432) Upgrade Netty to 4.1.104.Final

[ https://issues.apache.org/jira/browse/SPARK-46432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie resolved SPARK-46432. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44384

[jira] [Assigned] (SPARK-46432) Upgrade Netty to 4.1.104.Final

[ https://issues.apache.org/jira/browse/SPARK-46432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie reassigned SPARK-46432: Assignee: BingKun Pan > Upgrade Netty to 4.1.104.Final > -- > >

[jira] [Updated] (SPARK-46870) Support Spark Master Log UI

[ https://issues.apache.org/jira/browse/SPARK-46870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46870: --- Labels: pull-request-available (was: ) > Support Spark Master Log UI >

[jira] [Created] (SPARK-46870) Support Spark Master Log UI

Dongjoon Hyun created SPARK-46870: - Summary: Support Spark Master Log UI Key: SPARK-46870 URL: https://issues.apache.org/jira/browse/SPARK-46870 Project: Spark Issue Type: Sub-task

[jira] [Resolved] (SPARK-46869) Add `logrotate` to Spark docker files

[ https://issues.apache.org/jira/browse/SPARK-46869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-46869. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44889

[jira] [Updated] (SPARK-46869) Add `logrotate` to Spark docker files

[ https://issues.apache.org/jira/browse/SPARK-46869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-46869: -- Summary: Add `logrotate` to Spark docker files (was: Add `logrotate` to to Spark docker

[jira] [Updated] (SPARK-46869) Add `logrotate` to to Spark docker files

[ https://issues.apache.org/jira/browse/SPARK-46869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46869: --- Labels: pull-request-available (was: ) > Add `logrotate` to to Spark docker files >

[jira] [Assigned] (SPARK-46869) Add `logrotate` to to Spark docker files

[ https://issues.apache.org/jira/browse/SPARK-46869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-46869: - Assignee: Dongjoon Hyun > Add `logrotate` to to Spark docker files >

[jira] [Created] (SPARK-46869) Add `logrotate` to to Spark docker files

Dongjoon Hyun created SPARK-46869: - Summary: Add `logrotate` to to Spark docker files Key: SPARK-46869 URL: https://issues.apache.org/jira/browse/SPARK-46869 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-46868) Support Spark Worker Log UI

[ https://issues.apache.org/jira/browse/SPARK-46868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-46868: - Assignee: Dongjoon Hyun > Support Spark Worker Log UI > --- >

[jira] [Updated] (SPARK-46868) Support Spark Worker Log UI

[ https://issues.apache.org/jira/browse/SPARK-46868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46868: --- Labels: pull-request-available (was: ) > Support Spark Worker Log UI >

[jira] [Created] (SPARK-46868) Support Spark Worker Log UI

Dongjoon Hyun created SPARK-46868: - Summary: Support Spark Worker Log UI Key: SPARK-46868 URL: https://issues.apache.org/jira/browse/SPARK-46868 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-46373) Create DataFrame Bug

[ https://issues.apache.org/jira/browse/SPARK-46373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17811020#comment-17811020 ] Eric Richardson commented on SPARK-46373: - Is there a summary issue for Scala 3 support for

[jira] [Created] (SPARK-46866) Streaming python data source API

Chaoqin Li created SPARK-46866: -- Summary: Streaming python data source API Key: SPARK-46866 URL: https://issues.apache.org/jira/browse/SPARK-46866 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-46863) Clean up custom.css

[ https://issues.apache.org/jira/browse/SPARK-46863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46863: --- Labels: pull-request-available (was: ) > Clean up custom.css > --- > >

[jira] [Updated] (SPARK-46865) Add Batch Support for TransformWithState Operator

[ https://issues.apache.org/jira/browse/SPARK-46865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Marnadi updated SPARK-46865: - Description: Add Batch support for the TransformWithState operator to maintain parity between

[jira] [Created] (SPARK-46865) Add Batch Support for TransformWithState Operator

Eric Marnadi created SPARK-46865: Summary: Add Batch Support for TransformWithState Operator Key: SPARK-46865 URL: https://issues.apache.org/jira/browse/SPARK-46865 Project: Spark Issue

[jira] [Updated] (SPARK-46864) Onboard Arbtirary State V2 onto New Error Class Framework

[ https://issues.apache.org/jira/browse/SPARK-46864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46864: --- Labels: pull-request-available (was: ) > Onboard Arbtirary State V2 onto New Error Class

[jira] [Created] (SPARK-46864) Onboard Arbtirary State V2 onto New Error Class Framework

Eric Marnadi created SPARK-46864: Summary: Onboard Arbtirary State V2 onto New Error Class Framework Key: SPARK-46864 URL: https://issues.apache.org/jira/browse/SPARK-46864 Project: Spark

[jira] [Created] (SPARK-46863) Clean up custom.css

Nicholas Chammas created SPARK-46863: Summary: Clean up custom.css Key: SPARK-46863 URL: https://issues.apache.org/jira/browse/SPARK-46863 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-46861) Avoid Deadlock in DAGScheduler

[ https://issues.apache.org/jira/browse/SPARK-46861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-46861: -- Labels: pull-request-available releasenotes (was: pull-request-available) > Avoid Deadlock

[jira] [Updated] (SPARK-46861) Avoid Deadlock in DAGScheduler

[ https://issues.apache.org/jira/browse/SPARK-46861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-46861: -- Summary: Avoid Deadlock in DAGScheduler (was: Potential Deadlock in DAGScheduler) > Avoid

[jira] [Commented] (SPARK-46861) Avoid Deadlock in DAGScheduler

[ https://issues.apache.org/jira/browse/SPARK-46861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17810972#comment-17810972 ] Dongjoon Hyun commented on SPARK-46861: --- I added a label, `releasenotes`, because this is an

[jira] [Resolved] (SPARK-46861) Potential Deadlock in DAGScheduler

[ https://issues.apache.org/jira/browse/SPARK-46861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-46861. --- Fix Version/s: 3.4.3 3.5.1 4.0.0 Resolution:

[jira] [Assigned] (SPARK-46861) Potential Deadlock in DAGScheduler

[ https://issues.apache.org/jira/browse/SPARK-46861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-46861: - Assignee: Fredrik Klauß > Potential Deadlock in DAGScheduler >

[jira] [Assigned] (SPARK-46855) Add `sketch` to the dependencies of the `catalyst` module in `module.py`

[ https://issues.apache.org/jira/browse/SPARK-46855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-46855: - Assignee: Yang Jie > Add `sketch` to the dependencies of the `catalyst` module in

[jira] [Resolved] (SPARK-46855) Add `sketch` to the dependencies of the `catalyst` module in `module.py`

[ https://issues.apache.org/jira/browse/SPARK-46855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-46855. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44878

[jira] [Resolved] (SPARK-46850) Convert `_LEGACY_ERROR_TEMP_2102 ` to `UNSUPPORTED_DATATYPE`

[ https://issues.apache.org/jira/browse/SPARK-46850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-46850. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44871

[jira] [Assigned] (SPARK-46850) Convert `_LEGACY_ERROR_TEMP_2102 ` to `UNSUPPORTED_DATATYPE`

[ https://issues.apache.org/jira/browse/SPARK-46850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-46850: Assignee: BingKun Pan > Convert `_LEGACY_ERROR_TEMP_2102 ` to `UNSUPPORTED_DATATYPE` >

[jira] [Updated] (SPARK-46862) Incorrect count() of a dataframe loaded from CSV datasource

[ https://issues.apache.org/jira/browse/SPARK-46862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46862: --- Labels: pull-request-available (was: ) > Incorrect count() of a dataframe loaded from CSV

[jira] [Created] (SPARK-46862) Incorrect count() of a dataframe loaded from CSV datasource

Max Gekk created SPARK-46862: Summary: Incorrect count() of a dataframe loaded from CSV datasource Key: SPARK-46862 URL: https://issues.apache.org/jira/browse/SPARK-46862 Project: Spark Issue

[jira] [Updated] (SPARK-46862) Incorrect count() of a dataframe loaded from CSV datasource

[ https://issues.apache.org/jira/browse/SPARK-46862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-46862: - Attachment: es-939111-data.csv > Incorrect count() of a dataframe loaded from CSV datasource >

[jira] [Updated] (SPARK-46861) Potential Deadlock in DAGScheduler

[ https://issues.apache.org/jira/browse/SPARK-46861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46861: --- Labels: pull-request-available (was: ) > Potential Deadlock in DAGScheduler >

[jira] [Created] (SPARK-46861) Potential Deadlock in DAGScheduler

Fredrik Klauß created SPARK-46861: - Summary: Potential Deadlock in DAGScheduler Key: SPARK-46861 URL: https://issues.apache.org/jira/browse/SPARK-46861 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-46830) Introducing collation concept into Spark

[ https://issues.apache.org/jira/browse/SPARK-46830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17810853#comment-17810853 ] Aleksandar Tomic commented on SPARK-46830: -- [~kabhwan]  Please take a look now. > Introducing

[jira] [Commented] (SPARK-32118) Use fine-grained read write lock for each database in HiveExternalCatalog

[ https://issues.apache.org/jira/browse/SPARK-32118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17810776#comment-17810776 ] xichenglin commented on SPARK-32118: Why does spark3 have this problem, but spark2 does not? spark2

[jira] [Updated] (SPARK-46860) Credentials with https url not working for --jars, --files, --archives & --py-files options on spark-submit command

[ https://issues.apache.org/jira/browse/SPARK-46860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vikram Janarthanan updated SPARK-46860: --- Affects Version/s: 3.5.0 3.3.3 > Credentials with https url

[jira] [Created] (SPARK-46860) Credentials with https url not working for --jars, --files, --archives & --py-files options on spark-submit command

Vikram Janarthanan created SPARK-46860: -- Summary: Credentials with https url not working for --jars, --files, --archives & --py-files options on spark-submit command Key: SPARK-46860 URL:

[jira] [Resolved] (SPARK-46620) Introduce a basic fallback mechanism for frame methods

[ https://issues.apache.org/jira/browse/SPARK-46620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-46620. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44869

[jira] [Created] (SPARK-46859) Specify the return type to avoid schema inference

Ruifeng Zheng created SPARK-46859: - Summary: Specify the return type to avoid schema inference Key: SPARK-46859 URL: https://issues.apache.org/jira/browse/SPARK-46859 Project: Spark Issue

[jira] [Assigned] (SPARK-46620) Introduce a basic fallback mechanism for frame methods

[ https://issues.apache.org/jira/browse/SPARK-46620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-46620: -- Assignee: Apache Spark > Introduce a basic fallback mechanism for frame methods >

[jira] [Assigned] (SPARK-46620) Introduce a basic fallback mechanism for frame methods

[ https://issues.apache.org/jira/browse/SPARK-46620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-46620: -- Assignee: (was: Apache Spark) > Introduce a basic fallback mechanism for frame

[jira] [Resolved] (SPARK-46851) Upgrade `buf` to v1.29.0

[ https://issues.apache.org/jira/browse/SPARK-46851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-46851. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 44877

[jira] [Assigned] (SPARK-46824) Enable test without optional dependency on PyPy

[ https://issues.apache.org/jira/browse/SPARK-46824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot reassigned SPARK-46824: -- Assignee: Apache Spark > Enable test without optional dependency on PyPy >

[jira] [Updated] (SPARK-46858) Upgrade Pandas to 2.2.0

[ https://issues.apache.org/jira/browse/SPARK-46858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46858: --- Labels: pull-request-available (was: ) > Upgrade Pandas to 2.2.0 > ---

[jira] [Updated] (SPARK-46857) Make `Master` log `appId` in case of worker loss

[ https://issues.apache.org/jira/browse/SPARK-46857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-46857: -- Parent: (was: SPARK-45869) Issue Type: Bug (was: Sub-task) > Make `Master` log

[jira] [Closed] (SPARK-46857) Make `Master` log `appId` in case of worker loss

[ https://issues.apache.org/jira/browse/SPARK-46857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-46857. - > Make `Master` log `appId` in case of worker loss >

[jira] [Updated] (SPARK-46857) Make `Master` log `appId` in case of worker loss

[ https://issues.apache.org/jira/browse/SPARK-46857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-46857: --- Labels: pull-request-available (was: ) > Make `Master` log `appId` in case of worker loss