[jira] [Created] (SPARK-37557) Replace object hash with sort aggregate if child is already sorted

Cheng Su created SPARK-37557: Summary: Replace object hash with sort aggregate if child is already sorted Key: SPARK-37557 URL: https://issues.apache.org/jira/browse/SPARK-37557 Project: Spark

[jira] [Commented] (SPARK-37552) Add the convert_timezone() function

[ https://issues.apache.org/jira/browse/SPARK-37552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17453815#comment-17453815 ] Apache Spark commented on SPARK-37552: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37552) Add the convert_timezone() function

[ https://issues.apache.org/jira/browse/SPARK-37552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37552: Assignee: Max Gekk (was: Apache Spark) > Add the convert_timezone() function >

[jira] [Assigned] (SPARK-37552) Add the convert_timezone() function

[ https://issues.apache.org/jira/browse/SPARK-37552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37552: Assignee: Apache Spark (was: Max Gekk) > Add the convert_timezone() function >

[jira] [Commented] (SPARK-37556) Deser void class fail with Java serialization

[ https://issues.apache.org/jira/browse/SPARK-37556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17453804#comment-17453804 ] Apache Spark commented on SPARK-37556: -- User 'daijyc' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37556) Deser void class fail with Java serialization

[ https://issues.apache.org/jira/browse/SPARK-37556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37556: Assignee: Apache Spark > Deser void class fail with Java serialization >

[jira] [Assigned] (SPARK-37556) Deser void class fail with Java serialization

[ https://issues.apache.org/jira/browse/SPARK-37556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37556: Assignee: (was: Apache Spark) > Deser void class fail with Java serialization >

[jira] [Commented] (SPARK-37556) Deser void class fail with Java serialization

[ https://issues.apache.org/jira/browse/SPARK-37556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17453803#comment-17453803 ] Apache Spark commented on SPARK-37556: -- User 'daijyc' has created a pull request for this issue:

[jira] [Created] (SPARK-37556) Deser void class fail with Java serialization

Daniel Dai created SPARK-37556: -- Summary: Deser void class fail with Java serialization Key: SPARK-37556 URL: https://issues.apache.org/jira/browse/SPARK-37556 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-37555) spark-sql should pass last unclosed comment to backend and execute throw a exception

[ https://issues.apache.org/jira/browse/SPARK-37555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37555: Assignee: (was: Apache Spark) > spark-sql should pass last unclosed comment to

[jira] [Assigned] (SPARK-37555) spark-sql should pass last unclosed comment to backend and execute throw a exception

[ https://issues.apache.org/jira/browse/SPARK-37555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37555: Assignee: Apache Spark > spark-sql should pass last unclosed comment to backend and

[jira] [Commented] (SPARK-37555) spark-sql should pass last unclosed comment to backend and execute throw a exception

[ https://issues.apache.org/jira/browse/SPARK-37555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17453796#comment-17453796 ] Apache Spark commented on SPARK-37555: -- User 'AngersZh' has created a pull request for this

[jira] [Assigned] (SPARK-37550) from_json documentation lacks examples for complex types

[ https://issues.apache.org/jira/browse/SPARK-37550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-37550: Assignee: Brian Yue > from_json documentation lacks examples for complex types >

[jira] [Resolved] (SPARK-37550) from_json documentation lacks examples for complex types

[ https://issues.apache.org/jira/browse/SPARK-37550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-37550. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34809

[jira] [Commented] (SPARK-37004) Job cancellation causes py4j errors on Jupyter due to pinned thread mode

[ https://issues.apache.org/jira/browse/SPARK-37004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17453787#comment-17453787 ] Apache Spark commented on SPARK-37004: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-37004) Job cancellation causes py4j errors on Jupyter due to pinned thread mode

[ https://issues.apache.org/jira/browse/SPARK-37004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37004: Assignee: (was: Apache Spark) > Job cancellation causes py4j errors on Jupyter due

[jira] [Commented] (SPARK-37004) Job cancellation causes py4j errors on Jupyter due to pinned thread mode

[ https://issues.apache.org/jira/browse/SPARK-37004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17453786#comment-17453786 ] Apache Spark commented on SPARK-37004: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-37004) Job cancellation causes py4j errors on Jupyter due to pinned thread mode

[ https://issues.apache.org/jira/browse/SPARK-37004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37004: Assignee: Apache Spark > Job cancellation causes py4j errors on Jupyter due to pinned

[jira] [Created] (SPARK-37555) spark-sql should pass last unclosed comment to backend and execute throw a exception

angerszhu created SPARK-37555: - Summary: spark-sql should pass last unclosed comment to backend and execute throw a exception Key: SPARK-37555 URL: https://issues.apache.org/jira/browse/SPARK-37555

[jira] [Resolved] (SPARK-37554) Add PyArrow, pandas and plotly to release Docker image dependencies

[ https://issues.apache.org/jira/browse/SPARK-37554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-37554. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34813

[jira] [Assigned] (SPARK-37554) Add PyArrow, pandas and plotly to release Docker image dependencies

[ https://issues.apache.org/jira/browse/SPARK-37554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-37554: Assignee: Hyukjin Kwon > Add PyArrow, pandas and plotly to release Docker image

[jira] [Commented] (SPARK-37554) Add PyArrow, pandas and plotly to release Docker image dependencies

[ https://issues.apache.org/jira/browse/SPARK-37554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17453756#comment-17453756 ] Apache Spark commented on SPARK-37554: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-37554) Add PyArrow, pandas and plotly to release Docker image dependencies

[ https://issues.apache.org/jira/browse/SPARK-37554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37554: Assignee: (was: Apache Spark) > Add PyArrow, pandas and plotly to release Docker

[jira] [Commented] (SPARK-37554) Add PyArrow, pandas and plotly to release Docker image dependencies

[ https://issues.apache.org/jira/browse/SPARK-37554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17453755#comment-17453755 ] Apache Spark commented on SPARK-37554: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-37554) Add PyArrow, pandas and plotly to release Docker image dependencies

[ https://issues.apache.org/jira/browse/SPARK-37554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37554: Assignee: Apache Spark > Add PyArrow, pandas and plotly to release Docker image

[jira] [Created] (SPARK-37554) Add PyArrow, pandas and plotly to release Docker image dependencies

Hyukjin Kwon created SPARK-37554: Summary: Add PyArrow, pandas and plotly to release Docker image dependencies Key: SPARK-37554 URL: https://issues.apache.org/jira/browse/SPARK-37554 Project: Spark

[jira] [Updated] (SPARK-37554) Add PyArrow, pandas and plotly to release Docker image dependencies

[ https://issues.apache.org/jira/browse/SPARK-37554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-37554: - Priority: Critical (was: Blocker) > Add PyArrow, pandas and plotly to release Docker image

[jira] [Updated] (SPARK-37554) Add PyArrow, pandas and plotly to release Docker image dependencies

[ https://issues.apache.org/jira/browse/SPARK-37554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-37554: - Priority: Blocker (was: Critical) > Add PyArrow, pandas and plotly to release Docker image

[jira] [Updated] (SPARK-37510) Support TimedeltaIndex in pandas API on Spark

[ https://issues.apache.org/jira/browse/SPARK-37510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-37510: - Parent: SPARK-37525 Issue Type: Sub-task (was: Umbrella) > Support TimedeltaIndex in

[jira] [Updated] (SPARK-37511) Introduce TimedeltaIndex to pandas API on Spark

[ https://issues.apache.org/jira/browse/SPARK-37511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-37511: - Parent: (was: SPARK-37510) Issue Type: Improvement (was: Sub-task) > Introduce

[jira] [Updated] (SPARK-37511) Introduce TimedeltaIndex to pandas API on Spark

[ https://issues.apache.org/jira/browse/SPARK-37511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-37511: - Parent: SPARK-37525 Issue Type: Sub-task (was: Improvement) > Introduce TimedeltaIndex

[jira] [Updated] (SPARK-37512) Support TimedeltaIndex creation (from Series/Index) and TimedeltaIndex.astype

[ https://issues.apache.org/jira/browse/SPARK-37512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-37512: - Parent: SPARK-37525 Issue Type: Sub-task (was: Improvement) > Support TimedeltaIndex

[jira] [Reopened] (SPARK-37525) Support TimedeltaIndex in pandas API on Spark

[ https://issues.apache.org/jira/browse/SPARK-37525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reopened SPARK-37525: -- > Support TimedeltaIndex in pandas API on Spark > - >

[jira] (SPARK-37525) Support TimedeltaIndex in pandas API on Spark

[ https://issues.apache.org/jira/browse/SPARK-37525 ] Hyukjin Kwon deleted comment on SPARK-37525: -- was (Author: apachespark): User 'xinrong-databricks' has created a pull request for this issue: https://github.com/apache/spark/pull/34787 >

[jira] [Updated] (SPARK-37512) Support TimedeltaIndex creation (from Series/Index) and TimedeltaIndex.astype

[ https://issues.apache.org/jira/browse/SPARK-37512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-37512: - Parent: (was: SPARK-37525) Issue Type: Improvement (was: Sub-task) > Support

[jira] [Resolved] (SPARK-37525) Support TimedeltaIndex in pandas API on Spark

[ https://issues.apache.org/jira/browse/SPARK-37525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-37525. -- Resolution: Fixed > Support TimedeltaIndex in pandas API on Spark >

[jira] [Updated] (SPARK-37512) Support TimedeltaIndex creation (from Series/Index) and TimedeltaIndex.astype

[ https://issues.apache.org/jira/browse/SPARK-37512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-37512: - Parent: SPARK-37525 Issue Type: Sub-task (was: Improvement) > Support TimedeltaIndex

[jira] [Updated] (SPARK-37525) Support TimedeltaIndex in pandas API on Spark

[ https://issues.apache.org/jira/browse/SPARK-37525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-37525: - Description: Since DayTimeIntervalType is supported in PySpark, we may add TimedeltaIndex

[jira] [Assigned] (SPARK-37525) Support TimedeltaIndex in pandas API on Spark

[ https://issues.apache.org/jira/browse/SPARK-37525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37525: Assignee: Apache Spark > Support TimedeltaIndex in pandas API on Spark >

[jira] [Commented] (SPARK-37525) Support TimedeltaIndex in pandas API on Spark

[ https://issues.apache.org/jira/browse/SPARK-37525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17453742#comment-17453742 ] Apache Spark commented on SPARK-37525: -- User 'xinrong-databricks' has created a pull request for

[jira] [Updated] (SPARK-37512) Support TimedeltaIndex creation (from Series/Index) and TimedeltaIndex.astype

[ https://issues.apache.org/jira/browse/SPARK-37512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-37512: - Parent: (was: SPARK-37510) Issue Type: Improvement (was: Sub-task) > Support

[jira] [Updated] (SPARK-37525) Support TimedeltaIndex in pandas API on Spark

[ https://issues.apache.org/jira/browse/SPARK-37525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-37525: - Summary: Support TimedeltaIndex in pandas API on Spark (was: Support basic operations of

[jira] [Updated] (SPARK-37525) Support basic operations of timedelta Series/Index

[ https://issues.apache.org/jira/browse/SPARK-37525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-37525: - Parent: (was: SPARK-37510) Issue Type: Umbrella (was: Sub-task) > Support basic

[jira] [Assigned] (SPARK-37525) Support TimedeltaIndex in pandas API on Spark

[ https://issues.apache.org/jira/browse/SPARK-37525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37525: Assignee: (was: Apache Spark) > Support TimedeltaIndex in pandas API on Spark >

[jira] [Created] (SPARK-37553) pandas.DataFrame.pivot_table KeyError on underscore in name

Nicholas Gustafson created SPARK-37553: -- Summary: pandas.DataFrame.pivot_table KeyError on underscore in name Key: SPARK-37553 URL: https://issues.apache.org/jira/browse/SPARK-37553 Project:

[jira] [Commented] (SPARK-37552) Add the convert_timezone() function

[ https://issues.apache.org/jira/browse/SPARK-37552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17453661#comment-17453661 ] Max Gekk commented on SPARK-37552: -- I am working on this. > Add the convert_timezone() function >

[jira] [Created] (SPARK-37552) Add the convert_timezone() function

Max Gekk created SPARK-37552: Summary: Add the convert_timezone() function Key: SPARK-37552 URL: https://issues.apache.org/jira/browse/SPARK-37552 Project: Spark Issue Type: New Feature

[jira] [Commented] (SPARK-37551) Argument 1 to "rename" of "DataFrame" has incompatible type in pandas.LocIndexerLike.__getitem__

[ https://issues.apache.org/jira/browse/SPARK-37551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17453626#comment-17453626 ] Maciej Szymkiewicz commented on SPARK-37551: cc [~itholic] [~ueshin], since you're both

[jira] [Created] (SPARK-37551) Argument 1 to "rename" of "DataFrame" has incompatible type in pandas.LocIndexerLike.__getitem__

Maciej Szymkiewicz created SPARK-37551: -- Summary: Argument 1 to "rename" of "DataFrame" has incompatible type in pandas.LocIndexerLike.__getitem__ Key: SPARK-37551 URL:

[jira] [Assigned] (SPARK-37451) Performance improvement regressed String to Decimal cast

[ https://issues.apache.org/jira/browse/SPARK-37451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37451: Assignee: Apache Spark > Performance improvement regressed String to Decimal cast >

[jira] [Assigned] (SPARK-37451) Performance improvement regressed String to Decimal cast

[ https://issues.apache.org/jira/browse/SPARK-37451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37451: Assignee: (was: Apache Spark) > Performance improvement regressed String to Decimal

[jira] [Commented] (SPARK-37451) Performance improvement regressed String to Decimal cast

[ https://issues.apache.org/jira/browse/SPARK-37451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17453619#comment-17453619 ] Apache Spark commented on SPARK-37451: -- User 'wangyum' has created a pull request for this issue:

[jira] [Commented] (SPARK-37451) Performance improvement regressed String to Decimal cast

[ https://issues.apache.org/jira/browse/SPARK-37451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17453618#comment-17453618 ] Apache Spark commented on SPARK-37451: -- User 'wangyum' has created a pull request for this issue:

[jira] [Commented] (SPARK-37549) Support set parallel through data source properties

[ https://issues.apache.org/jira/browse/SPARK-37549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17453599#comment-17453599 ] Apache Spark commented on SPARK-37549: -- User 'wangyum' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37549) Support set parallel through data source properties

[ https://issues.apache.org/jira/browse/SPARK-37549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37549: Assignee: (was: Apache Spark) > Support set parallel through data source properties

[jira] [Assigned] (SPARK-37549) Support set parallel through data source properties

[ https://issues.apache.org/jira/browse/SPARK-37549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37549: Assignee: Apache Spark > Support set parallel through data source properties >

[jira] [Commented] (SPARK-37549) Support set parallel through data source properties

[ https://issues.apache.org/jira/browse/SPARK-37549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17453598#comment-17453598 ] Apache Spark commented on SPARK-37549: -- User 'wangyum' has created a pull request for this issue:

[jira] (SPARK-37515) minRatePerPartition works as "max messages per partition per a batch" (it should be per seconds)

[ https://issues.apache.org/jira/browse/SPARK-37515 ] Sungpeo Kook deleted comment on SPARK-37515: -- was (Author: elixir kook): I couldn't find the proper JIRA's components for Kafka Streaming.    > minRatePerPartition works as "max messages

[jira] [Assigned] (SPARK-37550) from_json documentation lacks examples for complex types

[ https://issues.apache.org/jira/browse/SPARK-37550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37550: Assignee: Apache Spark > from_json documentation lacks examples for complex types >

[jira] [Assigned] (SPARK-37550) from_json documentation lacks examples for complex types

[ https://issues.apache.org/jira/browse/SPARK-37550?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37550: Assignee: (was: Apache Spark) > from_json documentation lacks examples for complex

[jira] [Commented] (SPARK-37550) from_json documentation lacks examples for complex types

[ https://issues.apache.org/jira/browse/SPARK-37550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17453595#comment-17453595 ] Apache Spark commented on SPARK-37550: -- User 'byyue' has created a pull request for this issue:

[jira] [Created] (SPARK-37550) from_json documentation lacks examples for complex types

Brian Yue created SPARK-37550: - Summary: from_json documentation lacks examples for complex types Key: SPARK-37550 URL: https://issues.apache.org/jira/browse/SPARK-37550 Project: Spark Issue

[jira] [Created] (SPARK-37549) Support set parallel through data source properties

Yuming Wang created SPARK-37549: --- Summary: Support set parallel through data source properties Key: SPARK-37549 URL: https://issues.apache.org/jira/browse/SPARK-37549 Project: Spark Issue