[jira] [Updated] (SPARK-42596) [YARN] OMP_NUM_THREADS not set to number of executor cores by default

[ https://issues.apache.org/jira/browse/SPARK-42596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated SPARK-42596: --- Description: Run this PySpark script with `spark.executor.cores=1` {code:python} import os from

[jira] [Updated] (SPARK-42596) [YARN] OMP_NUM_THREADS not set to number of executor cores by default

[ https://issues.apache.org/jira/browse/SPARK-42596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated SPARK-42596: --- Description: Run this PySpark script with `spark.executor.cores=1` {code:python} import os from

[jira] [Commented] (SPARK-42596) [YARN] OMP_NUM_THREADS not set to number of executor cores by default

[ https://issues.apache.org/jira/browse/SPARK-42596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17693837#comment-17693837 ] John Zhuge commented on SPARK-42596: Looks like a regression from SPARK-41188 where it removed the

[jira] [Commented] (SPARK-42572) Logic error for StateStore.validateStateRowFormat

[ https://issues.apache.org/jira/browse/SPARK-42572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17693835#comment-17693835 ] Apache Spark commented on SPARK-42572: -- User 'WweiL' has created a pull request for this issue:

[jira] [Assigned] (SPARK-42572) Logic error for StateStore.validateStateRowFormat

[ https://issues.apache.org/jira/browse/SPARK-42572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42572: Assignee: Apache Spark > Logic error for StateStore.validateStateRowFormat >

[jira] [Assigned] (SPARK-42572) Logic error for StateStore.validateStateRowFormat

[ https://issues.apache.org/jira/browse/SPARK-42572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42572: Assignee: (was: Apache Spark) > Logic error for StateStore.validateStateRowFormat >

[jira] [Commented] (SPARK-42572) Logic error for StateStore.validateStateRowFormat

[ https://issues.apache.org/jira/browse/SPARK-42572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17693834#comment-17693834 ] Wei Liu commented on SPARK-42572: - I'm not very sure what's the true process here.. We should still use

[jira] [Created] (SPARK-42596) [YARN] OMP_NUM_THREADS not set to number of executor cores by default

John Zhuge created SPARK-42596: -- Summary: [YARN] OMP_NUM_THREADS not set to number of executor cores by default Key: SPARK-42596 URL: https://issues.apache.org/jira/browse/SPARK-42596 Project: Spark

[jira] [Created] (SPARK-42595) Support query inserted partitions after insert data into table when hive.exec.dynamic.partition=true

zhang haoyan created SPARK-42595: Summary: Support query inserted partitions after insert data into table when hive.exec.dynamic.partition=true Key: SPARK-42595 URL:

[jira] [Resolved] (SPARK-42594) spark can not read lastest view sql when run `create or replace view` by hive

[ https://issues.apache.org/jira/browse/SPARK-42594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-42594. - Resolution: Not A Bug > spark can not read lastest view sql when run `create or replace view`

[jira] [Reopened] (SPARK-42594) spark can not read lastest view sql when run `create or replace view` by hive

[ https://issues.apache.org/jira/browse/SPARK-42594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reopened SPARK-42594: - > spark can not read lastest view sql when run `create or replace view` by hive >

[jira] [Resolved] (SPARK-42594) spark can not read lastest view sql when run `create or replace view` by hive

[ https://issues.apache.org/jira/browse/SPARK-42594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ming95 resolved SPARK-42594. Resolution: Fixed > spark can not read lastest view sql when run `create or replace view` by hive

[jira] [Commented] (SPARK-42594) spark can not read lastest view sql when run `create or replace view` by hive

[ https://issues.apache.org/jira/browse/SPARK-42594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17693810#comment-17693810 ] ming95 commented on SPARK-42594: OK ,Thanks~ [~yumwang]  > spark can not read lastest view sql

[jira] [Resolved] (SPARK-42528) Optimize PercentileHeap

[ https://issues.apache.org/jira/browse/SPARK-42528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-42528. - Fix Version/s: 3.5.0 (was: 3.4.0) Resolution: Fixed Issue

[jira] [Commented] (SPARK-42594) spark can not read lastest view sql when run `create or replace view` by hive

[ https://issues.apache.org/jira/browse/SPARK-42594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17693799#comment-17693799 ] Yuming Wang commented on SPARK-42594: - Spark saves information to table properties, Hive does not

[jira] [Commented] (SPARK-42594) spark can not read lastest view sql when run `create or replace view` by hive

[ https://issues.apache.org/jira/browse/SPARK-42594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17693795#comment-17693795 ] ming95 commented on SPARK-42594: [~yumwang] [~gurwls223]  gentel ping ~ > spark can not read

[jira] [Updated] (SPARK-42594) spark can not read lastest view sql when run `create or replace view` by hive

[ https://issues.apache.org/jira/browse/SPARK-42594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ming95 updated SPARK-42594: --- Description: 1. Spark would save view schema as tabel param.  2. Spark will make tabel param as

[jira] [Updated] (SPARK-42594) spark can not read lastest view sql when run `create or replace view` by hive

[ https://issues.apache.org/jira/browse/SPARK-42594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ming95 updated SPARK-42594: --- Attachment: image-2023-02-27-13-31-20-420.png > spark can not read lastest view sql when run

[jira] [Created] (SPARK-42594) spark can not read lastest view sql when run `create or replace view` by hive

ming95 created SPARK-42594: -- Summary: spark can not read lastest view sql when run `create or replace view` by hive Key: SPARK-42594 URL: https://issues.apache.org/jira/browse/SPARK-42594 Project:

[jira] [Updated] (SPARK-42593) Deprecate the APIs that will be removed in pandas 2.0.

[ https://issues.apache.org/jira/browse/SPARK-42593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-42593: Description: pandas is preparing to release 2.0 which includes bunch of API changes.

[jira] [Commented] (SPARK-42593) Deprecate the APIs that will be removed in pandas 2.0.

[ https://issues.apache.org/jira/browse/SPARK-42593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17693788#comment-17693788 ] Haejoon Lee commented on SPARK-42593: - I'm taking a look at this one. Will submit a PR soon. >

[jira] [Created] (SPARK-42593) Deprecate the APIs that will be removed in pandas 2.0.

Haejoon Lee created SPARK-42593: --- Summary: Deprecate the APIs that will be removed in pandas 2.0. Key: SPARK-42593 URL: https://issues.apache.org/jira/browse/SPARK-42593 Project: Spark Issue

[jira] [Commented] (SPARK-42592) Document SS guide doc for supporting multiple stateful operators (especially chained aggregations)

[ https://issues.apache.org/jira/browse/SPARK-42592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17693786#comment-17693786 ] Jungtaek Lim commented on SPARK-42592: -- Ideally this should be a part of Spark 3.4.0... Since RC

[jira] [Created] (SPARK-42592) Document SS guide doc for supporting multiple stateful operators (especially chained aggregations)

Jungtaek Lim created SPARK-42592: Summary: Document SS guide doc for supporting multiple stateful operators (especially chained aggregations) Key: SPARK-42592 URL:

[jira] [Created] (SPARK-42591) Document SS guide doc for introducing watermark propagation among operators

Jungtaek Lim created SPARK-42591: Summary: Document SS guide doc for introducing watermark propagation among operators Key: SPARK-42591 URL: https://issues.apache.org/jira/browse/SPARK-42591 Project:

[jira] [Commented] (SPARK-42581) Add SparkSession implicits

[ https://issues.apache.org/jira/browse/SPARK-42581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17693782#comment-17693782 ] Apache Spark commented on SPARK-42581: -- User 'hvanhovell' has created a pull request for this

[jira] [Updated] (SPARK-42362) Upgrade kubernetes-client from 6.4.0 to 6.4.1

[ https://issues.apache.org/jira/browse/SPARK-42362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-42362: -- Affects Version/s: 3.4.0 (was: 3.5.0) > Upgrade kubernetes-client

[jira] [Commented] (SPARK-42586) Implement RuntimeConf

[ https://issues.apache.org/jira/browse/SPARK-42586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17693777#comment-17693777 ] Apache Spark commented on SPARK-42586: -- User 'hvanhovell' has created a pull request for this

[jira] [Updated] (SPARK-42497) Support of pandas API on Spark for Spark Connect.

[ https://issues.apache.org/jira/browse/SPARK-42497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-42497: Affects Version/s: 3.5.0 (was: 3.4.0) > Support of pandas API on Spark

[jira] [Commented] (SPARK-42569) Throw unsupported exceptions for non-supported API

[ https://issues.apache.org/jira/browse/SPARK-42569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17693776#comment-17693776 ] Apache Spark commented on SPARK-42569: -- User 'amaliujia' has created a pull request for this issue:

[jira] [Commented] (SPARK-42569) Throw unsupported exceptions for non-supported API

[ https://issues.apache.org/jira/browse/SPARK-42569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17693775#comment-17693775 ] Apache Spark commented on SPARK-42569: -- User 'amaliujia' has created a pull request for this issue:

[jira] [Resolved] (SPARK-42589) Exclude `RelationalGroupedDataset.apply` from CompatibilitySuite

[ https://issues.apache.org/jira/browse/SPARK-42589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-42589. --- Resolution: Cannot Reproduce > Exclude `RelationalGroupedDataset.apply` from

[jira] [Updated] (SPARK-40821) Introduce window_time function to extract event time from the window column

[ https://issues.apache.org/jira/browse/SPARK-40821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-40821: - Summary: Introduce window_time function to extract event time from the window column (was: Fix

[jira] [Commented] (SPARK-42587) Use wrapper versions for SBT and Maven in `connect` module tests

[ https://issues.apache.org/jira/browse/SPARK-42587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17693772#comment-17693772 ] Apache Spark commented on SPARK-42587: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Commented] (SPARK-42538) `functions#lit` support more types

[ https://issues.apache.org/jira/browse/SPARK-42538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17693770#comment-17693770 ] Yang Jie commented on SPARK-42538: -- Got it > `functions#lit` support more types >

[jira] [Created] (SPARK-42590) Introduce Decimal128 as the physical type for DecimalType

jiaan.geng created SPARK-42590: -- Summary: Introduce Decimal128 as the physical type for DecimalType Key: SPARK-42590 URL: https://issues.apache.org/jira/browse/SPARK-42590 Project: Spark Issue

[jira] [Commented] (SPARK-42538) `functions#lit` support more types

[ https://issues.apache.org/jira/browse/SPARK-42538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17693768#comment-17693768 ] Herman van Hövell commented on SPARK-42538: --- It technically could be released without it.

[jira] [Commented] (SPARK-42588) collapse two adjacent windows with the equivalent partition/order expression

[ https://issues.apache.org/jira/browse/SPARK-42588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17693766#comment-17693766 ] Apache Spark commented on SPARK-42588: -- User 'zml1206' has created a pull request for this issue:

[jira] [Assigned] (SPARK-42588) collapse two adjacent windows with the equivalent partition/order expression

[ https://issues.apache.org/jira/browse/SPARK-42588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42588: Assignee: (was: Apache Spark) > collapse two adjacent windows with the equivalent

[jira] [Assigned] (SPARK-42588) collapse two adjacent windows with the equivalent partition/order expression

[ https://issues.apache.org/jira/browse/SPARK-42588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42588: Assignee: Apache Spark > collapse two adjacent windows with the equivalent

[jira] [Commented] (SPARK-42589) Exclude `RelationalGroupedDataset.apply` from CompatibilitySuite

[ https://issues.apache.org/jira/browse/SPARK-42589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17693764#comment-17693764 ] Apache Spark commented on SPARK-42589: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-42589) Exclude `RelationalGroupedDataset.apply` from CompatibilitySuite

[ https://issues.apache.org/jira/browse/SPARK-42589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42589: Assignee: (was: Apache Spark) > Exclude `RelationalGroupedDataset.apply` from

[jira] [Updated] (SPARK-42588) collapse two adjacent windows with the equivalent partition/order expression

[ https://issues.apache.org/jira/browse/SPARK-42588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhuml updated SPARK-42588: -- Description: Extend the CollapseWindow rule to collapse Window nodes with the equivalent partition/order

[jira] [Commented] (SPARK-42589) Exclude `RelationalGroupedDataset.apply` from CompatibilitySuite

[ https://issues.apache.org/jira/browse/SPARK-42589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17693763#comment-17693763 ] Apache Spark commented on SPARK-42589: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-42589) Exclude `RelationalGroupedDataset.apply` from CompatibilitySuite

[ https://issues.apache.org/jira/browse/SPARK-42589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42589: Assignee: Apache Spark > Exclude `RelationalGroupedDataset.apply` from

[jira] [Created] (SPARK-42589) Exclude `RelationalGroupedDataset.apply` from CompatibilitySuite

Dongjoon Hyun created SPARK-42589: - Summary: Exclude `RelationalGroupedDataset.apply` from CompatibilitySuite Key: SPARK-42589 URL: https://issues.apache.org/jira/browse/SPARK-42589 Project: Spark

[jira] [Assigned] (SPARK-42586) Implement RuntimeConf

[ https://issues.apache.org/jira/browse/SPARK-42586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hövell reassigned SPARK-42586: - Assignee: Herman van Hövell > Implement RuntimeConf > - >

[jira] [Resolved] (SPARK-42560) Implement ColumnName

[ https://issues.apache.org/jira/browse/SPARK-42560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hövell resolved SPARK-42560. --- Fix Version/s: 3.4.1 Resolution: Fixed > Implement ColumnName >

[jira] [Assigned] (SPARK-42587) Use wrapper versions for SBT and Maven in `connect` module tests

[ https://issues.apache.org/jira/browse/SPARK-42587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-42587: - Assignee: Dongjoon Hyun > Use wrapper versions for SBT and Maven in `connect` module

[jira] [Resolved] (SPARK-42587) Use wrapper versions for SBT and Maven in `connect` module tests

[ https://issues.apache.org/jira/browse/SPARK-42587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-42587. --- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 40180

[jira] [Commented] (SPARK-42538) `functions#lit` support more types

[ https://issues.apache.org/jira/browse/SPARK-42538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17693757#comment-17693757 ] Yang Jie commented on SPARK-42538: -- [~hvanhovell]  Should fix version be 3.4.0? It hasn't been

[jira] [Created] (SPARK-42588) collapse two adjacent windows with the equivalent partition/order expression

zhuml created SPARK-42588: - Summary: collapse two adjacent windows with the equivalent partition/order expression Key: SPARK-42588 URL: https://issues.apache.org/jira/browse/SPARK-42588 Project: Spark

[jira] [Commented] (SPARK-42577) A large stage could run indefinitely due to executor lost

[ https://issues.apache.org/jira/browse/SPARK-42577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17693753#comment-17693753 ] Tengfei Huang commented on SPARK-42577: --- I am working on this. Thanks. [~Ngone51] > A large stage

[jira] [Updated] (SPARK-42587) Use wrapper versions for SBT and Maven in `connect` module tests

[ https://issues.apache.org/jira/browse/SPARK-42587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-42587: -- Priority: Minor (was: Major) > Use wrapper versions for SBT and Maven in `connect` module

[jira] [Commented] (SPARK-42587) Use wrapper versions for SBT and Maven in `connect` module tests

[ https://issues.apache.org/jira/browse/SPARK-42587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17693749#comment-17693749 ] Apache Spark commented on SPARK-42587: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-42587) Use wrapper versions for SBT and Maven in `connect` module tests

[ https://issues.apache.org/jira/browse/SPARK-42587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42587: Assignee: (was: Apache Spark) > Use wrapper versions for SBT and Maven in `connect`

[jira] [Assigned] (SPARK-42587) Use wrapper versions for SBT and Maven in `connect` module tests

[ https://issues.apache.org/jira/browse/SPARK-42587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42587: Assignee: Apache Spark > Use wrapper versions for SBT and Maven in `connect` module

[jira] [Commented] (SPARK-42587) Use wrapper versions for SBT and Maven in `connect` module tests

[ https://issues.apache.org/jira/browse/SPARK-42587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17693748#comment-17693748 ] Apache Spark commented on SPARK-42587: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Created] (SPARK-42587) Use wrapper versions for SBT and Maven in `connect` module tests

Dongjoon Hyun created SPARK-42587: - Summary: Use wrapper versions for SBT and Maven in `connect` module tests Key: SPARK-42587 URL: https://issues.apache.org/jira/browse/SPARK-42587 Project: Spark

[jira] [Commented] (SPARK-42485) SPIP: Shutting down spark structured streaming when the streaming process completed current process

[ https://issues.apache.org/jira/browse/SPARK-42485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17693745#comment-17693745 ] Hyukjin Kwon commented on SPARK-42485: -- For the proper SPIP, should better read

[jira] [Commented] (SPARK-42517) Add documentation for Protobuf connector

[ https://issues.apache.org/jira/browse/SPARK-42517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17693744#comment-17693744 ] Hyukjin Kwon commented on SPARK-42517: -- Duplicate of SPARK-40776? > Add documentation for Protobuf

[jira] [Commented] (SPARK-42572) Logic error for StateStore.validateStateRowFormat

[ https://issues.apache.org/jira/browse/SPARK-42572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17693742#comment-17693742 ] Hyukjin Kwon commented on SPARK-42572: -- [~WweiL] are you saying that we should revert

[jira] [Resolved] (SPARK-42499) Support for Runtime SQL configuration

[ https://issues.apache.org/jira/browse/SPARK-42499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-42499. -- Resolution: Duplicate > Support for Runtime SQL configuration >

[jira] [Assigned] (SPARK-42559) Implement DataFrameNaFunctions

[ https://issues.apache.org/jira/browse/SPARK-42559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hövell reassigned SPARK-42559: - Assignee: BingKun Pan > Implement DataFrameNaFunctions >

[jira] [Created] (SPARK-42586) Implement RuntimeConf

Herman van Hövell created SPARK-42586: - Summary: Implement RuntimeConf Key: SPARK-42586 URL: https://issues.apache.org/jira/browse/SPARK-42586 Project: Spark Issue Type: New Feature

[jira] [Updated] (SPARK-42585) Streaming createDataFrame implementation

[ https://issues.apache.org/jira/browse/SPARK-42585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-42585: - Description: createDataFrame in Spark Connect is now one protobuf message which doesn't allow

[jira] [Created] (SPARK-42585) Streaming createDataFrame implementation

Hyukjin Kwon created SPARK-42585: Summary: Streaming createDataFrame implementation Key: SPARK-42585 URL: https://issues.apache.org/jira/browse/SPARK-42585 Project: Spark Issue Type:

[jira] [Commented] (SPARK-42581) Add SparkSession implicits

[ https://issues.apache.org/jira/browse/SPARK-42581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17693736#comment-17693736 ] Herman van Hövell commented on SPARK-42581: --- Waiting for SPARK-42560 > Add SparkSession

[jira] [Resolved] (SPARK-42564) Implement Dataset.version and Dataset.time

[ https://issues.apache.org/jira/browse/SPARK-42564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hövell resolved SPARK-42564. --- Fix Version/s: 3.4.1 Resolution: Fixed > Implement Dataset.version and

[jira] [Resolved] (SPARK-42419) Migrate `TypeError` into error framework for Spark Connect column API.

[ https://issues.apache.org/jira/browse/SPARK-42419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-42419. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 39991

[jira] [Assigned] (SPARK-42419) Migrate `TypeError` into error framework for Spark Connect column API.

[ https://issues.apache.org/jira/browse/SPARK-42419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-42419: Assignee: Haejoon Lee > Migrate `TypeError` into error framework for Spark Connect

[jira] [Resolved] (SPARK-42569) Throw unsupported exceptions for non-supported API

[ https://issues.apache.org/jira/browse/SPARK-42569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-42569. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 40172

[jira] [Resolved] (SPARK-42574) DataFrame.toPandas should handle duplicated column names

[ https://issues.apache.org/jira/browse/SPARK-42574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-42574. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 40170

[jira] [Assigned] (SPARK-42574) DataFrame.toPandas should handle duplicated column names

[ https://issues.apache.org/jira/browse/SPARK-42574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-42574: Assignee: Takuya Ueshin > DataFrame.toPandas should handle duplicated column names >

[jira] [Commented] (SPARK-42560) Implement ColumnName

[ https://issues.apache.org/jira/browse/SPARK-42560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17693730#comment-17693730 ] Apache Spark commented on SPARK-42560: -- User 'hvanhovell' has created a pull request for this

[jira] [Commented] (SPARK-42560) Implement ColumnName

[ https://issues.apache.org/jira/browse/SPARK-42560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17693729#comment-17693729 ] Apache Spark commented on SPARK-42560: -- User 'hvanhovell' has created a pull request for this

[jira] [Created] (SPARK-42584) Improve output of Column.explain

Herman van Hövell created SPARK-42584: - Summary: Improve output of Column.explain Key: SPARK-42584 URL: https://issues.apache.org/jira/browse/SPARK-42584 Project: Spark Issue Type: New

[jira] [Commented] (SPARK-42407) `with as` executed again

[ https://issues.apache.org/jira/browse/SPARK-42407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17693723#comment-17693723 ] Pablo Langa Blanco commented on SPARK-42407: In my opinion, "WITH AS" syntax is intended to

[jira] [Comment Edited] (SPARK-40525) Floating-point value with an INT/BYTE/SHORT/LONG type errors out in DataFrame but evaluates to a rounded value in SparkSQL

[ https://issues.apache.org/jira/browse/SPARK-40525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17693721#comment-17693721 ] Pablo Langa Blanco edited comment on SPARK-40525 at 2/26/23 10:57 PM:

[jira] [Commented] (SPARK-40525) Floating-point value with an INT/BYTE/SHORT/LONG type errors out in DataFrame but evaluates to a rounded value in SparkSQL

[ https://issues.apache.org/jira/browse/SPARK-40525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17693721#comment-17693721 ] Pablo Langa Blanco commented on SPARK-40525: Hi [~x/sys] , When you are working with Spark

[jira] [Updated] (SPARK-42583) Remove outer join if all aggregate functions are distinct

[ https://issues.apache.org/jira/browse/SPARK-42583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-42583: Description: To support more cases:

[jira] [Commented] (SPARK-42583) Remove outer join if all aggregate functions are distinct

[ https://issues.apache.org/jira/browse/SPARK-42583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17693649#comment-17693649 ] Apache Spark commented on SPARK-42583: -- User 'wangyum' has created a pull request for this issue:

[jira] [Assigned] (SPARK-42583) Remove outer join if all aggregate functions are distinct

[ https://issues.apache.org/jira/browse/SPARK-42583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42583: Assignee: (was: Apache Spark) > Remove outer join if all aggregate functions are

[jira] [Commented] (SPARK-42583) Remove outer join if all aggregate functions are distinct

[ https://issues.apache.org/jira/browse/SPARK-42583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17693648#comment-17693648 ] Apache Spark commented on SPARK-42583: -- User 'wangyum' has created a pull request for this issue:

[jira] [Assigned] (SPARK-42583) Remove outer join if all aggregate functions are distinct

[ https://issues.apache.org/jira/browse/SPARK-42583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42583: Assignee: Apache Spark > Remove outer join if all aggregate functions are distinct >

[jira] [Created] (SPARK-42583) Remove outer join if all aggregate functions are distinct

Yuming Wang created SPARK-42583: --- Summary: Remove outer join if all aggregate functions are distinct Key: SPARK-42583 URL: https://issues.apache.org/jira/browse/SPARK-42583 Project: Spark

[jira] [Commented] (SPARK-42564) Implement Dataset.version and Dataset.time

[ https://issues.apache.org/jira/browse/SPARK-42564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17693643#comment-17693643 ] Apache Spark commented on SPARK-42564: -- User 'panbingkun' has created a pull request for this

[jira] [Assigned] (SPARK-42564) Implement Dataset.version and Dataset.time

[ https://issues.apache.org/jira/browse/SPARK-42564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42564: Assignee: BingKun Pan (was: Apache Spark) > Implement Dataset.version and Dataset.time

[jira] [Assigned] (SPARK-42564) Implement Dataset.version and Dataset.time

[ https://issues.apache.org/jira/browse/SPARK-42564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42564: Assignee: Apache Spark (was: BingKun Pan) > Implement Dataset.version and Dataset.time

[jira] [Commented] (SPARK-42564) Implement Dataset.version and Dataset.time

[ https://issues.apache.org/jira/browse/SPARK-42564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17693642#comment-17693642 ] Apache Spark commented on SPARK-42564: -- User 'panbingkun' has created a pull request for this

[jira] [Commented] (SPARK-42559) Implement DataFrameNaFunctions

[ https://issues.apache.org/jira/browse/SPARK-42559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17693641#comment-17693641 ] BingKun Pan commented on SPARK-42559: - I work on it. > Implement DataFrameNaFunctions >