[jira] [Assigned] (SPARK-38651) Writing out empty or nested empty schemas in Datasource should be configurable

[ https://issues.apache.org/jira/browse/SPARK-38651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38651: Assignee: Apache Spark > Writing out empty or nested empty schemas in Datasource should

[jira] [Commented] (SPARK-38651) Writing out empty or nested empty schemas in Datasource should be configurable

[ https://issues.apache.org/jira/browse/SPARK-38651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17512202#comment-17512202 ] Apache Spark commented on SPARK-38651: -- User 'thejdeep' has created a pull request for this issue:

[jira] [Assigned] (SPARK-38651) Writing out empty or nested empty schemas in Datasource should be configurable

[ https://issues.apache.org/jira/browse/SPARK-38651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38651: Assignee: (was: Apache Spark) > Writing out empty or nested empty schemas in

[jira] [Commented] (SPARK-38654) Show default index type in SQL plans for pandas API on Spark

[ https://issues.apache.org/jira/browse/SPARK-38654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17512198#comment-17512198 ] Apache Spark commented on SPARK-38654: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Commented] (SPARK-38654) Show default index type in SQL plans for pandas API on Spark

[ https://issues.apache.org/jira/browse/SPARK-38654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17512197#comment-17512197 ] Apache Spark commented on SPARK-38654: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-38654) Show default index type in SQL plans for pandas API on Spark

[ https://issues.apache.org/jira/browse/SPARK-38654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38654: Assignee: Apache Spark > Show default index type in SQL plans for pandas API on Spark >

[jira] [Assigned] (SPARK-38654) Show default index type in SQL plans for pandas API on Spark

[ https://issues.apache.org/jira/browse/SPARK-38654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38654: Assignee: (was: Apache Spark) > Show default index type in SQL plans for pandas API

[jira] [Updated] (SPARK-38654) Show default index type in SQL plans for pandas API on Spark

[ https://issues.apache.org/jira/browse/SPARK-38654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-38654: - Priority: Minor (was: Major) > Show default index type in SQL plans for pandas API on Spark >

[jira] [Created] (SPARK-38654) Show default index type in SQL plans for pandas API on Spark

Hyukjin Kwon created SPARK-38654: Summary: Show default index type in SQL plans for pandas API on Spark Key: SPARK-38654 URL: https://issues.apache.org/jira/browse/SPARK-38654 Project: Spark

[jira] [Commented] (SPARK-38570) Incorrect DynamicPartitionPruning caused by Literal

[ https://issues.apache.org/jira/browse/SPARK-38570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17512195#comment-17512195 ] Apache Spark commented on SPARK-38570: -- User 'mcdull-zhang' has created a pull request for this

[jira] [Resolved] (SPARK-38610) UI for Pandas API on Spark

[ https://issues.apache.org/jira/browse/SPARK-38610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-38610. -- Resolution: Invalid Will focus on improving existing SQL UI to show the related info. > UI

[jira] [Assigned] (SPARK-38646) Pull a trait out for Python functions

[ https://issues.apache.org/jira/browse/SPARK-38646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-38646: Assignee: Zhen Li > Pull a trait out for Python functions >

[jira] [Resolved] (SPARK-38646) Pull a trait out for Python functions

[ https://issues.apache.org/jira/browse/SPARK-38646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-38646. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 35964

[jira] [Updated] (SPARK-38653) Repartition by Column that is Int not working properly only on particular numbers. (11, 33)

[ https://issues.apache.org/jira/browse/SPARK-38653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Engelhart updated SPARK-38653: --- Description: My understanding is when you call .repartition(a column). For each unique key

[jira] [Updated] (SPARK-38653) Repartition by Column that is Int not working properly only on particular numbers. (11, 33)

[ https://issues.apache.org/jira/browse/SPARK-38653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Engelhart updated SPARK-38653: --- Description: My understanding is when you call .repartition(a column). For each unique key

[jira] [Updated] (SPARK-38653) Repartition by Column that is Int not working properly only on particular numbers. (11, 33)

[ https://issues.apache.org/jira/browse/SPARK-38653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Engelhart updated SPARK-38653: --- Summary: Repartition by Column that is Int not working properly only on particular numbers.

[jira] [Created] (SPARK-38653) Repartition by Column that is Int not working properly only particular numbers. (11, 33)

John Engelhart created SPARK-38653: -- Summary: Repartition by Column that is Int not working properly only particular numbers. (11, 33) Key: SPARK-38653 URL: https://issues.apache.org/jira/browse/SPARK-38653

[jira] [Updated] (SPARK-38652) K8S IT Test DepsTestsSuite blocks with PathIOException in hadoop-aws-3.3.2

[ https://issues.apache.org/jira/browse/SPARK-38652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] qian updated SPARK-38652: - Description: DepsTestsSuite in k8s IT test is blocked with PathIOException in hadoop-aws-3.3.2. Exception

[jira] [Updated] (SPARK-38652) K8S IT Test DepsTestsSuite blocks with PathIOException in hadoop-aws-3.3.2

[ https://issues.apache.org/jira/browse/SPARK-38652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] qian updated SPARK-38652: - Description: DepsTestsSuite in k8s IT test is blocked with PathIOException in hadoop-aws-3.3.2. Exception

[jira] [Commented] (SPARK-38652) K8S IT Test DepsTestsSuite blocks with PathIOException in hadoop-aws-3.3.2

[ https://issues.apache.org/jira/browse/SPARK-38652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17512163#comment-17512163 ] qian commented on SPARK-38652: -- I am working on it. cc [~chaosun]  & [~dongjoon]  > K8S IT Test

[jira] [Created] (SPARK-38652) K8S IT Test DepsTestsSuite blocks with PathIOException in hadoop-aws-3.3.2

qian created SPARK-38652: Summary: K8S IT Test DepsTestsSuite blocks with PathIOException in hadoop-aws-3.3.2 Key: SPARK-38652 URL: https://issues.apache.org/jira/browse/SPARK-38652 Project: Spark

[jira] [Created] (SPARK-38651) Writing out empty or nested empty schemas in Datasource should be configurable

Thejdeep Gudivada created SPARK-38651: - Summary: Writing out empty or nested empty schemas in Datasource should be configurable Key: SPARK-38651 URL: https://issues.apache.org/jira/browse/SPARK-38651

[jira] [Resolved] (SPARK-38570) Incorrect DynamicPartitionPruning caused by Literal

[ https://issues.apache.org/jira/browse/SPARK-38570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-38570. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35878

[jira] [Assigned] (SPARK-38570) Incorrect DynamicPartitionPruning caused by Literal

[ https://issues.apache.org/jira/browse/SPARK-38570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-38570: --- Assignee: mcdull_zhang > Incorrect DynamicPartitionPruning caused by Literal >

[jira] [Updated] (SPARK-38645) Support `spark.sql.codegen.cleanedSourcePrint` flag to print Codegen cleanedSource

[ https://issues.apache.org/jira/browse/SPARK-38645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-38645: - Fix Version/s: (was: 3.2.1) > Support `spark.sql.codegen.cleanedSourcePrint` flag to print

[jira] [Commented] (SPARK-38650) Better ParseException message for char without length

[ https://issues.apache.org/jira/browse/SPARK-38650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17512126#comment-17512126 ] Apache Spark commented on SPARK-38650: -- User 'anchovYu' has created a pull request for this issue:

[jira] [Assigned] (SPARK-38650) Better ParseException message for char without length

[ https://issues.apache.org/jira/browse/SPARK-38650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38650: Assignee: Apache Spark > Better ParseException message for char without length >

[jira] [Assigned] (SPARK-38650) Better ParseException message for char without length

[ https://issues.apache.org/jira/browse/SPARK-38650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38650: Assignee: (was: Apache Spark) > Better ParseException message for char without

[jira] [Created] (SPARK-38650) Better ParseException message for char without length

Xinyi Yu created SPARK-38650: Summary: Better ParseException message for char without length Key: SPARK-38650 URL: https://issues.apache.org/jira/browse/SPARK-38650 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-38641) Get rid of invalid configuration elements in mvn_scalafmt in main pom.xml

[ https://issues.apache.org/jira/browse/SPARK-38641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-38641. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 35956

[jira] [Assigned] (SPARK-38641) Get rid of invalid configuration elements in mvn_scalafmt in main pom.xml

[ https://issues.apache.org/jira/browse/SPARK-38641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-38641: Assignee: morvenhuang > Get rid of invalid configuration elements in mvn_scalafmt in

[jira] [Comment Edited] (SPARK-26639) The reuse subquery function maybe does not work in SPARK SQL

[ https://issues.apache.org/jira/browse/SPARK-26639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17511358#comment-17511358 ] Stu edited comment on SPARK-26639 at 3/24/22, 10:13 PM: Here's another example

[jira] [Created] (SPARK-38649) Fix SECURITY.md

Bjørn Jørgensen created SPARK-38649: --- Summary: Fix SECURITY.md Key: SPARK-38649 URL: https://issues.apache.org/jira/browse/SPARK-38649 Project: Spark Issue Type: Bug Components:

[jira] [Assigned] (SPARK-38647) Add SupportsReportOrdering mix in interface for Scan

[ https://issues.apache.org/jira/browse/SPARK-38647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38647: Assignee: (was: Apache Spark) > Add SupportsReportOrdering mix in interface for Scan

[jira] [Commented] (SPARK-38647) Add SupportsReportOrdering mix in interface for Scan

[ https://issues.apache.org/jira/browse/SPARK-38647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17511943#comment-17511943 ] Apache Spark commented on SPARK-38647: -- User 'EnricoMi' has created a pull request for this issue:

[jira] [Assigned] (SPARK-38647) Add SupportsReportOrdering mix in interface for Scan

[ https://issues.apache.org/jira/browse/SPARK-38647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38647: Assignee: Apache Spark > Add SupportsReportOrdering mix in interface for Scan >

[jira] [Updated] (SPARK-38438) Can't update spark.jars.packages on existing global/default context

[ https://issues.apache.org/jira/browse/SPARK-38438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-38438: - Issue Type: Improvement (was: Bug) Priority: Minor (was: Major) This is not a bug. As

[jira] [Resolved] (SPARK-38346) Add cache in MLlib BinaryClassificationMetrics

[ https://issues.apache.org/jira/browse/SPARK-38346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-38346. -- Resolution: Won't Fix > Add cache in MLlib BinaryClassificationMetrics >

[jira] [Resolved] (SPARK-38210) Spark documentation build README is stale

[ https://issues.apache.org/jira/browse/SPARK-38210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-38210. -- Resolution: Not A Problem > Spark documentation build README is stale >

[jira] [Resolved] (SPARK-38202) Invalid URL in SparkContext.addedJars will constantly fails Executor.run()

[ https://issues.apache.org/jira/browse/SPARK-38202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-38202. -- Resolution: Not A Problem > Invalid URL in SparkContext.addedJars will constantly fails

[jira] [Resolved] (SPARK-37532) RDD name could be very long and memory costly

[ https://issues.apache.org/jira/browse/SPARK-37532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-37532. -- Resolution: Won't Fix > RDD name could be very long and memory costly >

[jira] [Created] (SPARK-38648) SPIP: Simplified API for DL Inferencing

Lee Yang created SPARK-38648: Summary: SPIP: Simplified API for DL Inferencing Key: SPARK-38648 URL: https://issues.apache.org/jira/browse/SPARK-38648 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-38647) Add SupportsReportOrdering mix in interface for Scan

[ https://issues.apache.org/jira/browse/SPARK-38647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enrico Minack updated SPARK-38647: -- Description: As {{SupportsReportPartitioning}} allows implementations of {{Scan}} provide

[jira] [Created] (SPARK-38647) Add SupportsReportOrdering mix in interface for Scan

Enrico Minack created SPARK-38647: - Summary: Add SupportsReportOrdering mix in interface for Scan Key: SPARK-38647 URL: https://issues.apache.org/jira/browse/SPARK-38647 Project: Spark Issue

[jira] [Resolved] (SPARK-37463) Read/Write Timestamp ntz from/to Orc uses int64

[ https://issues.apache.org/jira/browse/SPARK-37463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-37463. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34984

[jira] [Assigned] (SPARK-37463) Read/Write Timestamp ntz from/to Orc uses int64

[ https://issues.apache.org/jira/browse/SPARK-37463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-37463: --- Assignee: jiaan.geng > Read/Write Timestamp ntz from/to Orc uses int64 >

[jira] [Assigned] (SPARK-38646) Pull a trait out for Python functions

[ https://issues.apache.org/jira/browse/SPARK-38646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38646: Assignee: (was: Apache Spark) > Pull a trait out for Python functions >

[jira] [Commented] (SPARK-38646) Pull a trait out for Python functions

[ https://issues.apache.org/jira/browse/SPARK-38646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17511886#comment-17511886 ] Apache Spark commented on SPARK-38646: -- User 'zhenlineo' has created a pull request for this issue:

[jira] [Commented] (SPARK-38646) Pull a trait out for Python functions

[ https://issues.apache.org/jira/browse/SPARK-38646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17511887#comment-17511887 ] Apache Spark commented on SPARK-38646: -- User 'zhenlineo' has created a pull request for this issue:

[jira] [Assigned] (SPARK-38646) Pull a trait out for Python functions

[ https://issues.apache.org/jira/browse/SPARK-38646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38646: Assignee: Apache Spark > Pull a trait out for Python functions >

[jira] [Updated] (SPARK-37568) Support 2-arguments by the convert_timezone() function

[ https://issues.apache.org/jira/browse/SPARK-37568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-37568: - Fix Version/s: 3.3.0 > Support 2-arguments by the convert_timezone() function >

[jira] [Created] (SPARK-38646) Pull a trait out for Python functions

Zhen Li created SPARK-38646: --- Summary: Pull a trait out for Python functions Key: SPARK-38646 URL: https://issues.apache.org/jira/browse/SPARK-38646 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-38639) Support ignoreCorruptRecord flag to ensure querying broken sequence file table smoothly

[ https://issues.apache.org/jira/browse/SPARK-38639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17511874#comment-17511874 ] Apache Spark commented on SPARK-38639: -- User 'TonyDoen' has created a pull request for this issue:

[jira] [Commented] (SPARK-38639) Support ignoreCorruptRecord flag to ensure querying broken sequence file table smoothly

[ https://issues.apache.org/jira/browse/SPARK-38639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17511873#comment-17511873 ] Apache Spark commented on SPARK-38639: -- User 'TonyDoen' has created a pull request for this issue:

[jira] [Commented] (SPARK-38645) Support `spark.sql.codegen.cleanedSourcePrint` flag to print Codegen cleanedSource

[ https://issues.apache.org/jira/browse/SPARK-38645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17511858#comment-17511858 ] tonydoen commented on SPARK-38645: -- related pr : [https://github.com/apache/spark/pull/35962] >

[jira] [Commented] (SPARK-38639) Support ignoreCorruptRecord flag to ensure querying broken sequence file table smoothly

[ https://issues.apache.org/jira/browse/SPARK-38639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17511855#comment-17511855 ] Apache Spark commented on SPARK-38639: -- User 'TonyDoen' has created a pull request for this issue:

[jira] [Commented] (SPARK-38639) Support ignoreCorruptRecord flag to ensure querying broken sequence file table smoothly

[ https://issues.apache.org/jira/browse/SPARK-38639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17511854#comment-17511854 ] Apache Spark commented on SPARK-38639: -- User 'TonyDoen' has created a pull request for this issue:

[jira] [Created] (SPARK-38645) Support `spark.sql.codegen.cleanedSourcePrint` flag to print Codegen cleanedSource

tonydoen created SPARK-38645: Summary: Support `spark.sql.codegen.cleanedSourcePrint` flag to print Codegen cleanedSource Key: SPARK-38645 URL: https://issues.apache.org/jira/browse/SPARK-38645 Project:

[jira] [Assigned] (SPARK-38644) DS V2 topN push-down supports project with alias

[ https://issues.apache.org/jira/browse/SPARK-38644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38644: Assignee: (was: Apache Spark) > DS V2 topN push-down supports project with alias >

[jira] [Commented] (SPARK-38644) DS V2 topN push-down supports project with alias

[ https://issues.apache.org/jira/browse/SPARK-38644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17511846#comment-17511846 ] Apache Spark commented on SPARK-38644: -- User 'beliefer' has created a pull request for this issue:

[jira] [Assigned] (SPARK-38644) DS V2 topN push-down supports project with alias

[ https://issues.apache.org/jira/browse/SPARK-38644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38644: Assignee: Apache Spark > DS V2 topN push-down supports project with alias >

[jira] [Updated] (SPARK-38644) DS V2 topN push-down supports project with alias

[ https://issues.apache.org/jira/browse/SPARK-38644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-38644: --- Summary: DS V2 topN push-down supports project with alias (was: DS V2 aggregate push-down supports

[jira] [Updated] (SPARK-38644) DS V2 aggregate push-down supports project with alias

[ https://issues.apache.org/jira/browse/SPARK-38644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-38644: --- Issue Type: Improvement (was: New Feature) > DS V2 aggregate push-down supports project with alias

[jira] [Created] (SPARK-38644) DS V2 aggregate push-down supports project with alias

jiaan.geng created SPARK-38644: -- Summary: DS V2 aggregate push-down supports project with alias Key: SPARK-38644 URL: https://issues.apache.org/jira/browse/SPARK-38644 Project: Spark Issue

[jira] [Assigned] (SPARK-38461) Use error classes in org.apache.spark.broadcast

[ https://issues.apache.org/jira/browse/SPARK-38461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38461: Assignee: (was: Apache Spark) > Use error classes in org.apache.spark.broadcast >

[jira] [Assigned] (SPARK-38461) Use error classes in org.apache.spark.broadcast

[ https://issues.apache.org/jira/browse/SPARK-38461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38461: Assignee: Apache Spark > Use error classes in org.apache.spark.broadcast >

[jira] [Commented] (SPARK-38461) Use error classes in org.apache.spark.broadcast

[ https://issues.apache.org/jira/browse/SPARK-38461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17511834#comment-17511834 ] Apache Spark commented on SPARK-38461: -- User 'bozhang2820' has created a pull request for this

[jira] [Commented] (SPARK-18105) LZ4 failed to decompress a stream of shuffled data

[ https://issues.apache.org/jira/browse/SPARK-18105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17511815#comment-17511815 ] hujiahua commented on SPARK-18105: -- It's working in my case by setting spark.file.transferTo=false.

[jira] [Commented] (SPARK-38640) NPE with unpersisting memory-only RDD with RDD fetching from shuffle service enabled

[ https://issues.apache.org/jira/browse/SPARK-38640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17511809#comment-17511809 ] Apache Spark commented on SPARK-38640: -- User 'Kimahriman' has created a pull request for this

[jira] [Assigned] (SPARK-38640) NPE with unpersisting memory-only RDD with RDD fetching from shuffle service enabled

[ https://issues.apache.org/jira/browse/SPARK-38640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38640: Assignee: (was: Apache Spark) > NPE with unpersisting memory-only RDD with RDD

[jira] [Commented] (SPARK-38640) NPE with unpersisting memory-only RDD with RDD fetching from shuffle service enabled

[ https://issues.apache.org/jira/browse/SPARK-38640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17511810#comment-17511810 ] Apache Spark commented on SPARK-38640: -- User 'Kimahriman' has created a pull request for this

[jira] [Assigned] (SPARK-38640) NPE with unpersisting memory-only RDD with RDD fetching from shuffle service enabled

[ https://issues.apache.org/jira/browse/SPARK-38640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38640: Assignee: Apache Spark > NPE with unpersisting memory-only RDD with RDD fetching from

[jira] [Commented] (SPARK-38627) TypeError: Datetime subtraction can only be applied to datetime series

[ https://issues.apache.org/jira/browse/SPARK-38627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17511790#comment-17511790 ] Hyukjin Kwon commented on SPARK-38627: -- I used Mac. I haven't tested it Spark 3.2. Can you show the

[jira] [Resolved] (SPARK-38636) AttributeError: module 'pyspark.pandas' has no attribute 'Timestamp'

[ https://issues.apache.org/jira/browse/SPARK-38636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-38636. -- Resolution: Not A Problem > AttributeError: module 'pyspark.pandas' has no attribute

[jira] [Commented] (SPARK-38643) Validate input dataset of ml.regression

[ https://issues.apache.org/jira/browse/SPARK-38643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17511771#comment-17511771 ] Apache Spark commented on SPARK-38643: -- User 'zhengruifeng' has created a pull request for this

[jira] [Assigned] (SPARK-38643) Validate input dataset of ml.regression

[ https://issues.apache.org/jira/browse/SPARK-38643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38643: Assignee: (was: Apache Spark) > Validate input dataset of ml.regression >

[jira] [Assigned] (SPARK-38643) Validate input dataset of ml.regression

[ https://issues.apache.org/jira/browse/SPARK-38643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38643: Assignee: Apache Spark > Validate input dataset of ml.regression >

[jira] [Commented] (SPARK-38643) Validate input dataset of ml.regression

[ https://issues.apache.org/jira/browse/SPARK-38643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17511770#comment-17511770 ] Apache Spark commented on SPARK-38643: -- User 'zhengruifeng' has created a pull request for this

[jira] [Created] (SPARK-38643) Validate input dataset of ml.regression

zhengruifeng created SPARK-38643: Summary: Validate input dataset of ml.regression Key: SPARK-38643 URL: https://issues.apache.org/jira/browse/SPARK-38643 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-38627) TypeError: Datetime subtraction can only be applied to datetime series

[ https://issues.apache.org/jira/browse/SPARK-38627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17511765#comment-17511765 ] Prakhar Sandhu commented on SPARK-38627: Hi [~hyukjin.kwon] , Great ^^ # Did it work on spark

[jira] (SPARK-38627) TypeError: Datetime subtraction can only be applied to datetime series

[ https://issues.apache.org/jira/browse/SPARK-38627 ] Prakhar Sandhu deleted comment on SPARK-38627: was (Author: JIRAUSER286645): Hi [~hyukjin.kwon] , Nice ^^ # Did it work on spark 3.3? # What environment are you using? I have set up a

[jira] [Commented] (SPARK-38627) TypeError: Datetime subtraction can only be applied to datetime series

[ https://issues.apache.org/jira/browse/SPARK-38627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17511763#comment-17511763 ] Prakhar Sandhu commented on SPARK-38627: Hi [~hyukjin.kwon] , Nice ^^ # Did it work on spark

[jira] [Commented] (SPARK-38627) TypeError: Datetime subtraction can only be applied to datetime series

[ https://issues.apache.org/jira/browse/SPARK-38627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17511752#comment-17511752 ] Hyukjin Kwon commented on SPARK-38627: -- ^^ this works. Although you have to explicitly call

[jira] [Commented] (SPARK-38636) AttributeError: module 'pyspark.pandas' has no attribute 'Timestamp'

[ https://issues.apache.org/jira/browse/SPARK-38636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17511751#comment-17511751 ] Prakhar Sandhu commented on SPARK-38636: Hi [~hyukjin.kwon] ,  I was able to pass the above

[jira] [Commented] (SPARK-38627) TypeError: Datetime subtraction can only be applied to datetime series

[ https://issues.apache.org/jira/browse/SPARK-38627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17511748#comment-17511748 ] Prakhar Sandhu commented on SPARK-38627: Hi [~hyukjin.kwon] ,  I am not sure if I would be able

[jira] [Commented] (SPARK-25789) Support for Dataset of Avro

[ https://issues.apache.org/jira/browse/SPARK-25789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17511747#comment-17511747 ] IKozar commented on SPARK-25789: my development is also blocked due to the issue.  what are the

[jira] [Updated] (SPARK-38614) After Spark update, df.show() shows incorrect F.percent_rank results

[ https://issues.apache.org/jira/browse/SPARK-38614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZygD updated SPARK-38614: - Summary: After Spark update, df.show() shows incorrect F.percent_rank results (was: df.show() shows incorrect

[jira] [Commented] (SPARK-37568) Support 2-arguments by the convert_timezone() function

[ https://issues.apache.org/jira/browse/SPARK-37568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17511693#comment-17511693 ] Apache Spark commented on SPARK-37568: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Commented] (SPARK-37568) Support 2-arguments by the convert_timezone() function

[ https://issues.apache.org/jira/browse/SPARK-37568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17511692#comment-17511692 ] Apache Spark commented on SPARK-37568: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Created] (SPARK-38642) spark-sql can not enable isolatedClientLoader to extend dsv2 catalog when using builtin hiveMetastoreJar

suheng.cloud created SPARK-38642: Summary: spark-sql can not enable isolatedClientLoader to extend dsv2 catalog when using builtin hiveMetastoreJar Key: SPARK-38642 URL:

[jira] [Updated] (SPARK-38588) Validate input dataset of ml.classification

[ https://issues.apache.org/jira/browse/SPARK-38588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng updated SPARK-38588: - Fix Version/s: 3.4.0 > Validate input dataset of ml.classification >

[jira] [Resolved] (SPARK-38588) Validate input dataset of ml.classification

[ https://issues.apache.org/jira/browse/SPARK-38588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng resolved SPARK-38588. -- Resolution: Resolved > Validate input dataset of ml.classification >

[jira] [Resolved] (SPARK-37568) Support 2-arguments by the convert_timezone() function

[ https://issues.apache.org/jira/browse/SPARK-37568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-37568. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 35951

[jira] [Assigned] (SPARK-37568) Support 2-arguments by the convert_timezone() function

[ https://issues.apache.org/jira/browse/SPARK-37568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-37568: Assignee: Max Gekk > Support 2-arguments by the convert_timezone() function >

[jira] [Updated] (SPARK-38623) Add more comments and tests for HashShuffleSpec

[ https://issues.apache.org/jira/browse/SPARK-38623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-38623: Summary: Add more comments and tests for HashShuffleSpec (was: Simplify the compatibility check

[jira] [Resolved] (SPARK-38063) Support SQL split_part function

[ https://issues.apache.org/jira/browse/SPARK-38063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-38063. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35352

[jira] [Assigned] (SPARK-38063) Support SQL split_part function

[ https://issues.apache.org/jira/browse/SPARK-38063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-38063: --- Assignee: Rui Wang > Support SQL split_part function > --- > >

[jira] [Assigned] (SPARK-38585) Simplify the code of TreeNode.clone()

[ https://issues.apache.org/jira/browse/SPARK-38585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-38585: --- Assignee: Yang Jie > Simplify the code of TreeNode.clone() >

[jira] [Resolved] (SPARK-38585) Simplify the code of TreeNode.clone()

[ https://issues.apache.org/jira/browse/SPARK-38585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-38585. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 35890