[jira] [Resolved] (SPARK-38643) Validate input dataset of ml.regression

2022-03-24 Thread Huaxin Gao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxin Gao resolved SPARK-38643. Assignee: zhengruifeng Resolution: Fixed > Validate input dataset of ml.regression > ---

[jira] [Assigned] (SPARK-38651) Writing out empty or nested empty schemas in Datasource should be configurable

2022-03-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38651: Assignee: Apache Spark > Writing out empty or nested empty schemas in Datasource should b

[jira] [Commented] (SPARK-38651) Writing out empty or nested empty schemas in Datasource should be configurable

2022-03-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17512202#comment-17512202 ] Apache Spark commented on SPARK-38651: -- User 'thejdeep' has created a pull request

[jira] [Assigned] (SPARK-38651) Writing out empty or nested empty schemas in Datasource should be configurable

2022-03-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38651: Assignee: (was: Apache Spark) > Writing out empty or nested empty schemas in Datasour

[jira] [Commented] (SPARK-38654) Show default index type in SQL plans for pandas API on Spark

2022-03-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17512198#comment-17512198 ] Apache Spark commented on SPARK-38654: -- User 'HyukjinKwon' has created a pull reque

[jira] [Commented] (SPARK-38654) Show default index type in SQL plans for pandas API on Spark

2022-03-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17512197#comment-17512197 ] Apache Spark commented on SPARK-38654: -- User 'HyukjinKwon' has created a pull reque

[jira] [Assigned] (SPARK-38654) Show default index type in SQL plans for pandas API on Spark

2022-03-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38654: Assignee: Apache Spark > Show default index type in SQL plans for pandas API on Spark > -

[jira] [Assigned] (SPARK-38654) Show default index type in SQL plans for pandas API on Spark

2022-03-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38654: Assignee: (was: Apache Spark) > Show default index type in SQL plans for pandas API o

[jira] [Updated] (SPARK-38654) Show default index type in SQL plans for pandas API on Spark

2022-03-24 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-38654: - Priority: Minor (was: Major) > Show default index type in SQL plans for pandas API on Spark > -

[jira] [Created] (SPARK-38654) Show default index type in SQL plans for pandas API on Spark

2022-03-24 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-38654: Summary: Show default index type in SQL plans for pandas API on Spark Key: SPARK-38654 URL: https://issues.apache.org/jira/browse/SPARK-38654 Project: Spark

[jira] [Commented] (SPARK-38570) Incorrect DynamicPartitionPruning caused by Literal

2022-03-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17512195#comment-17512195 ] Apache Spark commented on SPARK-38570: -- User 'mcdull-zhang' has created a pull requ

[jira] [Resolved] (SPARK-38610) UI for Pandas API on Spark

2022-03-24 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-38610. -- Resolution: Invalid Will focus on improving existing SQL UI to show the related info. > UI fo

[jira] [Assigned] (SPARK-38646) Pull a trait out for Python functions

2022-03-24 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-38646: Assignee: Zhen Li > Pull a trait out for Python functions > -

[jira] [Resolved] (SPARK-38646) Pull a trait out for Python functions

2022-03-24 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-38646. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 35964 [https://gi

[jira] [Updated] (SPARK-38653) Repartition by Column that is Int not working properly only on particular numbers. (11, 33)

2022-03-24 Thread John Engelhart (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Engelhart updated SPARK-38653: --- Description: My understanding is when you call .repartition(a column). For each unique key

[jira] [Updated] (SPARK-38653) Repartition by Column that is Int not working properly only on particular numbers. (11, 33)

2022-03-24 Thread John Engelhart (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Engelhart updated SPARK-38653: --- Description: My understanding is when you call .repartition(a column). For each unique key

[jira] [Updated] (SPARK-38653) Repartition by Column that is Int not working properly only on particular numbers. (11, 33)

2022-03-24 Thread John Engelhart (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Engelhart updated SPARK-38653: --- Summary: Repartition by Column that is Int not working properly only on particular numbers.

[jira] [Created] (SPARK-38653) Repartition by Column that is Int not working properly only particular numbers. (11, 33)

2022-03-24 Thread John Engelhart (Jira)
John Engelhart created SPARK-38653: -- Summary: Repartition by Column that is Int not working properly only particular numbers. (11, 33) Key: SPARK-38653 URL: https://issues.apache.org/jira/browse/SPARK-38653

[jira] [Updated] (SPARK-38652) K8S IT Test DepsTestsSuite blocks with PathIOException in hadoop-aws-3.3.2

2022-03-24 Thread qian (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] qian updated SPARK-38652: - Description: DepsTestsSuite in k8s IT test is blocked with PathIOException in hadoop-aws-3.3.2. Exception Messa

[jira] [Updated] (SPARK-38652) K8S IT Test DepsTestsSuite blocks with PathIOException in hadoop-aws-3.3.2

2022-03-24 Thread qian (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] qian updated SPARK-38652: - Description: DepsTestsSuite in k8s IT test is blocked with PathIOException in hadoop-aws-3.3.2. Exception Messa

[jira] [Commented] (SPARK-38652) K8S IT Test DepsTestsSuite blocks with PathIOException in hadoop-aws-3.3.2

2022-03-24 Thread qian (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17512163#comment-17512163 ] qian commented on SPARK-38652: -- I am working on it. cc [~chaosun]  & [~dongjoon]  > K8S I

[jira] [Created] (SPARK-38652) K8S IT Test DepsTestsSuite blocks with PathIOException in hadoop-aws-3.3.2

2022-03-24 Thread qian (Jira)
qian created SPARK-38652: Summary: K8S IT Test DepsTestsSuite blocks with PathIOException in hadoop-aws-3.3.2 Key: SPARK-38652 URL: https://issues.apache.org/jira/browse/SPARK-38652 Project: Spark I

[jira] [Created] (SPARK-38651) Writing out empty or nested empty schemas in Datasource should be configurable

2022-03-24 Thread Thejdeep Gudivada (Jira)
Thejdeep Gudivada created SPARK-38651: - Summary: Writing out empty or nested empty schemas in Datasource should be configurable Key: SPARK-38651 URL: https://issues.apache.org/jira/browse/SPARK-38651

[jira] [Resolved] (SPARK-38570) Incorrect DynamicPartitionPruning caused by Literal

2022-03-24 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-38570. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35878 https://githu

[jira] [Assigned] (SPARK-38570) Incorrect DynamicPartitionPruning caused by Literal

2022-03-24 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-38570: --- Assignee: mcdull_zhang > Incorrect DynamicPartitionPruning caused by Literal >

[jira] [Updated] (SPARK-38645) Support `spark.sql.codegen.cleanedSourcePrint` flag to print Codegen cleanedSource

2022-03-24 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-38645: - Fix Version/s: (was: 3.2.1) > Support `spark.sql.codegen.cleanedSourcePrint` flag to print C

[jira] [Commented] (SPARK-38650) Better ParseException message for char without length

2022-03-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17512126#comment-17512126 ] Apache Spark commented on SPARK-38650: -- User 'anchovYu' has created a pull request

[jira] [Assigned] (SPARK-38650) Better ParseException message for char without length

2022-03-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38650: Assignee: Apache Spark > Better ParseException message for char without length >

[jira] [Assigned] (SPARK-38650) Better ParseException message for char without length

2022-03-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38650: Assignee: (was: Apache Spark) > Better ParseException message for char without length

[jira] [Created] (SPARK-38650) Better ParseException message for char without length

2022-03-24 Thread Xinyi Yu (Jira)
Xinyi Yu created SPARK-38650: Summary: Better ParseException message for char without length Key: SPARK-38650 URL: https://issues.apache.org/jira/browse/SPARK-38650 Project: Spark Issue Type: Imp

[jira] [Resolved] (SPARK-38641) Get rid of invalid configuration elements in mvn_scalafmt in main pom.xml

2022-03-24 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-38641. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 35956 [https://gi

[jira] [Assigned] (SPARK-38641) Get rid of invalid configuration elements in mvn_scalafmt in main pom.xml

2022-03-24 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-38641: Assignee: morvenhuang > Get rid of invalid configuration elements in mvn_scalafmt in main

[jira] [Comment Edited] (SPARK-26639) The reuse subquery function maybe does not work in SPARK SQL

2022-03-24 Thread Stu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17511358#comment-17511358 ] Stu edited comment on SPARK-26639 at 3/24/22, 10:13 PM: Here's a

[jira] [Created] (SPARK-38649) Fix SECURITY.md

2022-03-24 Thread Jira
Bjørn Jørgensen created SPARK-38649: --- Summary: Fix SECURITY.md Key: SPARK-38649 URL: https://issues.apache.org/jira/browse/SPARK-38649 Project: Spark Issue Type: Bug Components: D

[jira] [Assigned] (SPARK-38647) Add SupportsReportOrdering mix in interface for Scan

2022-03-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38647: Assignee: (was: Apache Spark) > Add SupportsReportOrdering mix in interface for Scan

[jira] [Commented] (SPARK-38647) Add SupportsReportOrdering mix in interface for Scan

2022-03-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17511943#comment-17511943 ] Apache Spark commented on SPARK-38647: -- User 'EnricoMi' has created a pull request

[jira] [Assigned] (SPARK-38647) Add SupportsReportOrdering mix in interface for Scan

2022-03-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38647: Assignee: Apache Spark > Add SupportsReportOrdering mix in interface for Scan > -

[jira] [Updated] (SPARK-38438) Can't update spark.jars.packages on existing global/default context

2022-03-24 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-38438: - Issue Type: Improvement (was: Bug) Priority: Minor (was: Major) This is not a bug. As di

[jira] [Resolved] (SPARK-38346) Add cache in MLlib BinaryClassificationMetrics

2022-03-24 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-38346. -- Resolution: Won't Fix > Add cache in MLlib BinaryClassificationMetrics > -

[jira] [Resolved] (SPARK-38210) Spark documentation build README is stale

2022-03-24 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-38210. -- Resolution: Not A Problem > Spark documentation build README is stale > --

[jira] [Resolved] (SPARK-38202) Invalid URL in SparkContext.addedJars will constantly fails Executor.run()

2022-03-24 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-38202. -- Resolution: Not A Problem > Invalid URL in SparkContext.addedJars will constantly fails Execut

[jira] [Resolved] (SPARK-37532) RDD name could be very long and memory costly

2022-03-24 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-37532. -- Resolution: Won't Fix > RDD name could be very long and memory costly > --

[jira] [Created] (SPARK-38648) SPIP: Simplified API for DL Inferencing

2022-03-24 Thread Lee Yang (Jira)
Lee Yang created SPARK-38648: Summary: SPIP: Simplified API for DL Inferencing Key: SPARK-38648 URL: https://issues.apache.org/jira/browse/SPARK-38648 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-38647) Add SupportsReportOrdering mix in interface for Scan

2022-03-24 Thread Enrico Minack (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enrico Minack updated SPARK-38647: -- Description: As {{SupportsReportPartitioning}} allows implementations of {{Scan}} provide Spa

[jira] [Created] (SPARK-38647) Add SupportsReportOrdering mix in interface for Scan

2022-03-24 Thread Enrico Minack (Jira)
Enrico Minack created SPARK-38647: - Summary: Add SupportsReportOrdering mix in interface for Scan Key: SPARK-38647 URL: https://issues.apache.org/jira/browse/SPARK-38647 Project: Spark Issue

[jira] [Resolved] (SPARK-37463) Read/Write Timestamp ntz from/to Orc uses int64

2022-03-24 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-37463. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34984 [https://gith

[jira] [Assigned] (SPARK-37463) Read/Write Timestamp ntz from/to Orc uses int64

2022-03-24 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-37463: --- Assignee: jiaan.geng > Read/Write Timestamp ntz from/to Orc uses int64 > --

[jira] [Assigned] (SPARK-38646) Pull a trait out for Python functions

2022-03-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38646: Assignee: (was: Apache Spark) > Pull a trait out for Python functions > -

[jira] [Commented] (SPARK-38646) Pull a trait out for Python functions

2022-03-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17511886#comment-17511886 ] Apache Spark commented on SPARK-38646: -- User 'zhenlineo' has created a pull request

[jira] [Commented] (SPARK-38646) Pull a trait out for Python functions

2022-03-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17511887#comment-17511887 ] Apache Spark commented on SPARK-38646: -- User 'zhenlineo' has created a pull request

[jira] [Assigned] (SPARK-38646) Pull a trait out for Python functions

2022-03-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38646: Assignee: Apache Spark > Pull a trait out for Python functions >

[jira] [Updated] (SPARK-37568) Support 2-arguments by the convert_timezone() function

2022-03-24 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-37568: - Fix Version/s: 3.3.0 > Support 2-arguments by the convert_timezone() function >

[jira] [Created] (SPARK-38646) Pull a trait out for Python functions

2022-03-24 Thread Zhen Li (Jira)
Zhen Li created SPARK-38646: --- Summary: Pull a trait out for Python functions Key: SPARK-38646 URL: https://issues.apache.org/jira/browse/SPARK-38646 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-38639) Support ignoreCorruptRecord flag to ensure querying broken sequence file table smoothly

2022-03-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17511874#comment-17511874 ] Apache Spark commented on SPARK-38639: -- User 'TonyDoen' has created a pull request

[jira] [Commented] (SPARK-38639) Support ignoreCorruptRecord flag to ensure querying broken sequence file table smoothly

2022-03-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17511873#comment-17511873 ] Apache Spark commented on SPARK-38639: -- User 'TonyDoen' has created a pull request

[jira] [Commented] (SPARK-38645) Support `spark.sql.codegen.cleanedSourcePrint` flag to print Codegen cleanedSource

2022-03-24 Thread tonydoen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17511858#comment-17511858 ] tonydoen commented on SPARK-38645: -- related pr : [https://github.com/apache/spark/pull/

[jira] [Commented] (SPARK-38639) Support ignoreCorruptRecord flag to ensure querying broken sequence file table smoothly

2022-03-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17511855#comment-17511855 ] Apache Spark commented on SPARK-38639: -- User 'TonyDoen' has created a pull request

[jira] [Commented] (SPARK-38639) Support ignoreCorruptRecord flag to ensure querying broken sequence file table smoothly

2022-03-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17511854#comment-17511854 ] Apache Spark commented on SPARK-38639: -- User 'TonyDoen' has created a pull request

[jira] [Created] (SPARK-38645) Support `spark.sql.codegen.cleanedSourcePrint` flag to print Codegen cleanedSource

2022-03-24 Thread tonydoen (Jira)
tonydoen created SPARK-38645: Summary: Support `spark.sql.codegen.cleanedSourcePrint` flag to print Codegen cleanedSource Key: SPARK-38645 URL: https://issues.apache.org/jira/browse/SPARK-38645 Project: S

[jira] [Assigned] (SPARK-38644) DS V2 topN push-down supports project with alias

2022-03-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38644: Assignee: (was: Apache Spark) > DS V2 topN push-down supports project with alias > --

[jira] [Commented] (SPARK-38644) DS V2 topN push-down supports project with alias

2022-03-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17511846#comment-17511846 ] Apache Spark commented on SPARK-38644: -- User 'beliefer' has created a pull request

[jira] [Assigned] (SPARK-38644) DS V2 topN push-down supports project with alias

2022-03-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38644: Assignee: Apache Spark > DS V2 topN push-down supports project with alias > -

[jira] [Updated] (SPARK-38644) DS V2 topN push-down supports project with alias

2022-03-24 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-38644: --- Summary: DS V2 topN push-down supports project with alias (was: DS V2 aggregate push-down supports

[jira] [Created] (SPARK-38644) DS V2 aggregate push-down supports project with alias

2022-03-24 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-38644: -- Summary: DS V2 aggregate push-down supports project with alias Key: SPARK-38644 URL: https://issues.apache.org/jira/browse/SPARK-38644 Project: Spark Issue Type:

[jira] [Updated] (SPARK-38644) DS V2 aggregate push-down supports project with alias

2022-03-24 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-38644: --- Issue Type: Improvement (was: New Feature) > DS V2 aggregate push-down supports project with alias

[jira] [Assigned] (SPARK-38461) Use error classes in org.apache.spark.broadcast

2022-03-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38461: Assignee: Apache Spark > Use error classes in org.apache.spark.broadcast > --

[jira] [Commented] (SPARK-38461) Use error classes in org.apache.spark.broadcast

2022-03-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17511834#comment-17511834 ] Apache Spark commented on SPARK-38461: -- User 'bozhang2820' has created a pull reque

[jira] [Assigned] (SPARK-38461) Use error classes in org.apache.spark.broadcast

2022-03-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38461: Assignee: (was: Apache Spark) > Use error classes in org.apache.spark.broadcast > ---

[jira] [Commented] (SPARK-18105) LZ4 failed to decompress a stream of shuffled data

2022-03-24 Thread hujiahua (Jira)
[ https://issues.apache.org/jira/browse/SPARK-18105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17511815#comment-17511815 ] hujiahua commented on SPARK-18105: -- It's working in my case by setting spark.file.trans

[jira] [Commented] (SPARK-38640) NPE with unpersisting memory-only RDD with RDD fetching from shuffle service enabled

2022-03-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17511809#comment-17511809 ] Apache Spark commented on SPARK-38640: -- User 'Kimahriman' has created a pull reques

[jira] [Commented] (SPARK-38640) NPE with unpersisting memory-only RDD with RDD fetching from shuffle service enabled

2022-03-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17511810#comment-17511810 ] Apache Spark commented on SPARK-38640: -- User 'Kimahriman' has created a pull reques

[jira] [Assigned] (SPARK-38640) NPE with unpersisting memory-only RDD with RDD fetching from shuffle service enabled

2022-03-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38640: Assignee: Apache Spark > NPE with unpersisting memory-only RDD with RDD fetching from shu

[jira] [Assigned] (SPARK-38640) NPE with unpersisting memory-only RDD with RDD fetching from shuffle service enabled

2022-03-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38640: Assignee: (was: Apache Spark) > NPE with unpersisting memory-only RDD with RDD fetchi

[jira] [Commented] (SPARK-38627) TypeError: Datetime subtraction can only be applied to datetime series

2022-03-24 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17511790#comment-17511790 ] Hyukjin Kwon commented on SPARK-38627: -- I used Mac. I haven't tested it Spark 3.2.

[jira] [Resolved] (SPARK-38636) AttributeError: module 'pyspark.pandas' has no attribute 'Timestamp'

2022-03-24 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-38636. -- Resolution: Not A Problem > AttributeError: module 'pyspark.pandas' has no attribute 'Timestam

[jira] [Commented] (SPARK-38643) Validate input dataset of ml.regression

2022-03-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17511771#comment-17511771 ] Apache Spark commented on SPARK-38643: -- User 'zhengruifeng' has created a pull requ

[jira] [Assigned] (SPARK-38643) Validate input dataset of ml.regression

2022-03-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38643: Assignee: (was: Apache Spark) > Validate input dataset of ml.regression > ---

[jira] [Assigned] (SPARK-38643) Validate input dataset of ml.regression

2022-03-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-38643: Assignee: Apache Spark > Validate input dataset of ml.regression > --

[jira] [Commented] (SPARK-38643) Validate input dataset of ml.regression

2022-03-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17511770#comment-17511770 ] Apache Spark commented on SPARK-38643: -- User 'zhengruifeng' has created a pull requ

[jira] [Created] (SPARK-38643) Validate input dataset of ml.regression

2022-03-24 Thread zhengruifeng (Jira)
zhengruifeng created SPARK-38643: Summary: Validate input dataset of ml.regression Key: SPARK-38643 URL: https://issues.apache.org/jira/browse/SPARK-38643 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-38627) TypeError: Datetime subtraction can only be applied to datetime series

2022-03-24 Thread Prakhar Sandhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17511765#comment-17511765 ] Prakhar Sandhu commented on SPARK-38627: Hi [~hyukjin.kwon] , Great ^^ # Did it

[jira] (SPARK-38627) TypeError: Datetime subtraction can only be applied to datetime series

2022-03-24 Thread Prakhar Sandhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38627 ] Prakhar Sandhu deleted comment on SPARK-38627: was (Author: JIRAUSER286645): Hi [~hyukjin.kwon] , Nice ^^ # Did it work on spark 3.3? # What environment are you using? I have set up a

[jira] [Commented] (SPARK-38627) TypeError: Datetime subtraction can only be applied to datetime series

2022-03-24 Thread Prakhar Sandhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17511763#comment-17511763 ] Prakhar Sandhu commented on SPARK-38627: Hi [~hyukjin.kwon] , Nice ^^ # Did it

[jira] [Commented] (SPARK-38627) TypeError: Datetime subtraction can only be applied to datetime series

2022-03-24 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17511752#comment-17511752 ] Hyukjin Kwon commented on SPARK-38627: -- ^^ this works. Although you have to explici

[jira] [Commented] (SPARK-38636) AttributeError: module 'pyspark.pandas' has no attribute 'Timestamp'

2022-03-24 Thread Prakhar Sandhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17511751#comment-17511751 ] Prakhar Sandhu commented on SPARK-38636: Hi [~hyukjin.kwon] ,  I was able to pa

[jira] [Commented] (SPARK-38627) TypeError: Datetime subtraction can only be applied to datetime series

2022-03-24 Thread Prakhar Sandhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17511748#comment-17511748 ] Prakhar Sandhu commented on SPARK-38627: Hi [~hyukjin.kwon] ,  I am not sure if

[jira] [Commented] (SPARK-25789) Support for Dataset of Avro

2022-03-24 Thread IKozar (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17511747#comment-17511747 ] IKozar commented on SPARK-25789: my development is also blocked due to the issue.  what

[jira] [Updated] (SPARK-38614) After Spark update, df.show() shows incorrect F.percent_rank results

2022-03-24 Thread ZygD (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZygD updated SPARK-38614: - Summary: After Spark update, df.show() shows incorrect F.percent_rank results (was: df.show() shows incorrect F

[jira] [Commented] (SPARK-37568) Support 2-arguments by the convert_timezone() function

2022-03-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17511693#comment-17511693 ] Apache Spark commented on SPARK-37568: -- User 'MaxGekk' has created a pull request f

[jira] [Commented] (SPARK-37568) Support 2-arguments by the convert_timezone() function

2022-03-24 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17511692#comment-17511692 ] Apache Spark commented on SPARK-37568: -- User 'MaxGekk' has created a pull request f

[jira] [Created] (SPARK-38642) spark-sql can not enable isolatedClientLoader to extend dsv2 catalog when using builtin hiveMetastoreJar

2022-03-24 Thread suheng.cloud (Jira)
suheng.cloud created SPARK-38642: Summary: spark-sql can not enable isolatedClientLoader to extend dsv2 catalog when using builtin hiveMetastoreJar Key: SPARK-38642 URL: https://issues.apache.org/jira/browse/SPARK

[jira] [Updated] (SPARK-38588) Validate input dataset of ml.classification

2022-03-24 Thread zhengruifeng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng updated SPARK-38588: - Fix Version/s: 3.4.0 > Validate input dataset of ml.classification > ---

[jira] [Resolved] (SPARK-38588) Validate input dataset of ml.classification

2022-03-24 Thread zhengruifeng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng resolved SPARK-38588. -- Resolution: Resolved > Validate input dataset of ml.classification > -

[jira] [Resolved] (SPARK-37568) Support 2-arguments by the convert_timezone() function

2022-03-24 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-37568. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 35951 [https://github.com

[jira] [Assigned] (SPARK-37568) Support 2-arguments by the convert_timezone() function

2022-03-24 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-37568: Assignee: Max Gekk > Support 2-arguments by the convert_timezone() function > ---