[jira] [Commented] (SPARK-39104) Null Pointer Exeption on unpersist call

[ https://issues.apache.org/jira/browse/SPARK-39104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17534136#comment-17534136 ] XiDuo You commented on SPARK-39104: --- it seems this bug also exists at 3.3.0 branch > Null Pointer

[jira] [Commented] (SPARK-39132) spark3.2.1 cache throw NPE

[ https://issues.apache.org/jira/browse/SPARK-39132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17534134#comment-17534134 ] XiDuo You commented on SPARK-39132: --- same bug with SPARK-39104 > spark3.2.1 cache throw NPE >

[jira] [Assigned] (SPARK-39135) DS V2 aggregate partial push-down should supports group by without aggregate functions

[ https://issues.apache.org/jira/browse/SPARK-39135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39135: Assignee: (was: Apache Spark) > DS V2 aggregate partial push-down should supports

[jira] [Commented] (SPARK-39135) DS V2 aggregate partial push-down should supports group by without aggregate functions

[ https://issues.apache.org/jira/browse/SPARK-39135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17534128#comment-17534128 ] Apache Spark commented on SPARK-39135: -- User 'beliefer' has created a pull request for this issue:

[jira] [Commented] (SPARK-39135) DS V2 aggregate partial push-down should supports group by without aggregate functions

[ https://issues.apache.org/jira/browse/SPARK-39135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17534129#comment-17534129 ] Apache Spark commented on SPARK-39135: -- User 'beliefer' has created a pull request for this issue:

[jira] [Assigned] (SPARK-39135) DS V2 aggregate partial push-down should supports group by without aggregate functions

[ https://issues.apache.org/jira/browse/SPARK-39135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39135: Assignee: Apache Spark > DS V2 aggregate partial push-down should supports group by

[jira] [Commented] (SPARK-39135) DS V2 aggregate partial push-down should supports group by without aggregate functions

[ https://issues.apache.org/jira/browse/SPARK-39135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17534127#comment-17534127 ] Apache Spark commented on SPARK-39135: -- User 'beliefer' has created a pull request for this issue:

[jira] [Updated] (SPARK-39135) DS V2 aggregate partial push-down should supports group by without aggregate functions

[ https://issues.apache.org/jira/browse/SPARK-39135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-39135: --- Description: Currently, the SQL show below not supported by DS V2 aggregate partial push-down.

[jira] [Updated] (SPARK-39135) DS V2 aggregate partial push-down should supports group by without aggregate functions

[ https://issues.apache.org/jira/browse/SPARK-39135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-39135: --- Summary: DS V2 aggregate partial push-down should supports group by without aggregate functions

[jira] [Created] (SPARK-39135) DS V2 aggregate push-down should supports group by without aggregate functions

jiaan.geng created SPARK-39135: -- Summary: DS V2 aggregate push-down should supports group by without aggregate functions Key: SPARK-39135 URL: https://issues.apache.org/jira/browse/SPARK-39135 Project:

[jira] [Commented] (SPARK-39134) Add custom metric of skipped null values for stream join operator

[ https://issues.apache.org/jira/browse/SPARK-39134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17534093#comment-17534093 ] Apache Spark commented on SPARK-39134: -- User 'viirya' has created a pull request for this issue:

[jira] [Assigned] (SPARK-39134) Add custom metric of skipped null values for stream join operator

[ https://issues.apache.org/jira/browse/SPARK-39134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39134: Assignee: (was: Apache Spark) > Add custom metric of skipped null values for stream

[jira] [Commented] (SPARK-39134) Add custom metric of skipped null values for stream join operator

[ https://issues.apache.org/jira/browse/SPARK-39134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17534092#comment-17534092 ] Apache Spark commented on SPARK-39134: -- User 'viirya' has created a pull request for this issue:

[jira] [Assigned] (SPARK-39134) Add custom metric of skipped null values for stream join operator

[ https://issues.apache.org/jira/browse/SPARK-39134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39134: Assignee: Apache Spark > Add custom metric of skipped null values for stream join

[jira] [Resolved] (SPARK-39077) Implement `skipna` of basic statistical functions of DataFrame and Series

[ https://issues.apache.org/jira/browse/SPARK-39077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-39077. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36414

[jira] [Assigned] (SPARK-39077) Implement `skipna` of basic statistical functions of DataFrame and Series

[ https://issues.apache.org/jira/browse/SPARK-39077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-39077: Assignee: Xinrong Meng > Implement `skipna` of basic statistical functions of DataFrame

[jira] [Created] (SPARK-39134) Add custom metric of skipped null values for stream join operator

L. C. Hsieh created SPARK-39134: --- Summary: Add custom metric of skipped null values for stream join operator Key: SPARK-39134 URL: https://issues.apache.org/jira/browse/SPARK-39134 Project: Spark

[jira] [Updated] (SPARK-39107) Silent change in regexp_replace's handling of empty strings

[ https://issues.apache.org/jira/browse/SPARK-39107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-39107: - Labels: correctness release-notes (was: correctness) > Silent change in regexp_replace's

[jira] [Assigned] (SPARK-39107) Silent change in regexp_replace's handling of empty strings

[ https://issues.apache.org/jira/browse/SPARK-39107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-39107: Assignee: Lorenzo Martini > Silent change in regexp_replace's handling of empty strings

[jira] [Resolved] (SPARK-39107) Silent change in regexp_replace's handling of empty strings

[ https://issues.apache.org/jira/browse/SPARK-39107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-39107. -- Fix Version/s: 3.3.0 3.2.2 3.1.3 Resolution:

[jira] [Comment Edited] (SPARK-38961) Enhance to automatically generate the pandas API support list

[ https://issues.apache.org/jira/browse/SPARK-38961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17534012#comment-17534012 ] Hyunwoo Park edited comment on SPARK-38961 at 5/9/22 9:01 PM: --

[jira] [Commented] (SPARK-38961) Enhance to automatically generate the pandas API support list

[ https://issues.apache.org/jira/browse/SPARK-38961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17534012#comment-17534012 ] Hyunwoo Park commented on SPARK-38961: -- [~hyukjin.kwon] Sure! I'm working on this :) > Enhance to

[jira] [Assigned] (SPARK-39133) Mention log level setting in PYSPARK_JVM_STACKTRACE_ENABLED

[ https://issues.apache.org/jira/browse/SPARK-39133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39133: Assignee: (was: Apache Spark) > Mention log level setting in

[jira] [Commented] (SPARK-39133) Mention log level setting in PYSPARK_JVM_STACKTRACE_ENABLED

[ https://issues.apache.org/jira/browse/SPARK-39133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17534009#comment-17534009 ] Apache Spark commented on SPARK-39133: -- User 'xinrong-databricks' has created a pull request for

[jira] [Assigned] (SPARK-39133) Mention log level setting in PYSPARK_JVM_STACKTRACE_ENABLED

[ https://issues.apache.org/jira/browse/SPARK-39133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39133: Assignee: Apache Spark > Mention log level setting in PYSPARK_JVM_STACKTRACE_ENABLED >

[jira] [Created] (SPARK-39133) Mention log level setting in PYSPARK_JVM_STACKTRACE_ENABLED

Xinrong Meng created SPARK-39133: Summary: Mention log level setting in PYSPARK_JVM_STACKTRACE_ENABLED Key: SPARK-39133 URL: https://issues.apache.org/jira/browse/SPARK-39133 Project: Spark

[jira] [Created] (SPARK-39132) spark3.2.1 cache throw NPE

cxb created SPARK-39132: --- Summary: spark3.2.1 cache throw NPE Key: SPARK-39132 URL: https://issues.apache.org/jira/browse/SPARK-39132 Project: Spark Issue Type: Bug Components: Structured

[jira] [Updated] (SPARK-39131) "Exists" is optimized too late (to LeftSemi) preventing filters to be inferred

[ https://issues.apache.org/jira/browse/SPARK-39131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alessandro Bellina updated SPARK-39131: --- Description: We would like to propose a slight change in the order of execution of

[jira] [Updated] (SPARK-39131) "Exists" is optimized too late (to LeftSemi) preventing filters to be inferred

[ https://issues.apache.org/jira/browse/SPARK-39131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alessandro Bellina updated SPARK-39131: --- Description: We would like to propose a slight change in the order of execution of

[jira] [Updated] (SPARK-39131) "Exists" is optimized too late (to LeftSemi) preventing filters to be inferred

[ https://issues.apache.org/jira/browse/SPARK-39131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alessandro Bellina updated SPARK-39131: --- Component/s: SQL > "Exists" is optimized too late (to LeftSemi) preventing filters

[jira] [Updated] (SPARK-39131) "Exists" is optimized too late (to LeftSemi) preventing filters to be inferred

[ https://issues.apache.org/jira/browse/SPARK-39131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alessandro Bellina updated SPARK-39131: --- Description: We would like to propose a slight change in the order of execution of

[jira] [Created] (SPARK-39131) "Exists" is optimized too late (to LeftSemi) preventing filters to be inferred

Alessandro Bellina created SPARK-39131: -- Summary: "Exists" is optimized too late (to LeftSemi) preventing filters to be inferred Key: SPARK-39131 URL: https://issues.apache.org/jira/browse/SPARK-39131

[jira] [Resolved] (SPARK-39130) How do I read parquet with python object

[ https://issues.apache.org/jira/browse/SPARK-39130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ben Wan resolved SPARK-39130. - Resolution: Won't Do > How do I read parquet with python object >

[jira] [Created] (SPARK-39130) How do I read parquet with python object

Ben Wan created SPARK-39130: --- Summary: How do I read parquet with python object Key: SPARK-39130 URL: https://issues.apache.org/jira/browse/SPARK-39130 Project: Spark Issue Type: Question

[jira] [Assigned] (SPARK-39112) UnsupportedOperationException if spark.sql.ui.explainMode is set to cost

[ https://issues.apache.org/jira/browse/SPARK-39112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39112: Assignee: (was: Apache Spark) > UnsupportedOperationException if

[jira] [Commented] (SPARK-39112) UnsupportedOperationException if spark.sql.ui.explainMode is set to cost

[ https://issues.apache.org/jira/browse/SPARK-39112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17533773#comment-17533773 ] Apache Spark commented on SPARK-39112: -- User 'ulysses-you' has created a pull request for this

[jira] [Assigned] (SPARK-39112) UnsupportedOperationException if spark.sql.ui.explainMode is set to cost

[ https://issues.apache.org/jira/browse/SPARK-39112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39112: Assignee: Apache Spark > UnsupportedOperationException if spark.sql.ui.explainMode is

[jira] [Commented] (SPARK-39102) Replace the usage of guava's Files.createTempDir() with java.nio.file.Files.createTempDirectory()

[ https://issues.apache.org/jira/browse/SPARK-39102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17533764#comment-17533764 ] pralabhkumar commented on SPARK-39102: -- Sure I'll work on this  > Replace the usage of guava's

[jira] [Commented] (SPARK-38961) Enhance to automatically generate the pandas API support list

[ https://issues.apache.org/jira/browse/SPARK-38961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17533733#comment-17533733 ] Hyukjin Kwon commented on SPARK-38961: -- [~beobest2] please go ahead with a PR! Maybe we can just

[jira] [Commented] (SPARK-38961) Enhance to automatically generate the pandas API support list

[ https://issues.apache.org/jira/browse/SPARK-38961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17533705#comment-17533705 ] Yikun Jiang commented on SPARK-38961: - See also here:

[jira] [Commented] (SPARK-38961) Enhance to automatically generate the pandas API support list

[ https://issues.apache.org/jira/browse/SPARK-38961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17533699#comment-17533699 ] Hyunwoo Park commented on SPARK-38961: -- How about this way? {code:python} from inspect import

[jira] [Commented] (SPARK-38939) Support ALTER TABLE ... DROP [IF EXISTS] COLUMN .. syntax

[ https://issues.apache.org/jira/browse/SPARK-38939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17533687#comment-17533687 ] Apache Spark commented on SPARK-38939: -- User 'dcoliversun' has created a pull request for this

[jira] [Commented] (SPARK-38939) Support ALTER TABLE ... DROP [IF EXISTS] COLUMN .. syntax

[ https://issues.apache.org/jira/browse/SPARK-38939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17533686#comment-17533686 ] Apache Spark commented on SPARK-38939: -- User 'dcoliversun' has created a pull request for this

[jira] [Assigned] (SPARK-39129) impl Groupby.ewm

[ https://issues.apache.org/jira/browse/SPARK-39129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39129: Assignee: Apache Spark > impl Groupby.ewm > > > Key:

[jira] [Assigned] (SPARK-39129) impl Groupby.ewm

[ https://issues.apache.org/jira/browse/SPARK-39129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39129: Assignee: (was: Apache Spark) > impl Groupby.ewm > > >

[jira] [Commented] (SPARK-39129) impl Groupby.ewm

[ https://issues.apache.org/jira/browse/SPARK-39129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17533663#comment-17533663 ] Apache Spark commented on SPARK-39129: -- User 'zhengruifeng' has created a pull request for this

[jira] [Created] (SPARK-39129) impl Groupby.ewm

zhengruifeng created SPARK-39129: Summary: impl Groupby.ewm Key: SPARK-39129 URL: https://issues.apache.org/jira/browse/SPARK-39129 Project: Spark Issue Type: Sub-task Components: