[
https://issues.apache.org/jira/browse/SPARK-42905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dronzer updated SPARK-42905:
Labels: correctness (was: )
> pyspark.ml.stat.Correlation - Spearman Correlation method giving incorrect
[
https://issues.apache.org/jira/browse/SPARK-42905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dronzer updated SPARK-42905:
Attachment: image-2023-03-23-10-55-26-879.png
> pyspark.ml.stat.Correlation - Spearman Correlation method
[
https://issues.apache.org/jira/browse/SPARK-42905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dronzer updated SPARK-42905:
Description:
pyspark.ml.stat.Correlation
Following is the Scenario where the Correlation function fails
[
https://issues.apache.org/jira/browse/SPARK-42905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dronzer updated SPARK-42905:
Attachment: image-2023-03-23-10-53-37-461.png
> pyspark.ml.stat.Correlation - Spearman Correlation method
[
https://issues.apache.org/jira/browse/SPARK-42905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dronzer updated SPARK-42905:
Attachment: image-2023-03-23-10-52-49-392.png
> pyspark.ml.stat.Correlation - Spearman Correlation method
[
https://issues.apache.org/jira/browse/SPARK-42905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dronzer updated SPARK-42905:
Attachment: image-2023-03-23-10-52-11-481.png
> pyspark.ml.stat.Correlation - Spearman Correlation method
[
https://issues.apache.org/jira/browse/SPARK-42905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dronzer updated SPARK-42905:
Attachment: image-2023-03-23-10-51-28-420.png
> pyspark.ml.stat.Correlation - Spearman Correlation method
dronzer created SPARK-42905:
---
Summary: pyspark.ml.stat.Correlation - Spearman Correlation method
giving incorrect and inconsistent results for the same DataFrame if it has huge
amount of Ties.
Key: SPARK-42905
URL:
Kent Yao created SPARK-42904:
Summary: Char/Varchar Support for JDBC Catalog
Key: SPARK-42904
URL: https://issues.apache.org/jira/browse/SPARK-42904
Project: Spark
Issue Type: Improvement
[
https://issues.apache.org/jira/browse/SPARK-42903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon updated SPARK-42903:
-
Priority: Trivial (was: Major)
> Avoid documenting None as as a return value in docstring
>
[
https://issues.apache.org/jira/browse/SPARK-42863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon resolved SPARK-42863.
--
Resolution: Done
> Review and fix issues in PySpark API docs
>
Hyukjin Kwon created SPARK-42903:
Summary: Avoid documenting None as as a return value in docstring
Key: SPARK-42903
URL: https://issues.apache.org/jira/browse/SPARK-42903
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-42901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon resolved SPARK-42901.
--
Fix Version/s: 3.5.0
Resolution: Fixed
Issue resolved by pull request 40518
[
https://issues.apache.org/jira/browse/SPARK-42901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon reassigned SPARK-42901:
Assignee: Yang Jie
> Should move message StorageLevel from `base.proto` to a separate
JacobZheng created SPARK-42902:
--
Summary: CVE-2020-13936 request for upgrading version of Velocity
Key: SPARK-42902
URL: https://issues.apache.org/jira/browse/SPARK-42902
Project: Spark
Issue
[
https://issues.apache.org/jira/browse/SPARK-42901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yang Jie updated SPARK-42901:
-
Summary: Should move message StorageLevel from `base.proto` to a separate
file (was: Should move
[
https://issues.apache.org/jira/browse/SPARK-41500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
JacobZheng resolved SPARK-41500.
Resolution: Won't Fix
> auto generate concat as Double when string minus an INTERVAL type
>
[
https://issues.apache.org/jira/browse/SPARK-42899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon resolved SPARK-42899.
--
Fix Version/s: 3.4.1
Assignee: Takuya Ueshin
Resolution: Fixed
Fixed in
[
https://issues.apache.org/jira/browse/SPARK-42901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yang Jie updated SPARK-42901:
-
Summary: Should move message StorageLevel from `base.proto` into a separate
file (was: Should move
[
https://issues.apache.org/jira/browse/SPARK-42901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yang Jie updated SPARK-42901:
-
Summary: Should move message StorageLevel from into a separate file (was:
Should move StorageLevel
[
https://issues.apache.org/jira/browse/SPARK-42901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yang Jie updated SPARK-42901:
-
Description:
[https://github.com/apache/spark/pull/40510] introduce `message StorageLevel`
to
[
https://issues.apache.org/jira/browse/SPARK-42901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yang Jie updated SPARK-42901:
-
Summary: Should move message StorageLevel from `base.proto` to a separate
file (was: Should move
[
https://issues.apache.org/jira/browse/SPARK-42901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yang Jie updated SPARK-42901:
-
Summary: Should move StorageLevel into a separate file (was: Move
StorageLevel into a separate file to
Yang Jie created SPARK-42901:
Summary: Move StorageLevel into a separate file to avoid potential
file recursively imports
Key: SPARK-42901
URL: https://issues.apache.org/jira/browse/SPARK-42901
Project:
[
https://issues.apache.org/jira/browse/SPARK-42895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17703815#comment-17703815
]
Rui Wang commented on SPARK-42895:
--
Of course though it probably need a better error message locally.
[
https://issues.apache.org/jira/browse/SPARK-42748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Herman van Hövell resolved SPARK-42748.
---
Fix Version/s: 3.5.0
Resolution: Fixed
> Server-side Artifact Management
>
[
https://issues.apache.org/jira/browse/SPARK-42895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17703814#comment-17703814
]
Rui Wang commented on SPARK-42895:
--
In fact spark.stop() does not stop the remote spark session but
Takuya Ueshin created SPARK-42900:
-
Summary: Fix createDataFrame to respect both type inference and
column names.
Key: SPARK-42900
URL: https://issues.apache.org/jira/browse/SPARK-42900
Project:
[
https://issues.apache.org/jira/browse/SPARK-42899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takuya Ueshin updated SPARK-42899:
--
Summary: DataFrame.to(schema) fails when it contains non-nullable nested
field in nullable
[
https://issues.apache.org/jira/browse/SPARK-42899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Takuya Ueshin updated SPARK-42899:
--
Description:
{{DataFrame.to(schema)}} fails when it contains non-nullable nested field in
Takuya Ueshin created SPARK-42899:
-
Summary: DataFrame.to(schema) fails with the schema of itself.
Key: SPARK-42899
URL: https://issues.apache.org/jira/browse/SPARK-42899
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-42892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Herman van Hövell resolved SPARK-42892.
---
Fix Version/s: 3.5.0
Resolution: Fixed
> Move sameType and relevant methods
[
https://issues.apache.org/jira/browse/SPARK-42832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun resolved SPARK-42832.
---
Fix Version/s: 3.5.0
Resolution: Fixed
Issue resolved by pull request 40462
[
https://issues.apache.org/jira/browse/SPARK-42832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun reassigned SPARK-42832:
-
Assignee: Yuming Wang
> Remove repartition if it is the child of LocalLimit
>
[
https://issues.apache.org/jira/browse/SPARK-42894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun reassigned SPARK-42894:
-
Assignee: Yang Jie
> Implement cache, persist, unpersist, and storageLevel
>
[
https://issues.apache.org/jira/browse/SPARK-42894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun resolved SPARK-42894.
---
Fix Version/s: 3.4.1
Resolution: Fixed
Issue resolved by pull request 40516
[
https://issues.apache.org/jira/browse/SPARK-42898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-42898:
Assignee: (was: Apache Spark)
> Cast from string to date and date to string say
[
https://issues.apache.org/jira/browse/SPARK-42898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17703720#comment-17703720
]
Apache Spark commented on SPARK-42898:
--
User 'revans2' has created a pull request for this issue:
[
https://issues.apache.org/jira/browse/SPARK-42898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-42898:
Assignee: Apache Spark
> Cast from string to date and date to string say timezone is
Robert Joseph Evans created SPARK-42898:
---
Summary: Cast from string to date and date to string say timezone
is needed, but it is not used
Key: SPARK-42898
URL:
[
https://issues.apache.org/jira/browse/SPARK-42815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenchen Fan reassigned SPARK-42815:
---
Assignee: XiDuo You
> Subexpression elimination support shortcut expression
>
[
https://issues.apache.org/jira/browse/SPARK-42897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17703686#comment-17703686
]
Apache Spark commented on SPARK-42897:
--
User 'wankunde' has created a pull request for this issue:
[
https://issues.apache.org/jira/browse/SPARK-42897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-42897:
Assignee: Apache Spark
> Avoid evaluate more than once for the variables from the left
[
https://issues.apache.org/jira/browse/SPARK-42897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-42897:
Assignee: (was: Apache Spark)
> Avoid evaluate more than once for the variables from
[
https://issues.apache.org/jira/browse/SPARK-42815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenchen Fan resolved SPARK-42815.
-
Fix Version/s: 3.5.0
Resolution: Fixed
Issue resolved by pull request 40446
Wan Kun created SPARK-42897:
---
Summary: Avoid evaluate more than once for the variables from the
left side in the FullOuter SMJ condition
Key: SPARK-42897
URL: https://issues.apache.org/jira/browse/SPARK-42897
[
https://issues.apache.org/jira/browse/SPARK-42101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17703677#comment-17703677
]
Apache Spark commented on SPARK-42101:
--
User 'cloud-fan' has created a pull request for this issue:
[
https://issues.apache.org/jira/browse/SPARK-42896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17703637#comment-17703637
]
Apache Spark commented on SPARK-42896:
--
User 'WeichenXu123' has created a pull request for this
[
https://issues.apache.org/jira/browse/SPARK-42896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-42896:
Assignee: (was: Apache Spark)
> Make `mapInPandas` / mapInArrow` support barrier
[
https://issues.apache.org/jira/browse/SPARK-42896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-42896:
Assignee: Apache Spark
> Make `mapInPandas` / mapInArrow` support barrier mode execution
Weichen Xu created SPARK-42896:
--
Summary: Make `mapInPandas` / mapInArrow` support barrier mode
execution
Key: SPARK-42896
URL: https://issues.apache.org/jira/browse/SPARK-42896
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-42864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17703615#comment-17703615
]
Apache Spark commented on SPARK-42864:
--
User 'zhengruifeng' has created a pull request for this
[
https://issues.apache.org/jira/browse/SPARK-42864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17703614#comment-17703614
]
Apache Spark commented on SPARK-42864:
--
User 'zhengruifeng' has created a pull request for this
Allison Wang created SPARK-42895:
Summary: ValueError when invoking any session operations on a
stopped Spark session
Key: SPARK-42895
URL: https://issues.apache.org/jira/browse/SPARK-42895
Project:
[
https://issues.apache.org/jira/browse/SPARK-42889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17703541#comment-17703541
]
Apache Spark commented on SPARK-42889:
--
User 'LuciferYang' has created a pull request for this
[
https://issues.apache.org/jira/browse/SPARK-42874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon resolved SPARK-42874.
--
Fix Version/s: 3.5.0
Resolution: Fixed
Issue resolved by pull request 40496
[
https://issues.apache.org/jira/browse/SPARK-42874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon reassigned SPARK-42874:
Assignee: Daniel
> Enable new golden file test framework for analysis for all input
[
https://issues.apache.org/jira/browse/SPARK-42893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon reassigned SPARK-42893:
Assignee: Xinrong Meng
> Block Arrow-optimized Python UDFs
>
[
https://issues.apache.org/jira/browse/SPARK-42893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon resolved SPARK-42893.
--
Fix Version/s: 3.4.0
Resolution: Fixed
Issue resolved by pull request 40513
[
https://issues.apache.org/jira/browse/SPARK-41233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon reassigned SPARK-41233:
Assignee: Takuya Ueshin
> High-order function: array_prepend
>
[
https://issues.apache.org/jira/browse/SPARK-42508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17703503#comment-17703503
]
Apache Spark commented on SPARK-42508:
--
User 'zhengruifeng' has created a pull request for this
[
https://issues.apache.org/jira/browse/SPARK-40082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mridul Muralidharan resolved SPARK-40082.
-
Fix Version/s: 3.5.0
Assignee: Fencheng Mei
Resolution: Fixed
>
62 matches
Mail list logo