[jira] [Assigned] (SPARK-44072) Update the incorrect sql example of insert table documentation

[ https://issues.apache.org/jira/browse/SPARK-44072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-44072: --- Assignee: Yang Zhang > Update the incorrect sql example of insert table documentation >

[jira] [Updated] (SPARK-44072) Update the incorrect sql example of insert table documentation

[ https://issues.apache.org/jira/browse/SPARK-44072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-44072: Fix Version/s: (was: 3.4.1) (was: 3.3.3) > Update the incorrect sql

[jira] [Updated] (SPARK-44072) Update the incorrect sql example of insert table documentation

[ https://issues.apache.org/jira/browse/SPARK-44072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-44072: Fix Version/s: 3.3.3 3.4.1 > Update the incorrect sql example of insert table

[jira] [Updated] (SPARK-44077) Session Configs were not getting honored in RDDs

[ https://issues.apache.org/jira/browse/SPARK-44077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kapil Singh updated SPARK-44077: Description: When calling SQLConf.get on executors, the configs are read from the local

[jira] [Created] (SPARK-44077) Session Configs were not getting honored in RDDs

Kapil Singh created SPARK-44077: --- Summary: Session Configs were not getting honored in RDDs Key: SPARK-44077 URL: https://issues.apache.org/jira/browse/SPARK-44077 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-44040) Incorrect result after count distinct

[ https://issues.apache.org/jira/browse/SPARK-44040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang reassigned SPARK-44040: --- Assignee: Yuming Wang > Incorrect result after count distinct >

[jira] [Commented] (SPARK-44075) Make 'transformStatCorr' lazy

[ https://issues.apache.org/jira/browse/SPARK-44075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17733308#comment-17733308 ] Snoot.io commented on SPARK-44075: -- User 'zhengruifeng' has created a pull request for this issue:

[jira] [Commented] (SPARK-43928) Add bit operations to Scala and Python

[ https://issues.apache.org/jira/browse/SPARK-43928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17733307#comment-17733307 ] Snoot.io commented on SPARK-43928: -- User 'beliefer' has created a pull request for this issue:

[jira] [Resolved] (SPARK-44040) Incorrect result after count distinct

[ https://issues.apache.org/jira/browse/SPARK-44040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-44040. - Fix Version/s: 3.3.3 3.5.0 3.4.1 Resolution: Fixed

[jira] [Created] (SPARK-44076) SPIP: Python Data Source API

Allison Wang created SPARK-44076: Summary: SPIP: Python Data Source API Key: SPARK-44076 URL: https://issues.apache.org/jira/browse/SPARK-44076 Project: Spark Issue Type: New Feature

[jira] [Created] (SPARK-44075) Make 'transformStatCorr' lazy

Ruifeng Zheng created SPARK-44075: - Summary: Make 'transformStatCorr' lazy Key: SPARK-44075 URL: https://issues.apache.org/jira/browse/SPARK-44075 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-43474) Add support to create DataFrame Reference in Spark connect

[ https://issues.apache.org/jira/browse/SPARK-43474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17733305#comment-17733305 ] Snoot.io commented on SPARK-43474: -- User 'rangadi' has created a pull request for this issue:

[jira] [Commented] (SPARK-44025) CSV Table Read Error with CharType(length) column

[ https://issues.apache.org/jira/browse/SPARK-44025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17733304#comment-17733304 ] Snoot.io commented on SPARK-44025: -- User 'panbingkun' has created a pull request for this issue:

[jira] [Commented] (SPARK-44072) Update the incorrect sql example of insert table documentation

[ https://issues.apache.org/jira/browse/SPARK-44072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17733303#comment-17733303 ] Snoot.io commented on SPARK-44072: -- User 'Yohahaha' has created a pull request for this issue:

[jira] [Commented] (SPARK-44060) Code-gen for build side outer shuffled hash join

[ https://issues.apache.org/jira/browse/SPARK-44060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17733302#comment-17733302 ] Snoot.io commented on SPARK-44060: -- User 'szehon-ho' has created a pull request for this issue:

[jira] [Commented] (SPARK-44060) Code-gen for build side outer shuffled hash join

[ https://issues.apache.org/jira/browse/SPARK-44060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17733301#comment-17733301 ] Snoot.io commented on SPARK-44060: -- User 'szehon-ho' has created a pull request for this issue:

[jira] [Commented] (SPARK-44065) Optimize BroadcastHashJoin skew when localShuffleReader is disabled

[ https://issues.apache.org/jira/browse/SPARK-44065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17733300#comment-17733300 ] GridGain Integration commented on SPARK-44065: -- User 'wForget' has created a pull request

[jira] [Created] (SPARK-44074) `Logging plan changes for execution` test failed

Yang Jie created SPARK-44074: Summary: `Logging plan changes for execution` test failed Key: SPARK-44074 URL: https://issues.apache.org/jira/browse/SPARK-44074 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-43929) Add date time functions to Scala and Python - part 1

[ https://issues.apache.org/jira/browse/SPARK-43929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-43929: -- Description: Add following functions: * date_diff * date_from_unix_date * date_part *

[jira] [Created] (SPARK-44073) Add date time functions to Scala and Python - part 2

Ruifeng Zheng created SPARK-44073: - Summary: Add date time functions to Scala and Python - part 2 Key: SPARK-44073 URL: https://issues.apache.org/jira/browse/SPARK-44073 Project: Spark Issue

[jira] [Updated] (SPARK-43929) Add date time functions to Scala and Python - part 1

[ https://issues.apache.org/jira/browse/SPARK-43929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-43929: -- Summary: Add date time functions to Scala and Python - part 1 (was: Add date time functions

[jira] [Resolved] (SPARK-44072) Update the incorrect sql example of insert table documentation

[ https://issues.apache.org/jira/browse/SPARK-44072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-44072. -- Fix Version/s: (was: 3.4.1) (was: 3.3.3) Resolution: Fixed Issue

[jira] [Comment Edited] (SPARK-41599) Memory leak in FileSystem.CACHE when submitting apps to secure cluster using InProcessLauncher

[ https://issues.apache.org/jira/browse/SPARK-41599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17733286#comment-17733286 ] Xieming Li edited comment on SPARK-41599 at 6/16/23 3:01 AM: -

[jira] [Commented] (SPARK-41599) Memory leak in FileSystem.CACHE when submitting apps to secure cluster using InProcessLauncher

[ https://issues.apache.org/jira/browse/SPARK-41599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17733286#comment-17733286 ] Xieming Li commented on SPARK-41599: [~ste...@apache.org] [~maciejsmolenski]  I am having this

[jira] [Updated] (SPARK-44072) Update the incorrect sql example of insert table documentation

[ https://issues.apache.org/jira/browse/SPARK-44072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Zhang updated SPARK-44072: --- Description: Latest docs of insert table has an incorrect sql example about 'Insert Using a Typed

[jira] [Updated] (SPARK-44072) Update the incorrect sql example of insert table documentation

[ https://issues.apache.org/jira/browse/SPARK-44072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Zhang updated SPARK-44072: --- Description: Latest docs of insert table has an incorrect sql example about 'Insert Using a Typed

[jira] [Commented] (SPARK-43201) Inconsistency between from_avro and from_json function

[ https://issues.apache.org/jira/browse/SPARK-43201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17733281#comment-17733281 ] Jia Fan commented on SPARK-43201: - If avroSchema1 not equals avroSchema2, the dataframe's schema would

[jira] [Created] (SPARK-44072) Update the incorrect sql example of insert table documentation

Yang Zhang created SPARK-44072: -- Summary: Update the incorrect sql example of insert table documentation Key: SPARK-44072 URL: https://issues.apache.org/jira/browse/SPARK-44072 Project: Spark

[jira] [Commented] (SPARK-44065) Optimize BroadcastHashJoin skew when localShuffleReader is disabled

[ https://issues.apache.org/jira/browse/SPARK-44065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17733271#comment-17733271 ] Zhen Wang commented on SPARK-44065: --- https://github.com/apache/spark/pull/41609 > Optimize

[jira] [Assigned] (SPARK-43937) Add ifnull,isnotnull,equal_null,nullif,nvl,nvl2 to Scala and Python

[ https://issues.apache.org/jira/browse/SPARK-43937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-43937: - Assignee: BingKun Pan > Add ifnull,isnotnull,equal_null,nullif,nvl,nvl2 to Scala and

[jira] [Resolved] (SPARK-43937) Add ifnull,isnotnull,equal_null,nullif,nvl,nvl2 to Scala and Python

[ https://issues.apache.org/jira/browse/SPARK-43937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-43937. --- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41534

[jira] [Updated] (SPARK-43925) Add some, bool_or,bool_and,every to Scala and Python

[ https://issues.apache.org/jira/browse/SPARK-43925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-43925: -- Description: Add following functions: * -any- * some * bool_or * bool_and * every to: *

[jira] [Updated] (SPARK-43925) Add some, bool_or,bool_and,every to Scala and Python

[ https://issues.apache.org/jira/browse/SPARK-43925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-43925: -- Summary: Add some, bool_or,bool_and,every to Scala and Python (was: Add any, some,

[jira] [Created] (SPARK-44071) Define UnresolvedNode trait to reduce redundancy

Ryan Johnson created SPARK-44071: Summary: Define UnresolvedNode trait to reduce redundancy Key: SPARK-44071 URL: https://issues.apache.org/jira/browse/SPARK-44071 Project: Spark Issue Type:

[jira] [Commented] (SPARK-43511) Implemented State APIs for Spark Connect Scala

[ https://issues.apache.org/jira/browse/SPARK-43511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17733195#comment-17733195 ] GridGain Integration commented on SPARK-43511: -- User 'bogao007' has created a pull request

[jira] [Created] (SPARK-44070) Bump snappy-java 1.1.10.1

Cheng Pan created SPARK-44070: - Summary: Bump snappy-java 1.1.10.1 Key: SPARK-44070 URL: https://issues.apache.org/jira/browse/SPARK-44070 Project: Spark Issue Type: Dependency upgrade

[jira] [Assigned] (SPARK-44055) Remove redundant `override` from `CheckpointRDD`

[ https://issues.apache.org/jira/browse/SPARK-44055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie reassigned SPARK-44055: Assignee: Yang Jie > Remove redundant `override` from `CheckpointRDD` >

[jira] [Resolved] (SPARK-44055) Remove redundant `override` from `CheckpointRDD`

[ https://issues.apache.org/jira/browse/SPARK-44055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie resolved SPARK-44055. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41597

[jira] [Updated] (SPARK-44069) maven test ReplSuite failed

[ https://issues.apache.org/jira/browse/SPARK-44069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-44069: - Description: https://github.com/LuciferYang/spark/actions/runs/5274544416/jobs/9541917589 (was:

[jira] [Created] (SPARK-44069) maven test ReplSuite failed

Yang Jie created SPARK-44069: Summary: maven test ReplSuite failed Key: SPARK-44069 URL: https://issues.apache.org/jira/browse/SPARK-44069 Project: Spark Issue Type: Bug Components:

[jira] [Created] (SPARK-44068) Support positional parameters in Scala connect client

Max Gekk created SPARK-44068: Summary: Support positional parameters in Scala connect client Key: SPARK-44068 URL: https://issues.apache.org/jira/browse/SPARK-44068 Project: Spark Issue Type:

[jira] [Commented] (SPARK-43942) Add string functions to Scala and Python - part 1

[ https://issues.apache.org/jira/browse/SPARK-43942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17733028#comment-17733028 ] Hudson commented on SPARK-43942: User 'panbingkun' has created a pull request for this issue:

[jira] [Commented] (SPARK-43942) Add string functions to Scala and Python - part 1

[ https://issues.apache.org/jira/browse/SPARK-43942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17733027#comment-17733027 ] Hudson commented on SPARK-43942: User 'panbingkun' has created a pull request for this issue:

[jira] [Created] (SPARK-44067) Warning for the pandas-related behavior changes in next major release

Haejoon Lee created SPARK-44067: --- Summary: Warning for the pandas-related behavior changes in next major release Key: SPARK-44067 URL: https://issues.apache.org/jira/browse/SPARK-44067 Project: Spark

[jira] [Commented] (SPARK-44066) Support positional parameters in parameterized query

[ https://issues.apache.org/jira/browse/SPARK-44066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17732955#comment-17732955 ] ASF GitHub Bot commented on SPARK-44066: User 'MaxGekk' has created a pull request for this

[jira] [Commented] (SPARK-44066) Support positional parameters in parameterized query

[ https://issues.apache.org/jira/browse/SPARK-44066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17732957#comment-17732957 ] ASF GitHub Bot commented on SPARK-44066: User 'MaxGekk' has created a pull request for this

[jira] [Commented] (SPARK-43952) Cancel Spark jobs not only by a single "jobgroup", but allow multiple "job tags"

[ https://issues.apache.org/jira/browse/SPARK-43952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17732947#comment-17732947 ] ASF GitHub Bot commented on SPARK-43952: User 'juliuszsompolski' has created a pull request for

[jira] [Commented] (SPARK-38200) [SQL] Spark JDBC Savemode Supports Upsert

[ https://issues.apache.org/jira/browse/SPARK-38200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17732945#comment-17732945 ] Enrico Minack commented on SPARK-38200: --- Created pull request for this:

[jira] [Commented] (SPARK-19335) Spark should support doing an efficient DataFrame Upsert via JDBC

[ https://issues.apache.org/jira/browse/SPARK-19335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17732943#comment-17732943 ] Enrico Minack commented on SPARK-19335: --- Created pull request for this:

[jira] [Commented] (SPARK-44052) Add util to get proper Column or DataFrame class for Spark Connect.

[ https://issues.apache.org/jira/browse/SPARK-44052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17732941#comment-17732941 ] Ignite TC Bot commented on SPARK-44052: --- User 'itholic' has created a pull request for this issue:

[jira] [Commented] (SPARK-43291) Match behavior for DataFrame.cov on string DataFrame

[ https://issues.apache.org/jira/browse/SPARK-43291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17732932#comment-17732932 ] Haejoon Lee commented on SPARK-43291: - With the major release of pandas 2.0.0 on April 3, 2023,

[jira] [Created] (SPARK-44066) Support positional parameters in parameterized query

Max Gekk created SPARK-44066: Summary: Support positional parameters in parameterized query Key: SPARK-44066 URL: https://issues.apache.org/jira/browse/SPARK-44066 Project: Spark Issue Type: New

[jira] [Created] (SPARK-44065) Optimize BroadcastHashJoin skew when localShuffleReader is disabled

Zhen Wang created SPARK-44065: - Summary: Optimize BroadcastHashJoin skew when localShuffleReader is disabled Key: SPARK-44065 URL: https://issues.apache.org/jira/browse/SPARK-44065 Project: Spark

[jira] [Resolved] (SPARK-44031) Upgrade silencer to 1.7.13

[ https://issues.apache.org/jira/browse/SPARK-44031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-44031. - Fix Version/s: 3.5.0 Assignee: Dongjoon Hyun Resolution: Fixed > Upgrade

[jira] [Resolved] (SPARK-43627) Enable pyspark.pandas.spark.functions.skew in Spark Connect.

[ https://issues.apache.org/jira/browse/SPARK-43627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-43627. --- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41604

[jira] [Resolved] (SPARK-43626) Enable pyspark.pandas.spark.functions.kurt in Spark Connect.

[ https://issues.apache.org/jira/browse/SPARK-43626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-43626. --- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41604