[jira] [Created] (SPARK-44154) Bitmap functions

Entong Shen created SPARK-44154: --- Summary: Bitmap functions Key: SPARK-44154 URL: https://issues.apache.org/jira/browse/SPARK-44154 Project: Spark Issue Type: Improvement Components:

[jira] [Resolved] (SPARK-44139) Discard completely pushed down filters in group-based MERGE operations

[ https://issues.apache.org/jira/browse/SPARK-44139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-44139. --- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41700

[jira] [Assigned] (SPARK-44139) Discard completely pushed down filters in group-based MERGE operations

[ https://issues.apache.org/jira/browse/SPARK-44139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-44139: - Assignee: Anton Okolnychyi > Discard completely pushed down filters in group-based

[jira] [Created] (SPARK-44153) Support `Heap Histogram` column in Executor tab

Dongjoon Hyun created SPARK-44153: - Summary: Support `Heap Histogram` column in Executor tab Key: SPARK-44153 URL: https://issues.apache.org/jira/browse/SPARK-44153 Project: Spark Issue

[jira] [Updated] (SPARK-44152) Document Spark Connect only API in PySpark

[ https://issues.apache.org/jira/browse/SPARK-44152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-44152: - Summary: Document Spark Connect only API in PySpark (was: Document Spark Connect only API) >

[jira] [Created] (SPARK-44152) Document Spark Connect only API

Hyukjin Kwon created SPARK-44152: Summary: Document Spark Connect only API Key: SPARK-44152 URL: https://issues.apache.org/jira/browse/SPARK-44152 Project: Spark Issue Type: Documentation

[jira] [Assigned] (SPARK-44134) Can't set resources (GPU/FPGA) to 0 when they are set to positive value in spark-defaults.conf

[ https://issues.apache.org/jira/browse/SPARK-44134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-44134: - Assignee: Thomas Graves > Can't set resources (GPU/FPGA) to 0 when they are set to

[jira] [Resolved] (SPARK-44134) Can't set resources (GPU/FPGA) to 0 when they are set to positive value in spark-defaults.conf

[ https://issues.apache.org/jira/browse/SPARK-44134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-44134. --- Fix Version/s: 3.3.3 3.5.0 3.4.1 Resolution:

[jira] [Resolved] (SPARK-43990) Upgrade kubernetes-client to 6.7.1

[ https://issues.apache.org/jira/browse/SPARK-43990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-43990. --- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41490

[jira] [Assigned] (SPARK-43990) Upgrade kubernetes-client to 6.7.1

[ https://issues.apache.org/jira/browse/SPARK-43990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-43990: - Assignee: Bjørn Jørgensen > Upgrade kubernetes-client to 6.7.1 >

[jira] [Resolved] (SPARK-44110) Forked unit tests don't receive proxy system properties

[ https://issues.apache.org/jira/browse/SPARK-44110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-44110. --- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41678

[jira] [Assigned] (SPARK-44110) Forked unit tests don't receive proxy system properties

[ https://issues.apache.org/jira/browse/SPARK-44110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-44110: - Assignee: Steve Vaughan > Forked unit tests don't receive proxy system properties >

[jira] [Commented] (SPARK-43438) Fix mismatched column list error on INSERT

[ https://issues.apache.org/jira/browse/SPARK-43438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17736317#comment-17736317 ] BingKun Pan commented on SPARK-43438: - Let me to do it. > Fix mismatched column list error on

[jira] [Assigned] (SPARK-44142) Utility to convert python types to spark types compares Python "type" object rather than user's "tpe" for categorical data types

[ https://issues.apache.org/jira/browse/SPARK-44142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-44142: Assignee: Ted Chester Jenks > Utility to convert python types to spark types compares

[jira] [Resolved] (SPARK-44142) Utility to convert python types to spark types compares Python "type" object rather than user's "tpe" for categorical data types

[ https://issues.apache.org/jira/browse/SPARK-44142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-44142. -- Fix Version/s: 3.3.3 3.5.0 3.4.1 Resolution:

[jira] [Created] (SPARK-44151) Upgrade commons-codec from 1.15 to 1.16.0

BingKun Pan created SPARK-44151: --- Summary: Upgrade commons-codec from 1.15 to 1.16.0 Key: SPARK-44151 URL: https://issues.apache.org/jira/browse/SPARK-44151 Project: Spark Issue Type:

[jira] [Created] (SPARK-44150) Explicit Arrow casting for mismatched return type in Arrow Python UDF

Xinrong Meng created SPARK-44150: Summary: Explicit Arrow casting for mismatched return type in Arrow Python UDF Key: SPARK-44150 URL: https://issues.apache.org/jira/browse/SPARK-44150 Project: Spark

[jira] [Assigned] (SPARK-44138) Prohibit non-deterministic expressions, subqueries and aggregates in MERGE conditions

[ https://issues.apache.org/jira/browse/SPARK-44138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-44138: - Assignee: Anton Okolnychyi > Prohibit non-deterministic expressions, subqueries and

[jira] [Resolved] (SPARK-44138) Prohibit non-deterministic expressions, subqueries and aggregates in MERGE conditions

[ https://issues.apache.org/jira/browse/SPARK-44138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-44138. --- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41694

[jira] [Created] (SPARK-44149) Support DataFrame Merge API

Huaxin Gao created SPARK-44149: -- Summary: Support DataFrame Merge API Key: SPARK-44149 URL: https://issues.apache.org/jira/browse/SPARK-44149 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-44148) Validate NO_BROADCAST_AND_REPLICATION hint is never set on both sides

Anton Okolnychyi created SPARK-44148: Summary: Validate NO_BROADCAST_AND_REPLICATION hint is never set on both sides Key: SPARK-44148 URL: https://issues.apache.org/jira/browse/SPARK-44148

[jira] [Resolved] (SPARK-44147) Add `spark.ui.allowFramingFrom` doc to `Security` page

[ https://issues.apache.org/jira/browse/SPARK-44147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-44147. --- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41702

[jira] [Assigned] (SPARK-44147) Add `spark.ui.allowFramingFrom` doc to `Security` page

[ https://issues.apache.org/jira/browse/SPARK-44147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-44147: - Assignee: Dongjoon Hyun > Add `spark.ui.allowFramingFrom` doc to `Security` page >

[jira] [Updated] (SPARK-44147) Add `spark.ui.allowFramingFrom` doc to `Security` page

[ https://issues.apache.org/jira/browse/SPARK-44147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-44147: -- Summary: Add `spark.ui.allowFramingFrom` doc to `Security` page (was: Add

[jira] [Created] (SPARK-44147) Add `spark.ui.allowFramingFrom` doc

Dongjoon Hyun created SPARK-44147: - Summary: Add `spark.ui.allowFramingFrom` doc Key: SPARK-44147 URL: https://issues.apache.org/jira/browse/SPARK-44147 Project: Spark Issue Type:

[jira] [Updated] (SPARK-44145) Callback prior to execution

[ https://issues.apache.org/jira/browse/SPARK-44145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Francois Desjeans Gauthier updated SPARK-44145: Description: Commands are eagerly executed after analysis

[jira] [Created] (SPARK-44146) Isolate Spark Connect session/artifacts

Venkata Sai Akhil Gudesa created SPARK-44146: Summary: Isolate Spark Connect session/artifacts Key: SPARK-44146 URL: https://issues.apache.org/jira/browse/SPARK-44146 Project: Spark

[jira] [Created] (SPARK-44145) Callback prior to execution

Jean-Francois Desjeans Gauthier created SPARK-44145: --- Summary: Callback prior to execution Key: SPARK-44145 URL: https://issues.apache.org/jira/browse/SPARK-44145 Project: Spark

[jira] [Created] (SPARK-44144) Enable `spark.authenticate` by default in K8s environment

Dongjoon Hyun created SPARK-44144: - Summary: Enable `spark.authenticate` by default in K8s environment Key: SPARK-44144 URL: https://issues.apache.org/jira/browse/SPARK-44144 Project: Spark

[jira] [Updated] (SPARK-44134) Can't set resources (GPU/FPGA) to 0 when they are set to positive value in spark-defaults.conf

[ https://issues.apache.org/jira/browse/SPARK-44134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-44134: -- Description: With resource aware scheduling, if you specify a default value in the

[jira] [Commented] (SPARK-43816) Spark Corrupts Data In-Transit for High Volume (> 20 TB/hr) of Data

[ https://issues.apache.org/jira/browse/SPARK-43816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17736170#comment-17736170 ] Sai Allu commented on SPARK-43816: -- Hello Yuming, could you please explain a bit more why this new

[jira] [Created] (SPARK-44143) Use checkError() to check Exception in *DDL*Suite

BingKun Pan created SPARK-44143: --- Summary: Use checkError() to check Exception in *DDL*Suite Key: SPARK-44143 URL: https://issues.apache.org/jira/browse/SPARK-44143 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-44140) Support positional parameters in PySpark sql()

[ https://issues.apache.org/jira/browse/SPARK-44140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-44140. -- Resolution: Fixed Issue resolved by pull request 41695 [https://github.com/apache/spark/pull/41695]

[jira] [Commented] (SPARK-44142) Utility to convert python types to spark types compares Python "type" object rather than user's "tpe" for categorical data types

[ https://issues.apache.org/jira/browse/SPARK-44142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17736100#comment-17736100 ] Ted Chester Jenks commented on SPARK-44142: --- https://github.com/apache/spark/pull/41697 >

[jira] [Updated] (SPARK-44142) Utility to convert python types to spark types compares Python "type" object rather than user's "tpe" for categorical data types

[ https://issues.apache.org/jira/browse/SPARK-44142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ted Chester Jenks updated SPARK-44142: -- Description: In the typehints utility that converts python types to spark types, the

[jira] [Created] (SPARK-44142) Utility to convert python types to spark types compares Python "type" object rather than user's "tpe" for categorical data types

Ted Chester Jenks created SPARK-44142: - Summary: Utility to convert python types to spark types compares Python "type" object rather than user's "tpe" for categorical data types Key: SPARK-44142 URL:

[jira] [Commented] (SPARK-38230) InsertIntoHadoopFsRelationCommand unnecessarily fetches details of partitions in most cases

[ https://issues.apache.org/jira/browse/SPARK-38230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17736041#comment-17736041 ] ASF GitHub Bot commented on SPARK-38230: User 'jeanlyn' has created a pull request for this

[jira] [Commented] (SPARK-36680) Supports Dynamic Table Options for Spark SQL

[ https://issues.apache.org/jira/browse/SPARK-36680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17736039#comment-17736039 ] ASF GitHub Bot commented on SPARK-36680: User 'szehon-ho' has created a pull request for this

[jira] [Commented] (SPARK-36680) Supports Dynamic Table Options for Spark SQL

[ https://issues.apache.org/jira/browse/SPARK-36680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17736038#comment-17736038 ] ASF GitHub Bot commented on SPARK-36680: User 'szehon-ho' has created a pull request for this

[jira] [Updated] (SPARK-44141) Remove need to preinstall the buf compiler

[ https://issues.apache.org/jira/browse/SPARK-44141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arnar Pall updated SPARK-44141: --- Description: In order to lower the barrier of entry even further for this project we can remove

[jira] [Created] (SPARK-44141) Remove need to preinstall the buf compiler

Arnar Pall created SPARK-44141: -- Summary: Remove need to preinstall the buf compiler Key: SPARK-44141 URL: https://issues.apache.org/jira/browse/SPARK-44141 Project: Spark Issue Type:

[jira] [Updated] (SPARK-44140) Support positional parameters in PySpark sql()

[ https://issues.apache.org/jira/browse/SPARK-44140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-44140: - Component/s: PySpark (was: SQL) > Support positional parameters in PySpark sql() >

[jira] [Created] (SPARK-44140) Support positional parameters in PySpark sql()

Max Gekk created SPARK-44140: Summary: Support positional parameters in PySpark sql() Key: SPARK-44140 URL: https://issues.apache.org/jira/browse/SPARK-44140 Project: Spark Issue Type: New

[jira] [Resolved] (SPARK-44066) Support positional parameters in parameterized query

[ https://issues.apache.org/jira/browse/SPARK-44066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-44066. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41568