[jira] [Created] (SPARK-44979) Cache results of simple udfs on executors if same arguments are passed.

Dinesh Dharme created SPARK-44979: - Summary: Cache results of simple udfs on executors if same arguments are passed. Key: SPARK-44979 URL: https://issues.apache.org/jira/browse/SPARK-44979 Project: Sp

[jira] [Updated] (SPARK-44979) Cache results of simple udfs on executors if same arguments are passed.

[ https://issues.apache.org/jira/browse/SPARK-44979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dinesh Dharme updated SPARK-44979: -- Description: Consider two dataframes : {{keyword_given = [ ["green pstr",], ["greenpstr",], [

[jira] [Updated] (SPARK-44264) DeepSpeed Distributor

[ https://issues.apache.org/jira/browse/SPARK-44264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-44264: Summary: DeepSpeed Distributor (was: DeepSpeed Distrobutor) > DeepSpeed Distributor > ---

[jira] [Assigned] (SPARK-44978) Fix SQLQueryTestSuite unable create table normally

[ https://issues.apache.org/jira/browse/SPARK-44978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-44978: Assignee: Jia Fan > Fix SQLQueryTestSuite unable create table normally >

[jira] [Resolved] (SPARK-44978) Fix SQLQueryTestSuite unable create table normally

[ https://issues.apache.org/jira/browse/SPARK-44978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-44978. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42691 [https://gi

[jira] [Updated] (SPARK-44131) Add call_function for Scala API

[ https://issues.apache.org/jira/browse/SPARK-44131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-44131: Summary: Add call_function for Scala API (was: Add call_function and deprecate call_udf for Scala API) >

[jira] [Commented] (SPARK-44131) Add call_function for Scala API

[ https://issues.apache.org/jira/browse/SPARK-44131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17759408#comment-17759408 ] Xiao Li commented on SPARK-44131: - [https://github.com/apache/spark/pull/41950] reverted

[jira] [Created] (SPARK-44980) createDataFrame should respect the names namedtuples properly

Hyukjin Kwon created SPARK-44980: Summary: createDataFrame should respect the names namedtuples properly Key: SPARK-44980 URL: https://issues.apache.org/jira/browse/SPARK-44980 Project: Spark

[jira] [Updated] (SPARK-44980) Fix inherited namedtuples to work in createDataFrame

[ https://issues.apache.org/jira/browse/SPARK-44980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-44980: - Summary: Fix inherited namedtuples to work in createDataFrame (was: createDataFrame should resp

[jira] [Updated] (SPARK-44980) createDataFrame should respect the names namedtuples properly

[ https://issues.apache.org/jira/browse/SPARK-44980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-44980: - Description: {code} from collections import namedtuple MyTuple = namedtuple("MyTuple", ["zz", "b

[jira] [Updated] (SPARK-44981) Filter out static configurations used in local mode

[ https://issues.apache.org/jira/browse/SPARK-44981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-44981: - Priority: Minor (was: Major) > Filter out static configurations used in local mode > --

[jira] [Created] (SPARK-44981) Filter out static configurations used in local mode

Hyukjin Kwon created SPARK-44981: Summary: Filter out static configurations used in local mode Key: SPARK-44981 URL: https://issues.apache.org/jira/browse/SPARK-44981 Project: Spark Issue Typ

[jira] [Created] (SPARK-44982) Mark Spark Connect configurations as static configuration

Hyukjin Kwon created SPARK-44982: Summary: Mark Spark Connect configurations as static configuration Key: SPARK-44982 URL: https://issues.apache.org/jira/browse/SPARK-44982 Project: Spark Iss

[jira] [Resolved] (SPARK-44963) Make PySpark (pyspark-ml module) tests passing without any optional dependency

[ https://issues.apache.org/jira/browse/SPARK-44963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-44963. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42678 [https://gi

[jira] [Assigned] (SPARK-44963) Make PySpark (pyspark-ml module) tests passing without any optional dependency

[ https://issues.apache.org/jira/browse/SPARK-44963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-44963: Assignee: Hyukjin Kwon > Make PySpark (pyspark-ml module) tests passing without any optio

[jira] [Assigned] (SPARK-44897) Local Property Propagation to Subquery Broadcast Exec

[ https://issues.apache.org/jira/browse/SPARK-44897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-44897: --- Assignee: Michael Chen > Local Property Propagation to Subquery Broadcast Exec > --

[jira] [Resolved] (SPARK-44897) Local Property Propagation to Subquery Broadcast Exec

[ https://issues.apache.org/jira/browse/SPARK-44897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-44897. - Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 42587 [https://gith

[jira] [Assigned] (SPARK-44945) Automate PySpark error class documentation

[ https://issues.apache.org/jira/browse/SPARK-44945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-44945: Assignee: Haejoon Lee > Automate PySpark error class documentation >

[jira] [Resolved] (SPARK-44945) Automate PySpark error class documentation

[ https://issues.apache.org/jira/browse/SPARK-44945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-44945. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42658 [https://gi

[jira] [Resolved] (SPARK-44091) Yarn module test failed on MacOs/Apple Slicon

[ https://issues.apache.org/jira/browse/SPARK-44091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-44091. --- Fix Version/s: 3.5.0 4.0.0 Resolution: Fixed Issue resolved by pul

[jira] [Assigned] (SPARK-44091) Yarn module test failed on MacOs/Apple Slicon

[ https://issues.apache.org/jira/browse/SPARK-44091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-44091: - Assignee: Yang Jie > Yarn module test failed on MacOs/Apple Slicon > --

[jira] [Updated] (SPARK-44091) Introduce withResourceTypes to `ResourceRequestTestHelper` to restore `resourceTypes` as default value after testing

[ https://issues.apache.org/jira/browse/SPARK-44091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-44091: -- Summary: Introduce withResourceTypes to `ResourceRequestTestHelper` to restore `resourceTypes`

[jira] [Resolved] (SPARK-44960) Unescape and consist error summary across UI pages

[ https://issues.apache.org/jira/browse/SPARK-44960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-44960. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 42674 [https://github.com

[jira] [Assigned] (SPARK-44960) Unescape and consist error summary across UI pages

[ https://issues.apache.org/jira/browse/SPARK-44960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao reassigned SPARK-44960: Assignee: Kent Yao > Unescape and consist error summary across UI pages > ---

[jira] [Commented] (SPARK-44862) Adding metric tracking number of jobs running on a cluster

[ https://issues.apache.org/jira/browse/SPARK-44862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17759459#comment-17759459 ] Kent Yao commented on SPARK-44862: -- There is a gauge named activeJobs in the DAGSchedul

[jira] [Commented] (SPARK-44819) Make Python the first language in all Spark code snippet

[ https://issues.apache.org/jira/browse/SPARK-44819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17759468#comment-17759468 ] BingKun Pan commented on SPARK-44819: - I work on it. > Make Python the first langua

[jira] [Resolved] (SPARK-44980) Fix inherited namedtuples to work in createDataFrame

[ https://issues.apache.org/jira/browse/SPARK-44980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-44980. -- Fix Version/s: 3.5.0 4.0.0 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-44980) Fix inherited namedtuples to work in createDataFrame

[ https://issues.apache.org/jira/browse/SPARK-44980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-44980: Assignee: Hyukjin Kwon > Fix inherited namedtuples to work in createDataFrame > -