[jira] [Updated] (SPARK-49700) Switch interface and implementations

[ https://issues.apache.org/jira/browse/SPARK-49700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-49700: --- Labels: pull-request-available (was: ) > Switch interface and implementations > ---

[jira] [Updated] (SPARK-50276) Add JSON path expression using the COLON operator

[ https://issues.apache.org/jira/browse/SPARK-50276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chenghao Lyu updated SPARK-50276: - Description: Support JSON path expression by adding COLON sign operator [1] and JSON path expre

[jira] [Updated] (SPARK-50276) Add JSON path expression using the COLON operator

[ https://issues.apache.org/jira/browse/SPARK-50276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chenghao Lyu updated SPARK-50276: - Description: Support JSON path expression by adding COLON sign operator [1] and JSON path expre

[jira] [Created] (SPARK-50284) Create easy API in Python to construct Variants

Harsh Motwani created SPARK-50284: - Summary: Create easy API in Python to construct Variants Key: SPARK-50284 URL: https://issues.apache.org/jira/browse/SPARK-50284 Project: Spark Issue Type:

[jira] [Updated] (SPARK-50276) Add JSON path expression using the COLON operator

[ https://issues.apache.org/jira/browse/SPARK-50276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chenghao Lyu updated SPARK-50276: - Description: Support JSON path expression by adding COLON sign operator [1] and JSON path expre

[jira] [Updated] (SPARK-50276) Add JSON path expression using the COLON operator

[ https://issues.apache.org/jira/browse/SPARK-50276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chenghao Lyu updated SPARK-50276: - Description: Support JSON path expression by adding COLON sign operator [1] and JSON path expre

[jira] [Created] (SPARK-50285) The staged table interface does not return metrics

Jan-Ole Sasse created SPARK-50285: - Summary: The staged table interface does not return metrics Key: SPARK-50285 URL: https://issues.apache.org/jira/browse/SPARK-50285 Project: Spark Issue Ty

[jira] [Resolved] (SPARK-50118) Spark removes working directory while Python UDF runs

[ https://issues.apache.org/jira/browse/SPARK-50118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-50118. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 48665 [https://gi

[jira] [Assigned] (SPARK-50118) Spark removes working directory while Python UDF runs

[ https://issues.apache.org/jira/browse/SPARK-50118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-50118: Assignee: Hyukjin Kwon > Spark removes working directory while Python UDF runs >

[jira] [Resolved] (SPARK-50281) pyspark local session `spark.jars` configuration does not work

[ https://issues.apache.org/jira/browse/SPARK-50281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-50281. -- Fix Version/s: 4.0.0 Assignee: Hyukjin Kwon Resolution: Fixed should be fixed

[jira] [Assigned] (SPARK-50152) Support handleInitialState with state data source reader

[ https://issues.apache.org/jira/browse/SPARK-50152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-50152: Assignee: Jing Zhan > Support handleInitialState with state data source reader >

[jira] [Resolved] (SPARK-50152) Support handleInitialState with state data source reader

[ https://issues.apache.org/jira/browse/SPARK-50152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-50152. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 48686 [https://gi

[jira] [Assigned] (SPARK-50282) Simplify TargetEncoderModel.transform

[ https://issues.apache.org/jira/browse/SPARK-50282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-50282: - Assignee: Ruifeng Zheng > Simplify TargetEncoderModel.transform > -

[jira] [Resolved] (SPARK-50282) Simplify TargetEncoderModel.transform

[ https://issues.apache.org/jira/browse/SPARK-50282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-50282. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 48816 [https://

[jira] [Resolved] (SPARK-50280) Refactor result sorting and empty bin filling in `compute_hist`

[ https://issues.apache.org/jira/browse/SPARK-50280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-50280. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 48814 [https://

[jira] [Updated] (SPARK-50278) Upgrade `netty-tcnative` to 2.0.69.Final

[ https://issues.apache.org/jira/browse/SPARK-50278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-50278: -- Parent: SPARK-47046 Issue Type: Sub-task (was: Improvement) > Upgrade `netty-tcnative

[jira] [Resolved] (SPARK-50278) Upgrade `netty-tcnative` to 2.0.69.Final

[ https://issues.apache.org/jira/browse/SPARK-50278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-50278. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 48810 [https://

[jira] [Assigned] (SPARK-50279) Add DataFrame.plot to PySpark docs

[ https://issues.apache.org/jira/browse/SPARK-50279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-50279: - Assignee: Xinrong Meng > Add DataFrame.plot to PySpark docs > -

[jira] [Resolved] (SPARK-50279) Add DataFrame.plot to PySpark docs

[ https://issues.apache.org/jira/browse/SPARK-50279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-50279. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 48813 [https://

[jira] [Updated] (SPARK-50242) (RCE) Command Execution Vulnerability in Spark

[ https://issues.apache.org/jira/browse/SPARK-50242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lu jiadong updated SPARK-50242: --- Description: Spark allow user to deploy jar packet with`extraJavaOptions` , but spark has incomplet

[jira] [Updated] (SPARK-50242) (RCE) Command Execution Vulnerability in Spark

[ https://issues.apache.org/jira/browse/SPARK-50242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lu jiadong updated SPARK-50242: --- Description: Spark allow user to deploy jar packet iwith`extraJavaOptions` , but spark has incomple

[jira] [Updated] (SPARK-50239) JavaOptions Injection Issue

[ https://issues.apache.org/jira/browse/SPARK-50239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zijie updated SPARK-50239: -- Description: First, we need to obtain regular user privileges and construct a JAR file according to custom ru

[jira] [Created] (SPARK-50283) Reuse `docs` image for linter

Ruifeng Zheng created SPARK-50283: - Summary: Reuse `docs` image for linter Key: SPARK-50283 URL: https://issues.apache.org/jira/browse/SPARK-50283 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-50283) Reuse `docs` image for linter

[ https://issues.apache.org/jira/browse/SPARK-50283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-50283: --- Labels: pull-request-available (was: ) > Reuse `docs` image for linter > --

[jira] [Updated] (SPARK-50281) pyspark local session `spark.jars` configuration does not work

[ https://issues.apache.org/jira/browse/SPARK-50281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weichen Xu updated SPARK-50281: --- Description: pyspark local session `spark.jars` configuration does not work    Reproducing code:

[jira] [Created] (SPARK-50281) pyspark local session `spark.jars` configuration does not work

Weichen Xu created SPARK-50281: -- Summary: pyspark local session `spark.jars` configuration does not work Key: SPARK-50281 URL: https://issues.apache.org/jira/browse/SPARK-50281 Project: Spark

[jira] [Updated] (SPARK-50281) pyspark local session `spark.jars` configuration does not work

[ https://issues.apache.org/jira/browse/SPARK-50281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weichen Xu updated SPARK-50281: --- Description: pyspark local session `spark.jars` configuration does not work    Reproducing code:

[jira] [Resolved] (SPARK-50262) Forbid change complex types during altering collation

[ https://issues.apache.org/jira/browse/SPARK-50262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-50262. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 48803 [https://gith

[jira] [Assigned] (SPARK-50262) Forbid change complex types during altering collation

[ https://issues.apache.org/jira/browse/SPARK-50262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-50262: --- Assignee: Aleksei Shishkin > Forbid change complex types during altering collation > --

[jira] [Created] (SPARK-50282) Simplify TargetEncoderModel.transform

Ruifeng Zheng created SPARK-50282: - Summary: Simplify TargetEncoderModel.transform Key: SPARK-50282 URL: https://issues.apache.org/jira/browse/SPARK-50282 Project: Spark Issue Type: Improveme

[jira] [Updated] (SPARK-50282) Simplify TargetEncoderModel.transform

[ https://issues.apache.org/jira/browse/SPARK-50282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-50282: --- Labels: pull-request-available (was: ) > Simplify TargetEncoderModel.transform > --

[jira] [Updated] (SPARK-50056) Codegen Support for ParseUrl (by Invoke & RuntimeReplaceable)

[ https://issues.apache.org/jira/browse/SPARK-50056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-50056: -- Parent: SPARK-50266 Issue Type: Sub-task (was: Improvement) > Codegen Support for Par

[jira] [Updated] (SPARK-50066) Codegen Support for `SchemaOfXml` (by Invoke & RuntimeReplaceable)

[ https://issues.apache.org/jira/browse/SPARK-50066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-50066: -- Parent: SPARK-50266 Issue Type: Sub-task (was: Improvement) > Codegen Support for `Sc

[jira] [Updated] (SPARK-50081) Codegen Support for `XPath*`(by Invoke & RuntimeReplaceable)

[ https://issues.apache.org/jira/browse/SPARK-50081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-50081: -- Parent: SPARK-50266 Issue Type: Sub-task (was: Improvement) > Codegen Support for `XP

[jira] [Updated] (SPARK-50288) Executors of stage failed still alive even through stage has been retried

[ https://issues.apache.org/jira/browse/SPARK-50288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu-Ting LIN updated SPARK-50288: Summary: Executors of stage failed still alive even through stage has been retried (was: Executor

[jira] [Created] (SPARK-50288) Executors of stage failed still alive even that stage has been retried

Yu-Ting LIN created SPARK-50288: --- Summary: Executors of stage failed still alive even that stage has been retried Key: SPARK-50288 URL: https://issues.apache.org/jira/browse/SPARK-50288 Project: Spark

[jira] [Updated] (SPARK-50288) Executors of stage failed still alive even that stage has been retried

[ https://issues.apache.org/jira/browse/SPARK-50288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yu-Ting LIN updated SPARK-50288: Attachment: trigger_duplicate_process.png > Executors of stage failed still alive even that stage

[jira] [Created] (SPARK-50289) Add a detailed upgrade to broadcast which table timed out

guihuawen created SPARK-50289: - Summary: Add a detailed upgrade to broadcast which table timed out Key: SPARK-50289 URL: https://issues.apache.org/jira/browse/SPARK-50289 Project: Spark Issue Typ

[jira] [Updated] (SPARK-50240) Command Execution Vulnerability in Spark (JavaOptions)

[ https://issues.apache.org/jira/browse/SPARK-50240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaotong updated SPARK-50240: - Description: We found that Spark allow user to upload jar packet with`extraJavaOptions` ,   When insta

[jira] [Assigned] (SPARK-50260) Refactor thread states from int to a case class in ExecuteThreadRunner

[ https://issues.apache.org/jira/browse/SPARK-50260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-50260: Assignee: Changgyoo Park > Refactor thread states from int to a case class in ExecuteThre

[jira] [Resolved] (SPARK-50260) Refactor thread states from int to a case class in ExecuteThreadRunner

[ https://issues.apache.org/jira/browse/SPARK-50260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-50260. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 48792 [https://gi

[jira] [Created] (SPARK-50286) Correctly propogate SQL options to WriteBuilder

Cheng Pan created SPARK-50286: - Summary: Correctly propogate SQL options to WriteBuilder Key: SPARK-50286 URL: https://issues.apache.org/jira/browse/SPARK-50286 Project: Spark Issue Type: Improve

[jira] [Resolved] (SPARK-50130) Add scalar and exists DataFrame APIs

[ https://issues.apache.org/jira/browse/SPARK-50130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-50130. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 48664 [https://gi

[jira] [Updated] (SPARK-50239) JavaOptions Injection Issue

[ https://issues.apache.org/jira/browse/SPARK-50239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zijie updated SPARK-50239: -- Description: First, we need to obtain regular user privileges and construct a JAR file according to custom ru

[jira] [Updated] (SPARK-50287) Merge options of table and relation when create WriteBuilder in FileTable

[ https://issues.apache.org/jira/browse/SPARK-50287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-50287: --- Labels: pull-request-available (was: ) > Merge options of table and relation when create Wr

[jira] [Created] (SPARK-50287) Merge options of table and relation when create WriteBuilder in FileTable

Cheng Pan created SPARK-50287: - Summary: Merge options of table and relation when create WriteBuilder in FileTable Key: SPARK-50287 URL: https://issues.apache.org/jira/browse/SPARK-50287 Project: Spark

[jira] [Updated] (SPARK-50240) Command Execution Vulnerability in Spark (JavaOptions)

[ https://issues.apache.org/jira/browse/SPARK-50240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaotong updated SPARK-50240: - Affects Version/s: 3.5.1 (was: 3.5.3) Description: We found that Sp

[jira] [Updated] (SPARK-50286) Correctly propogate SQL options to WriteBuilder

[ https://issues.apache.org/jira/browse/SPARK-50286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-50286: --- Labels: pull-request-available (was: ) > Correctly propogate SQL options to WriteBuilder >