[jira] [Updated] (SPARK-50423) Refine the docstring for misc and hash functions

[ https://issues.apache.org/jira/browse/SPARK-50423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-50423: --- Labels: pull-request-available (was: ) > Refine the docstring for misc and hash functions >

[jira] [Created] (SPARK-50423) Refine the docstring for misc and hash functions

Ruifeng Zheng created SPARK-50423: - Summary: Refine the docstring for misc and hash functions Key: SPARK-50423 URL: https://issues.apache.org/jira/browse/SPARK-50423 Project: Spark Issue Type

[jira] [Assigned] (SPARK-50422) Make `Parameterized SQL queries` of `SparkSession.sql` API GA

[ https://issues.apache.org/jira/browse/SPARK-50422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-50422: - Assignee: Dongjoon Hyun > Make `Parameterized SQL queries` of `SparkSession.sql` API GA

[jira] [Updated] (SPARK-50422) Make `Parameterized SQL queries` of `SparkSession.sql` API GA

[ https://issues.apache.org/jira/browse/SPARK-50422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-50422: --- Labels: pull-request-available (was: ) > Make `Parameterized SQL queries` of `SparkSession.

[jira] [Updated] (SPARK-50422) Make `Parameterized SQL queries` of `SparkSession.sql` API GA

[ https://issues.apache.org/jira/browse/SPARK-50422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-50422: -- Summary: Make `Parameterized SQL queries` of `SparkSession.sql` API GA (was: Parameterized SQ

[jira] [Created] (SPARK-50422) Parameterized SQL queries of `SparkSession.sql` APIs GA

Dongjoon Hyun created SPARK-50422: - Summary: Parameterized SQL queries of `SparkSession.sql` APIs GA Key: SPARK-50422 URL: https://issues.apache.org/jira/browse/SPARK-50422 Project: Spark Iss

[jira] [Resolved] (SPARK-50410) Refactor the sql function in SparkSession to Eliminate duplicate code

[ https://issues.apache.org/jira/browse/SPARK-50410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-50410. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 48955 [https://

[jira] [Updated] (SPARK-50310) Add a flag to disable DataFrameQueryContext for PySpark

[ https://issues.apache.org/jira/browse/SPARK-50310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-50310: --- Labels: pull-request-available (was: ) > Add a flag to disable DataFrameQueryContext for Py

[jira] [Resolved] (SPARK-50387) Update comp condition for timer expiry

[ https://issues.apache.org/jira/browse/SPARK-50387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-50387. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 48927 [https://gi

[jira] [Assigned] (SPARK-50017) Support Avro encoding for TransformWithState operator - ValueState

[ https://issues.apache.org/jira/browse/SPARK-50017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-50017: Assignee: Eric Marnadi > Support Avro encoding for TransformWithState operator - ValueSta

[jira] [Resolved] (SPARK-50017) Support Avro encoding for TransformWithState operator - ValueState

[ https://issues.apache.org/jira/browse/SPARK-50017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-50017. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 48401 [https://gi

[jira] [Commented] (SPARK-50421) Executor memory config is incorrect when multiple stage level resource profiles works

[ https://issues.apache.org/jira/browse/SPARK-50421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17901075#comment-17901075 ] Terry Wang commented on SPARK-50421: maybe fixed by https://github.com/apache/spark/

[jira] [Created] (SPARK-50420) Executor memory config is incorrect when multiple stage level resource profiles works

Terry Wang created SPARK-50420: -- Summary: Executor memory config is incorrect when multiple stage level resource profiles works Key: SPARK-50420 URL: https://issues.apache.org/jira/browse/SPARK-50420 Pro

[jira] [Updated] (SPARK-50421) Executor memory config is incorrect when multiple stage level resource profiles works

[ https://issues.apache.org/jira/browse/SPARK-50421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-50421: --- Labels: pull-request-available (was: ) > Executor memory config is incorrect when multiple

[jira] [Created] (SPARK-50421) Executor memory config is incorrect when multiple stage level resource profiles works

Terry Wang created SPARK-50421: -- Summary: Executor memory config is incorrect when multiple stage level resource profiles works Key: SPARK-50421 URL: https://issues.apache.org/jira/browse/SPARK-50421 Pro

[jira] [Created] (SPARK-50419) Executor memory config is incorrect when multiple stage level resource profiles works

Terry Wang created SPARK-50419: -- Summary: Executor memory config is incorrect when multiple stage level resource profiles works Key: SPARK-50419 URL: https://issues.apache.org/jira/browse/SPARK-50419 Pro

[jira] [Commented] (SPARK-50191) Flexible configuration of javaoptions.

[ https://issues.apache.org/jira/browse/SPARK-50191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17901065#comment-17901065 ] bingzuochen commented on SPARK-50191: - Can risk be guarded at Spark? If Spark does t

[jira] [Commented] (SPARK-50271) Spark App executes unintended commands

[ https://issues.apache.org/jira/browse/SPARK-50271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17901067#comment-17901067 ] XianXianwang commented on SPARK-50271: -- Spark submits it to Yarn for execution, in

[jira] [Updated] (SPARK-50410) Refactor the sql function in SparkSession to Eliminate duplicate code

[ https://issues.apache.org/jira/browse/SPARK-50410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-50410: --- Labels: pull-request-available (was: ) > Refactor the sql function in SparkSession to Elimi

[jira] [Updated] (SPARK-50407) accessing a MySQL database through a proxy that has a set timeout, an error occurs when using df.write.jdbc

[ https://issues.apache.org/jira/browse/SPARK-50407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chenzichu updated SPARK-50407: -- Description: When accessing a MySQL database through a proxy that has a set timeout, an error occurs

[jira] [Updated] (SPARK-50407) accessing a MySQL database through a proxy that has a set timeout, an error occurs when using df.write.jdbc

[ https://issues.apache.org/jira/browse/SPARK-50407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chenzichu updated SPARK-50407: -- Attachment: 图片1.png > accessing a MySQL database through a proxy that has a set timeout, an error > o

[jira] [Commented] (SPARK-49554) Inconsistent Behavior Between PySpark DataFrame API's `eqNullSafe` Join and SQL Null-Safe Equality `<=>`

[ https://issues.apache.org/jira/browse/SPARK-49554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17901046#comment-17901046 ] Hyukjin Kwon commented on SPARK-49554: -- I can't reproduce this in 3.4.2, 3.5.3 and

[jira] [Resolved] (SPARK-50415) Upgrade `zstd-jni` to 1.5.6-8

[ https://issues.apache.org/jira/browse/SPARK-50415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-50415. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 48959 [https://gi

[jira] [Updated] (SPARK-50418) Allow optional trailing comma at the end of SELECT lists

[ https://issues.apache.org/jira/browse/SPARK-50418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-50418: --- Labels: pull-request-available (was: ) > Allow optional trailing comma at the end of SELECT

[jira] [Created] (SPARK-50418) Allow optional trailing comma at the end of SELECT lists

Daniel created SPARK-50418: -- Summary: Allow optional trailing comma at the end of SELECT lists Key: SPARK-50418 URL: https://issues.apache.org/jira/browse/SPARK-50418 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-50417) Limit number of subdirectories that FallbackStorage creates per shuffle

[ https://issues.apache.org/jira/browse/SPARK-50417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-50417: --- Labels: pull-request-available (was: ) > Limit number of subdirectories that FallbackStorag

[jira] [Updated] (SPARK-50417) Limit number of subdirectories that FallbackStorage creates per shuffle

[ https://issues.apache.org/jira/browse/SPARK-50417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Enrico Minack updated SPARK-50417: -- Component/s: Spark Core (was: SQL) > Limit number of subdirectories that

[jira] [Created] (SPARK-50417) Limit number of subdirectories that FallbackStorage creates per shuffle

Enrico Minack created SPARK-50417: - Summary: Limit number of subdirectories that FallbackStorage creates per shuffle Key: SPARK-50417 URL: https://issues.apache.org/jira/browse/SPARK-50417 Project: Sp

[jira] [Created] (SPARK-50416) load-spark-env.sh test for terminal or pipe broken / non-portable

Phil Walker created SPARK-50416: --- Summary: load-spark-env.sh test for terminal or pipe broken / non-portable Key: SPARK-50416 URL: https://issues.apache.org/jira/browse/SPARK-50416 Project: Spark

[jira] [Updated] (SPARK-50415) Upgrade `zstd-jni` to 1.5.6-8

[ https://issues.apache.org/jira/browse/SPARK-50415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-50415: --- Labels: pull-request-available (was: ) > Upgrade `zstd-jni` to 1.5.6-8 > --

[jira] [Updated] (SPARK-50415) Upgrade `zstd-jni` to 1.5.6-8

[ https://issues.apache.org/jira/browse/SPARK-50415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-50415: -- Summary: Upgrade `zstd-jni` to 1.5.6-8 (was: Upgrade `zest-jni` to 1.5.6-8) > Upgrade `zstd-j

[jira] [Created] (SPARK-50415) Upgrade `zest-jni` to 1.5.6-8

Dongjoon Hyun created SPARK-50415: - Summary: Upgrade `zest-jni` to 1.5.6-8 Key: SPARK-50415 URL: https://issues.apache.org/jira/browse/SPARK-50415 Project: Spark Issue Type: Sub-task

[jira] [Assigned] (SPARK-50412) Rename `ojdbc11.version` to `ojdbc17.version`

[ https://issues.apache.org/jira/browse/SPARK-50412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-50412: - Assignee: Cheng Pan > Rename `ojdbc11.version` to `ojdbc17.version` > -

[jira] [Updated] (SPARK-50412) Rename `ojdbc11.version` to `ojdbc17.version`

[ https://issues.apache.org/jira/browse/SPARK-50412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-50412: -- Parent: SPARK-44111 Issue Type: Sub-task (was: Improvement) > Rename `ojdbc11.version

[jira] [Updated] (SPARK-50383) Support Virtual Threads in REST Submission API

[ https://issues.apache.org/jira/browse/SPARK-50383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-50383: -- Parent Issue: SPARK-45869 (was: SPARK-44111) > Support Virtual Threads in REST Submission API

[jira] [Resolved] (SPARK-50412) Rename `ojdbc11.version` to `ojdbc17.version`

[ https://issues.apache.org/jira/browse/SPARK-50412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-50412. --- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 48957 [https://

[jira] [Comment Edited] (SPARK-48344) [M1] SQL Script execution

[ https://issues.apache.org/jira/browse/SPARK-48344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17900965#comment-17900965 ] Milan Dankovic edited comment on SPARK-48344 at 11/25/24 4:55 PM:

[jira] [Commented] (SPARK-48344) [M1] SQL Script execution

[ https://issues.apache.org/jira/browse/SPARK-48344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17900965#comment-17900965 ] Milan Dankovic commented on SPARK-48344: [~cloud_fan] can we reopen this until t

[jira] [Updated] (SPARK-50414) Incorrect statement that causes java.lang.NoClassDefFoundError in Spark Connect documents

[ https://issues.apache.org/jira/browse/SPARK-50414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akihiro Okuno updated SPARK-50414: -- Description: h2. Issue Description I created a small Scala application to verify Spark Connec

[jira] [Created] (SPARK-50414) Incorrect statement that causes java.lang.NoClassDefFoundError in Spark Connect documents

Akihiro Okuno created SPARK-50414: - Summary: Incorrect statement that causes java.lang.NoClassDefFoundError in Spark Connect documents Key: SPARK-50414 URL: https://issues.apache.org/jira/browse/SPARK-50414

[jira] [Commented] (SPARK-50242) (RCE) Command Execution Vulnerability in Spark

[ https://issues.apache.org/jira/browse/SPARK-50242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17900947#comment-17900947 ] Soumasish Goswami commented on SPARK-50242: --- Duplicate of [SPARK-50240|https:/

[jira] [Created] (SPARK-50413) Add a flag to insert commands to know if they are part of CTAS

Stefan Kandic created SPARK-50413: - Summary: Add a flag to insert commands to know if they are part of CTAS Key: SPARK-50413 URL: https://issues.apache.org/jira/browse/SPARK-50413 Project: Spark

[jira] [Resolved] (SPARK-50287) Merge options of table and relation when create WriteBuilder in FileTable

[ https://issues.apache.org/jira/browse/SPARK-50287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-50287. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 48821 [https://gith

[jira] [Updated] (SPARK-50411) Spark Connect the default API in Spark 4

[ https://issues.apache.org/jira/browse/SPARK-50411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hövell updated SPARK-50411: -- Summary: Spark Connect the default API in Spark 4 (was: We to make Spark Connect the

[jira] [Updated] (SPARK-50411) Spark Connect as the default API in Spark 4

[ https://issues.apache.org/jira/browse/SPARK-50411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hövell updated SPARK-50411: -- Summary: Spark Connect as the default API in Spark 4 (was: Spark Connect the default

[jira] [Resolved] (SPARK-50286) Correctly propogate SQL options to WriteBuilder

[ https://issues.apache.org/jira/browse/SPARK-50286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-50286. - Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 48822 [https://gith

[jira] [Assigned] (SPARK-50286) Correctly propogate SQL options to WriteBuilder

[ https://issues.apache.org/jira/browse/SPARK-50286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-50286: --- Assignee: Cheng Pan > Correctly propogate SQL options to WriteBuilder > ---

[jira] [Updated] (SPARK-50412) Rename `ojdbc11.version` to `ojdbc17.version`

[ https://issues.apache.org/jira/browse/SPARK-50412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-50412: --- Labels: pull-request-available (was: ) > Rename `ojdbc11.version` to `ojdbc17.version` > --

[jira] [Created] (SPARK-50412) Rename `ojdbc11.version` to `ojdbc17.version`

Cheng Pan created SPARK-50412: - Summary: Rename `ojdbc11.version` to `ojdbc17.version` Key: SPARK-50412 URL: https://issues.apache.org/jira/browse/SPARK-50412 Project: Spark Issue Type: Improveme

[jira] [Created] (SPARK-50411) We to make Spark Connect the default API in Spark 4

Herman van Hövell created SPARK-50411: - Summary: We to make Spark Connect the default API in Spark 4 Key: SPARK-50411 URL: https://issues.apache.org/jira/browse/SPARK-50411 Project: Spark

[jira] [Commented] (SPARK-49875) error build spark 3.5.3

[ https://issues.apache.org/jira/browse/SPARK-49875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17900891#comment-17900891 ] Michael Schulte commented on SPARK-49875: - I am currently facing similar issues

[jira] [Created] (SPARK-50410) Refactor the sql function in SparkSession to Eliminate duplicate code

Yang Jie created SPARK-50410: Summary: Refactor the sql function in SparkSession to Eliminate duplicate code Key: SPARK-50410 URL: https://issues.apache.org/jira/browse/SPARK-50410 Project: Spark

[jira] [Assigned] (SPARK-50408) Refine the docstring for datetime functions - part 4

[ https://issues.apache.org/jira/browse/SPARK-50408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-50408: Assignee: Ruifeng Zheng > Refine the docstring for datetime functions - part 4 >

[jira] [Resolved] (SPARK-50408) Refine the docstring for datetime functions - part 4

[ https://issues.apache.org/jira/browse/SPARK-50408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-50408. -- Fix Version/s: 4.0.0 Resolution: Fixed Issue resolved by pull request 48953 [https://gi

[jira] [Updated] (SPARK-50408) Refine the docstring for datetime functions - part 4

[ https://issues.apache.org/jira/browse/SPARK-50408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated SPARK-50408: --- Labels: pull-request-available (was: ) > Refine the docstring for datetime functions - part

[jira] [Created] (SPARK-50408) Refine the docstring for datetime functions - part 4

Ruifeng Zheng created SPARK-50408: - Summary: Refine the docstring for datetime functions - part 4 Key: SPARK-50408 URL: https://issues.apache.org/jira/browse/SPARK-50408 Project: Spark Issue

[jira] [Created] (SPARK-50407) accessing a MySQL database through a proxy that has a set timeout, an error occurs when using df.write.jdbc

chenzichu created SPARK-50407: - Summary: accessing a MySQL database through a proxy that has a set timeout, an error occurs when using df.write.jdbc Key: SPARK-50407 URL: https://issues.apache.org/jira/browse/SPARK-50