[jira] [Commented] (SPARK-43956) Fix the bug doesn't display column's sql for Percentile[Cont|Disc]

[ https://issues.apache.org/jira/browse/SPARK-43956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17728926#comment-17728926 ] jiaan.geng commented on SPARK-43956: resolved by https://github.com/apache/spark/pul

[jira] [Resolved] (SPARK-43956) Fix the bug doesn't display column's sql for Percentile[Cont|Disc]

[ https://issues.apache.org/jira/browse/SPARK-43956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-43956. - Assignee: jiaan.geng Resolution: Fixed > Fix the bug doesn't display column's sql for Perc

[jira] [Created] (SPARK-43956) Fix the bug doesn't display column's sql for Percentile[Cont|Disc]

jiaan.geng created SPARK-43956: -- Summary: Fix the bug doesn't display column's sql for Percentile[Cont|Disc] Key: SPARK-43956 URL: https://issues.apache.org/jira/browse/SPARK-43956 Project: Spark

[jira] [Updated] (SPARK-43954) Upgrade sbt from 1.8.3 to 1.9.0

[ https://issues.apache.org/jira/browse/SPARK-43954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BingKun Pan updated SPARK-43954: Description: Release notes: [https://github.com/sbt/sbt/releases/tag/v1.9.0] > Upgrade sbt from 1.

[jira] [Updated] (SPARK-43955) Upgrade `scalafmt` from 3.7.3 to 3.7.4

[ https://issues.apache.org/jira/browse/SPARK-43955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BingKun Pan updated SPARK-43955: Description: Release notes: https://github.com/scalameta/scalafmt/releases/tag/v3.7.4 > Upgrade `

[jira] [Created] (SPARK-43955) Upgrade `scalafmt` from 3.7.3 to 3.7.4

BingKun Pan created SPARK-43955: --- Summary: Upgrade `scalafmt` from 3.7.3 to 3.7.4 Key: SPARK-43955 URL: https://issues.apache.org/jira/browse/SPARK-43955 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-43954) Upgrade sbt from 1.8.3 to 1.9.0

[ https://issues.apache.org/jira/browse/SPARK-43954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BingKun Pan updated SPARK-43954: Summary: Upgrade sbt from 1.8.3 to 1.9.0 (was: Upgrade sbt from 1.8.2 to 1.9.0) > Upgrade sbt fr

[jira] [Created] (SPARK-43954) Upgrade sbt from 1.8.2 to 1.9.0

BingKun Pan created SPARK-43954: --- Summary: Upgrade sbt from 1.8.2 to 1.9.0 Key: SPARK-43954 URL: https://issues.apache.org/jira/browse/SPARK-43954 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-43380) Fix Avro data type conversion issues to avoid producing incorrect results

[ https://issues.apache.org/jira/browse/SPARK-43380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-43380. Fix Version/s: 3.5.0 (was: 3.4.0) Resolution: Fixed Issue re

[jira] [Resolved] (SPARK-36612) Support left outer join build left or right outer join build right in shuffled hash join

[ https://issues.apache.org/jira/browse/SPARK-36612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun resolved SPARK-36612. -- Fix Version/s: 3.5.0 Resolution: Fixed > Support left outer join build left or right outer join

[jira] [Assigned] (SPARK-36612) Support left outer join build left or right outer join build right in shuffled hash join

[ https://issues.apache.org/jira/browse/SPARK-36612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned SPARK-36612: Assignee: Szehon Ho > Support left outer join build left or right outer join build right in > sh

[jira] [Created] (SPARK-43953) Remove pass

Bjørn Jørgensen created SPARK-43953: --- Summary: Remove pass Key: SPARK-43953 URL: https://issues.apache.org/jira/browse/SPARK-43953 Project: Spark Issue Type: Improvement Component

[jira] [Resolved] (SPARK-43904) Upgrade jackson to 2.15.2

[ https://issues.apache.org/jira/browse/SPARK-43904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-43904. --- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41414 [https://

[jira] [Assigned] (SPARK-43950) Upgrade kubernetes-client to 6.7.0

[ https://issues.apache.org/jira/browse/SPARK-43950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-43950: - Assignee: Bjørn Jørgensen > Upgrade kubernetes-client to 6.7.0 > --

[jira] [Resolved] (SPARK-43950) Upgrade kubernetes-client to 6.7.0

[ https://issues.apache.org/jira/browse/SPARK-43950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-43950. --- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41434 [https://

[jira] [Assigned] (SPARK-43904) Upgrade jackson to 2.15.2

[ https://issues.apache.org/jira/browse/SPARK-43904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-43904: - Assignee: BingKun Pan > Upgrade jackson to 2.15.2 > - > >

[jira] [Assigned] (SPARK-43945) Fix bug for `SQLQueryTestSuite` when run on local env

[ https://issues.apache.org/jira/browse/SPARK-43945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-43945: Assignee: BingKun Pan > Fix bug for `SQLQueryTestSuite` when run on local env > -

[jira] [Resolved] (SPARK-43945) Fix bug for `SQLQueryTestSuite` when run on local env

[ https://issues.apache.org/jira/browse/SPARK-43945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-43945. -- Fix Version/s: 3.5.0 Resolution: Fixed Issue resolved by pull request 41431 [https://github.com

[jira] [Updated] (SPARK-43922) Add named argument support in parser for function call

[ https://issues.apache.org/jira/browse/SPARK-43922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Richard Yu updated SPARK-43922: --- Summary: Add named argument support in parser for function call (was: Add named parameter support i

[jira] [Updated] (SPARK-43922) Add named argument support in parser for function call

[ https://issues.apache.org/jira/browse/SPARK-43922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Richard Yu updated SPARK-43922: --- Description: Today, we are implementing named argument support for user defined functions, some bui

[jira] [Commented] (SPARK-43952) Cancel Spark jobs not only by a single "jobgroup", but allow multiple "job tags"

[ https://issues.apache.org/jira/browse/SPARK-43952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17728791#comment-17728791 ] Juliusz Sompolski commented on SPARK-43952: --- Indirectly related with https://i

[jira] [Commented] (SPARK-43754) Spark Connect Session & Query lifecycle

[ https://issues.apache.org/jira/browse/SPARK-43754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17728790#comment-17728790 ] Juliusz Sompolski commented on SPARK-43754: --- Indirectly related to https://iss

[jira] [Created] (SPARK-43952) Cancel Spark jobs not only by a single "jobgroup", but allow multiple "job tags"

Juliusz Sompolski created SPARK-43952: - Summary: Cancel Spark jobs not only by a single "jobgroup", but allow multiple "job tags" Key: SPARK-43952 URL: https://issues.apache.org/jira/browse/SPARK-43952

[jira] [Comment Edited] (SPARK-43864) Versions of the package net.sourceforge.htmlunit:htmlunit from 0 and before 3.0.0 are vulnerable to Remote Code Execution (RCE) via XSTL

[ https://issues.apache.org/jira/browse/SPARK-43864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17728759#comment-17728759 ] BingKun Pan edited comment on SPARK-43864 at 6/2/23 2:20 PM: -

[jira] [Commented] (SPARK-43864) Versions of the package net.sourceforge.htmlunit:htmlunit from 0 and before 3.0.0 are vulnerable to Remote Code Execution (RCE) via XSTL

[ https://issues.apache.org/jira/browse/SPARK-43864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17728759#comment-17728759 ] BingKun Pan commented on SPARK-43864: - [~gaoyajun02] Okay, Let me investigate it fir

[jira] [Commented] (SPARK-43943) Add math functions to Scala and Python

[ https://issues.apache.org/jira/browse/SPARK-43943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17728738#comment-17728738 ] GridGain Integration commented on SPARK-43943: -- User 'zhengruifeng' has cre

[jira] [Resolved] (SPARK-43951) RocksDB state store can become corrupt on task retries

[ https://issues.apache.org/jira/browse/SPARK-43951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Binford resolved SPARK-43951. -- Resolution: Fixed > RocksDB state store can become corrupt on task retries > -

[jira] [Commented] (SPARK-43951) RocksDB state store can become corrupt on task retries

[ https://issues.apache.org/jira/browse/SPARK-43951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17728736#comment-17728736 ] Adam Binford commented on SPARK-43951: -- Of course as soon as I finish figuring all

[jira] [Created] (SPARK-43951) RocksDB state store can become corrupt on task retries

Adam Binford created SPARK-43951: Summary: RocksDB state store can become corrupt on task retries Key: SPARK-43951 URL: https://issues.apache.org/jira/browse/SPARK-43951 Project: Spark Issue

[jira] [Commented] (SPARK-43864) Versions of the package net.sourceforge.htmlunit:htmlunit from 0 and before 3.0.0 are vulnerable to Remote Code Execution (RCE) via XSTL

[ https://issues.apache.org/jira/browse/SPARK-43864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17728724#comment-17728724 ] gaoyajun02 commented on SPARK-43864: It looks like a series of test package dependen

[jira] [Resolved] (SPARK-43351) Support Golang in Spark Connect

[ https://issues.apache.org/jira/browse/SPARK-43351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-43351. -- Assignee: BoYang Resolution: Fixed Fixed in https://github.com/apache/spark-connect-go/p

[jira] [Created] (SPARK-43950) Upgrade kubernetes-client to 6.7.0

Bjørn Jørgensen created SPARK-43950: --- Summary: Upgrade kubernetes-client to 6.7.0 Key: SPARK-43950 URL: https://issues.apache.org/jira/browse/SPARK-43950 Project: Spark Issue Type: Dependen

[jira] [Resolved] (SPARK-43949) Upgrade Cloudpickle to 2.2.1

[ https://issues.apache.org/jira/browse/SPARK-43949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-43949. -- Fix Version/s: 3.4.1 3.5.0 Assignee: Hyukjin Kwon Resolution

[jira] [Commented] (SPARK-43075) Change gRPC to grpcio when it is not installed.

[ https://issues.apache.org/jira/browse/SPARK-43075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17728681#comment-17728681 ] GridGain Integration commented on SPARK-43075: -- User 'bjornjorgensen' has c

[jira] [Commented] (SPARK-43063) `df.show` handle null should print NULL instead of null

[ https://issues.apache.org/jira/browse/SPARK-43063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17728682#comment-17728682 ] GridGain Integration commented on SPARK-43063: -- User 'dongjoon-hyun' has cr

[jira] [Updated] (SPARK-43916) Add percentile like functions to Scala and Python API

[ https://issues.apache.org/jira/browse/SPARK-43916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-43916: --- Summary: Add percentile like functions to Scala and Python API (was: Add percentile* to Scala and P

[jira] [Updated] (SPARK-43916) Add percentile* to Scala and Python API

[ https://issues.apache.org/jira/browse/SPARK-43916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-43916: --- Summary: Add percentile* to Scala and Python API (was: Add percentile to Scala and Python API) > A

[jira] [Updated] (SPARK-43911) Use toSet to deduplicate the iterator data to prevent the creation of large Array

[ https://issues.apache.org/jira/browse/SPARK-43911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mcdull_zhang updated SPARK-43911: - Summary: Use toSet to deduplicate the iterator data to prevent the creation of large Array (was

[jira] [Commented] (SPARK-43866) Partition filter condition should pushed down to metastore query if it is equivalence Predicate

[ https://issues.apache.org/jira/browse/SPARK-43866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17728643#comment-17728643 ] ASF GitHub Bot commented on SPARK-43866: User 'ming95' has created a pull re

[jira] [Commented] (SPARK-43879) Decouple handle command and send response on server side

[ https://issues.apache.org/jira/browse/SPARK-43879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17728642#comment-17728642 ] ASF GitHub Bot commented on SPARK-43879: User 'beliefer' has created a pull requ

[jira] [Updated] (SPARK-43943) Add math functions to Scala and Python

[ https://issues.apache.org/jira/browse/SPARK-43943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-43943: -- Description: Add following functions: * ceiling * e * pi * ln * negative * positive * power *

[jira] [Commented] (SPARK-43911) Directly use Set to consume iterator data to deduplicate, thereby reducing memory usage

[ https://issues.apache.org/jira/browse/SPARK-43911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17728634#comment-17728634 ] ASF GitHub Bot commented on SPARK-43911: User 'mcdull-zhang' has created a pull

[jira] [Created] (SPARK-43949) Upgrade Cloudpickle to 2.2.1

Hyukjin Kwon created SPARK-43949: Summary: Upgrade Cloudpickle to 2.2.1 Key: SPARK-43949 URL: https://issues.apache.org/jira/browse/SPARK-43949 Project: Spark Issue Type: Bug Compon

[jira] [Created] (SPARK-43948) Assign names to the error class _LEGACY_ERROR_TEMP_[0050|0058|0059|1204]

BingKun Pan created SPARK-43948: --- Summary: Assign names to the error class _LEGACY_ERROR_TEMP_[0050|0058|0059|1204] Key: SPARK-43948 URL: https://issues.apache.org/jira/browse/SPARK-43948 Project: Spark

[jira] [Commented] (SPARK-43946) Add rule to remove unused CTEDef

[ https://issues.apache.org/jira/browse/SPARK-43946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17728620#comment-17728620 ] jinhai-cloud commented on SPARK-43946: -- [~cloud_fan], Can you take a look at this i

[jira] [Updated] (SPARK-43946) Add rule to remove unused CTEDef

[ https://issues.apache.org/jira/browse/SPARK-43946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jinhai-cloud updated SPARK-43946: - Description: {code:java} // code placeholder with t1 as (   select rand() c3 ), t2 as (select *

[jira] [Updated] (SPARK-43946) Add rule to remove unused CTEDef

[ https://issues.apache.org/jira/browse/SPARK-43946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jinhai-cloud updated SPARK-43946: - Description: {code:java} // code placeholder with t1 as (   select rand() c3 ), t2 as (select *

[jira] [Updated] (SPARK-43947) Incorrect SparkException when missing config in resources in Stage-Level Scheduling

[ https://issues.apache.org/jira/browse/SPARK-43947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacek Laskowski updated SPARK-43947: Summary: Incorrect SparkException when missing config in resources in Stage-Level Scheduli

[jira] [Created] (SPARK-43947) Incorrect SparkException when missing amount in resources in Stage-Level Scheduling

Jacek Laskowski created SPARK-43947: --- Summary: Incorrect SparkException when missing amount in resources in Stage-Level Scheduling Key: SPARK-43947 URL: https://issues.apache.org/jira/browse/SPARK-43947

[jira] [Created] (SPARK-43946) Add rule to remove unused CTEDef

jinhai-cloud created SPARK-43946: Summary: Add rule to remove unused CTEDef Key: SPARK-43946 URL: https://issues.apache.org/jira/browse/SPARK-43946 Project: Spark Issue Type: Bug Co

[jira] [Commented] (SPARK-43945) Fix bug for `SQLQueryTestSuite` when run on local env

[ https://issues.apache.org/jira/browse/SPARK-43945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17728603#comment-17728603 ] Hudson commented on SPARK-43945: User 'panbingkun' has created a pull request for this i

[jira] [Commented] (SPARK-43798) Initial support for Python UDTFs

[ https://issues.apache.org/jira/browse/SPARK-43798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17728601#comment-17728601 ] Hudson commented on SPARK-43798: User 'allisonwang-db' has created a pull request for th

[jira] [Created] (SPARK-43945) Fix bug for `SQLQueryTestSuite` when run on local env

BingKun Pan created SPARK-43945: --- Summary: Fix bug for `SQLQueryTestSuite` when run on local env Key: SPARK-43945 URL: https://issues.apache.org/jira/browse/SPARK-43945 Project: Spark Issue Typ

[jira] [Updated] (SPARK-43916) Add percentile to Scala and Python API

[ https://issues.apache.org/jira/browse/SPARK-43916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-43916: --- Summary: Add percentile to Scala and Python API (was: Add percentile to Scala, Python and R API) >

[jira] [Resolved] (SPARK-43927) Add cast alias to Scala and Python

[ https://issues.apache.org/jira/browse/SPARK-43927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng resolved SPARK-43927. --- Resolution: Not A Problem > Add cast alias to Scala and Python > ---

[jira] [Commented] (SPARK-43907) Add SQL functions into Scala, Python and R API

[ https://issues.apache.org/jira/browse/SPARK-43907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17728598#comment-17728598 ] jiaan.geng commented on SPARK-43907: [~gurwls223]Thank you for your feedback. > Add