[jira] [Assigned] (SPARK-40341) Implement `Rolling.median`.

[ https://issues.apache.org/jira/browse/SPARK-40341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-40341: - Assignee: Yikun Jiang > Implement `Rolling.median`. > --- > >

[jira] [Assigned] (SPARK-40344) Implement `ExpandingGroupby.median`.

[ https://issues.apache.org/jira/browse/SPARK-40344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-40344: - Assignee: Yikun Jiang > Implement `ExpandingGroupby.median`. >

[jira] [Assigned] (SPARK-40330) Implement `Series.searchsorted`.

[ https://issues.apache.org/jira/browse/SPARK-40330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-40330: - Assignee: Ruifeng Zheng > Implement `Series.searchsorted`. > --

[jira] [Assigned] (SPARK-40347) Implement `RollingGroupby.median`.

[ https://issues.apache.org/jira/browse/SPARK-40347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng reassigned SPARK-40347: - Assignee: Yikun Jiang > Implement `RollingGroupby.median`. > --

[jira] [Commented] (SPARK-33782) Place spark.files, spark.jars and spark.files under the current working directory on the driver in K8S cluster mode

[ https://issues.apache.org/jira/browse/SPARK-33782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17603426#comment-17603426 ] pralabhkumar commented on SPARK-33782: -- [~dongjoon] Please review the PR .  > Plac

[jira] [Assigned] (SPARK-40342) Implement `Rolling.quantile`.

[ https://issues.apache.org/jira/browse/SPARK-40342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40342: Assignee: (was: Apache Spark) > Implement `Rolling.quantile`. > -

[jira] [Commented] (SPARK-40342) Implement `Rolling.quantile`.

[ https://issues.apache.org/jira/browse/SPARK-40342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17603429#comment-17603429 ] Apache Spark commented on SPARK-40342: -- User 'Yikun' has created a pull request for

[jira] [Assigned] (SPARK-40342) Implement `Rolling.quantile`.

[ https://issues.apache.org/jira/browse/SPARK-40342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40342: Assignee: Apache Spark > Implement `Rolling.quantile`. > - >

[jira] [Commented] (SPARK-40342) Implement `Rolling.quantile`.

[ https://issues.apache.org/jira/browse/SPARK-40342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17603430#comment-17603430 ] Apache Spark commented on SPARK-40342: -- User 'Yikun' has created a pull request for

[jira] [Updated] (SPARK-40412) limit(x,y) + 子查询 出现数据丢失和乱序问题

[ https://issues.apache.org/jira/browse/SPARK-40412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-40412: Fix Version/s: (was: 2.4.5) > limit(x,y) + 子查询 出现数据丢失和乱序问题 > > >

[jira] [Commented] (SPARK-40412) limit(x,y) + 子查询 出现数据丢失和乱序问题

[ https://issues.apache.org/jira/browse/SPARK-40412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17603436#comment-17603436 ] Yuming Wang commented on SPARK-40412: - Could you test the latest Spark? > limit(x,y

[jira] [Updated] (SPARK-40412) limit(x,y) + 子查询 出现数据丢失和乱序问题

[ https://issues.apache.org/jira/browse/SPARK-40412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-40412: Target Version/s: (was: 2.4.5) > limit(x,y) + 子查询 出现数据丢失和乱序问题 > > >

[jira] [Created] (SPARK-40413) Column.isin produces non-boolean results

Andreas Franz created SPARK-40413: - Summary: Column.isin produces non-boolean results Key: SPARK-40413 URL: https://issues.apache.org/jira/browse/SPARK-40413 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-38734) Test the error class: INDEX_OUT_OF_BOUNDS

[ https://issues.apache.org/jira/browse/SPARK-38734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-38734: Assignee: Max Gekk > Test the error class: INDEX_OUT_OF_BOUNDS >

[jira] [Resolved] (SPARK-38734) Test the error class: INDEX_OUT_OF_BOUNDS

[ https://issues.apache.org/jira/browse/SPARK-38734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-38734. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37857 [https://github.com

[jira] [Updated] (SPARK-40413) Column.isin produces non-boolean results

[ https://issues.apache.org/jira/browse/SPARK-40413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andreas Franz updated SPARK-40413: -- Description: I observed an inconsistent behaviour using the Column.isin function. The [docume

[jira] [Updated] (SPARK-40177) Simplify join condition of form (a==b) || (a==null&&b==null) to a<=>b

[ https://issues.apache.org/jira/browse/SPARK-40177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-40177: Fix Version/s: (was: 3.3.1) > Simplify join condition of form (a==b) || (a==null&&b==null) to

[jira] [Updated] (SPARK-40177) Simplify join condition of form (a==b) || (a==null&&b==null) to a<=>b

[ https://issues.apache.org/jira/browse/SPARK-40177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-40177: Target Version/s: (was: 3.3.1) > Simplify join condition of form (a==b) || (a==null&&b==null) to

[jira] [Updated] (SPARK-40178) Rebalance/Repartition Hints Not Working in PySpark

[ https://issues.apache.org/jira/browse/SPARK-40178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-40178: Fix Version/s: (was: 3.2.0) (was: 3.3.0) (was: 3.

[jira] [Updated] (SPARK-40178) Rebalance/Repartition Hints Not Working in PySpark

[ https://issues.apache.org/jira/browse/SPARK-40178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-40178: Target Version/s: (was: 3.2.0, 3.2.1, 3.3.0, 3.2.2, 3.4.0, 3.3.1) > Rebalance/Repartition Hints

[jira] [Commented] (SPARK-40324) Provide a query context of ParseException

[ https://issues.apache.org/jira/browse/SPARK-40324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17603462#comment-17603462 ] Apache Spark commented on SPARK-40324: -- User 'gengliangwang' has created a pull req

[jira] [Commented] (SPARK-40334) Implement `GroupBy.prod`.

[ https://issues.apache.org/jira/browse/SPARK-40334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17603463#comment-17603463 ] Artsiom Yudovin commented on SPARK-40334: - [~itholic], Hi, I have been started t

[jira] [Commented] (SPARK-40412) limit(x,y) + 子查询 出现数据丢失和乱序问题

[ https://issues.apache.org/jira/browse/SPARK-40412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17603473#comment-17603473 ] FengJia commented on SPARK-40412: - The Huawei cloud I use cannot change the Spark versio

[jira] [Resolved] (SPARK-40412) limit(x,y) + 子查询 出现数据丢失和乱序问题

[ https://issues.apache.org/jira/browse/SPARK-40412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-40412. - Resolution: Invalid Spark SQL does not support \{{limit n, m}}. Please contact the huawei cloud.

[jira] [Commented] (SPARK-40412) limit(x,y) + 子查询 出现数据丢失和乱序问题

[ https://issues.apache.org/jira/browse/SPARK-40412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17603485#comment-17603485 ] FengJia commented on SPARK-40412: - Huawei Cloud's solution is to add Order by > limit(x

[jira] [Created] (SPARK-40414) Fix PythonArrowInput and PythonArrowOutput to be more generic to handle complicated type/data

Jungtaek Lim created SPARK-40414: Summary: Fix PythonArrowInput and PythonArrowOutput to be more generic to handle complicated type/data Key: SPARK-40414 URL: https://issues.apache.org/jira/browse/SPARK-40414

[jira] [Commented] (SPARK-40414) Fix PythonArrowInput and PythonArrowOutput to be more generic to handle complicated type/data

[ https://issues.apache.org/jira/browse/SPARK-40414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17603559#comment-17603559 ] Jungtaek Lim commented on SPARK-40414: -- Will submit a PR sooner. > Fix PythonArrow

[jira] [Assigned] (SPARK-40414) Fix PythonArrowInput and PythonArrowOutput to be more generic to handle complicated type/data

[ https://issues.apache.org/jira/browse/SPARK-40414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40414: Assignee: (was: Apache Spark) > Fix PythonArrowInput and PythonArrowOutput to be more

[jira] [Commented] (SPARK-40414) Fix PythonArrowInput and PythonArrowOutput to be more generic to handle complicated type/data

[ https://issues.apache.org/jira/browse/SPARK-40414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17603571#comment-17603571 ] Apache Spark commented on SPARK-40414: -- User 'HeartSaVioR' has created a pull reque

[jira] [Assigned] (SPARK-40414) Fix PythonArrowInput and PythonArrowOutput to be more generic to handle complicated type/data

[ https://issues.apache.org/jira/browse/SPARK-40414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40414: Assignee: Apache Spark > Fix PythonArrowInput and PythonArrowOutput to be more generic to

[jira] [Resolved] (SPARK-40384) Do base image real in time build only when infra dockerfile is changed

[ https://issues.apache.org/jira/browse/SPARK-40384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-40384. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37828 [https://gi

[jira] [Assigned] (SPARK-40384) Do base image real in time build only when infra dockerfile is changed

[ https://issues.apache.org/jira/browse/SPARK-40384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-40384: Assignee: Yikun Jiang > Do base image real in time build only when infra dockerfile is ch

[jira] [Commented] (SPARK-20624) SPIP: Add better handling for node shutdown

[ https://issues.apache.org/jira/browse/SPARK-20624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17603577#comment-17603577 ] Juliusz Sompolski commented on SPARK-20624: --- [~holden] Are these new APIs docu

[jira] [Resolved] (SPARK-40400) Pass error message parameters to exceptions as a map

[ https://issues.apache.org/jira/browse/SPARK-40400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-40400. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37834 [https://github.com

[jira] [Commented] (SPARK-40384) Do base image real in time build only when infra dockerfile is changed

[ https://issues.apache.org/jira/browse/SPARK-40384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17603603#comment-17603603 ] Apache Spark commented on SPARK-40384: -- User 'Yikun' has created a pull request for

[jira] [Commented] (SPARK-40334) Implement `GroupBy.prod`.

[ https://issues.apache.org/jira/browse/SPARK-40334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17603611#comment-17603611 ] Haejoon Lee commented on SPARK-40334: - [~ayudovin] No worries! Please keep working o

[jira] [Commented] (SPARK-40362) Bug in Canonicalization of expressions like Add & Multiply i.e Commutative Operators

[ https://issues.apache.org/jira/browse/SPARK-40362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17603619#comment-17603619 ] Apache Spark commented on SPARK-40362: -- User 'peter-toth' has created a pull reques

[jira] [Created] (SPARK-40415) Wrong version of okio in spark-deps file

Yang Jie created SPARK-40415: Summary: Wrong version of okio in spark-deps file Key: SPARK-40415 URL: https://issues.apache.org/jira/browse/SPARK-40415 Project: Spark Issue Type: Bug Co

[jira] [Updated] (SPARK-40415) Wrong version of okio in spark-deps file

[ https://issues.apache.org/jira/browse/SPARK-40415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-40415: - Attachment: tree-before.txt > Wrong version of okio in spark-deps file > ---

[jira] [Commented] (SPARK-40415) Wrong version of okio in spark-deps file

[ https://issues.apache.org/jira/browse/SPARK-40415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17603631#comment-17603631 ] Apache Spark commented on SPARK-40415: -- User 'LuciferYang' has created a pull reque

[jira] [Assigned] (SPARK-40415) Wrong version of okio in spark-deps file

[ https://issues.apache.org/jira/browse/SPARK-40415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40415: Assignee: (was: Apache Spark) > Wrong version of okio in spark-deps file > --

[jira] [Assigned] (SPARK-40415) Wrong version of okio in spark-deps file

[ https://issues.apache.org/jira/browse/SPARK-40415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40415: Assignee: Apache Spark > Wrong version of okio in spark-deps file > -

[jira] [Commented] (SPARK-40397) Migrate selenium-java from 3.1 to 4.2 and upgrade org.scalatestplus:selenium to 3.2.13.0

[ https://issues.apache.org/jira/browse/SPARK-40397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17603650#comment-17603650 ] Apache Spark commented on SPARK-40397: -- User 'LuciferYang' has created a pull reque

[jira] [Assigned] (SPARK-40397) Migrate selenium-java from 3.1 to 4.2 and upgrade org.scalatestplus:selenium to 3.2.13.0

[ https://issues.apache.org/jira/browse/SPARK-40397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40397: Assignee: Apache Spark > Migrate selenium-java from 3.1 to 4.2 and upgrade org.scalatestp

[jira] [Assigned] (SPARK-40397) Migrate selenium-java from 3.1 to 4.2 and upgrade org.scalatestplus:selenium to 3.2.13.0

[ https://issues.apache.org/jira/browse/SPARK-40397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40397: Assignee: (was: Apache Spark) > Migrate selenium-java from 3.1 to 4.2 and upgrade org

[jira] [Commented] (SPARK-40397) Migrate selenium-java from 3.1 to 4.2 and upgrade org.scalatestplus:selenium to 3.2.13.0

[ https://issues.apache.org/jira/browse/SPARK-40397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17603652#comment-17603652 ] Apache Spark commented on SPARK-40397: -- User 'LuciferYang' has created a pull reque

[jira] [Commented] (SPARK-33152) SPIP: Constraint Propagation code causes OOM issues or increasing compilation time to hours

[ https://issues.apache.org/jira/browse/SPARK-33152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17603663#comment-17603663 ] Apache Spark commented on SPARK-33152: -- User 'ahshahid' has created a pull request

[jira] [Commented] (SPARK-33152) SPIP: Constraint Propagation code causes OOM issues or increasing compilation time to hours

[ https://issues.apache.org/jira/browse/SPARK-33152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17603664#comment-17603664 ] Apache Spark commented on SPARK-33152: -- User 'ahshahid' has created a pull request

[jira] [Updated] (SPARK-33152) SPIP: Constraint Propagation code causes OOM issues or increasing compilation time to hours

[ https://issues.apache.org/jira/browse/SPARK-33152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Asif updated SPARK-33152: - Shepherd: Wenchen Fan (was: Arnaud Doucet) Description: h2. Q1. What are you trying to do? Articulate yo

[jira] [Resolved] (SPARK-40415) Wrong version of okio in spark-deps file

[ https://issues.apache.org/jira/browse/SPARK-40415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie resolved SPARK-40415. -- Resolution: Won't Fix > Wrong version of okio in spark-deps file > ---

[jira] [Updated] (SPARK-40394) Move CheckAnalysis error messages to use the new error framework

[ https://issues.apache.org/jira/browse/SPARK-40394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel updated SPARK-40394: --- Summary: Move CheckAnalysis error messages to use the new error framework (was: Move subquery expression Ch

[jira] [Created] (SPARK-40416) Add error classes for subquery expression CheckAnalysis failures

Daniel created SPARK-40416: -- Summary: Add error classes for subquery expression CheckAnalysis failures Key: SPARK-40416 URL: https://issues.apache.org/jira/browse/SPARK-40416 Project: Spark Issue T

[jira] [Commented] (SPARK-40416) Add error classes for subquery expression CheckAnalysis failures

[ https://issues.apache.org/jira/browse/SPARK-40416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17603682#comment-17603682 ] Apache Spark commented on SPARK-40416: -- User 'dtenedor' has created a pull request

[jira] [Assigned] (SPARK-40416) Add error classes for subquery expression CheckAnalysis failures

[ https://issues.apache.org/jira/browse/SPARK-40416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40416: Assignee: Apache Spark > Add error classes for subquery expression CheckAnalysis failures

[jira] [Assigned] (SPARK-40416) Add error classes for subquery expression CheckAnalysis failures

[ https://issues.apache.org/jira/browse/SPARK-40416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40416: Assignee: (was: Apache Spark) > Add error classes for subquery expression CheckAnalys

[jira] [Created] (SPARK-40417) Use YuniKorn v1.1+

Dongjoon Hyun created SPARK-40417: - Summary: Use YuniKorn v1.1+ Key: SPARK-40417 URL: https://issues.apache.org/jira/browse/SPARK-40417 Project: Spark Issue Type: Improvement Compon

[jira] [Assigned] (SPARK-40417) Use YuniKorn v1.1+

[ https://issues.apache.org/jira/browse/SPARK-40417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40417: Assignee: Apache Spark > Use YuniKorn v1.1+ > -- > > Key:

[jira] [Commented] (SPARK-40417) Use YuniKorn v1.1+

[ https://issues.apache.org/jira/browse/SPARK-40417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17603710#comment-17603710 ] Apache Spark commented on SPARK-40417: -- User 'dongjoon-hyun' has created a pull req

[jira] [Assigned] (SPARK-40417) Use YuniKorn v1.1+

[ https://issues.apache.org/jira/browse/SPARK-40417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40417: Assignee: (was: Apache Spark) > Use YuniKorn v1.1+ > -- > >

[jira] [Updated] (SPARK-40417) Use YuniKorn v1.1+

[ https://issues.apache.org/jira/browse/SPARK-40417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-40417: -- Issue Type: Documentation (was: Improvement) > Use YuniKorn v1.1+ > -- > >

[jira] [Created] (SPARK-40418) Increase default initialNumPartitions to 10

Ziqi Liu created SPARK-40418: Summary: Increase default initialNumPartitions to 10 Key: SPARK-40418 URL: https://issues.apache.org/jira/browse/SPARK-40418 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-40362) Bug in Canonicalization of expressions like Add & Multiply i.e Commutative Operators

[ https://issues.apache.org/jira/browse/SPARK-40362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-40362. --- Fix Version/s: 3.3.1 Resolution: Fixed Issue resolved by pull request 37866 [https://

[jira] [Assigned] (SPARK-40362) Bug in Canonicalization of expressions like Add & Multiply i.e Commutative Operators

[ https://issues.apache.org/jira/browse/SPARK-40362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-40362: - Assignee: Peter Toth > Bug in Canonicalization of expressions like Add & Multiply i.e C

[jira] [Resolved] (SPARK-40417) Use YuniKorn v1.1+

[ https://issues.apache.org/jira/browse/SPARK-40417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-40417. --- Fix Version/s: 3.3.1 3.4.0 Resolution: Fixed Issue resolved by pul

[jira] [Assigned] (SPARK-40417) Use YuniKorn v1.1+

[ https://issues.apache.org/jira/browse/SPARK-40417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-40417: - Assignee: Dongjoon Hyun > Use YuniKorn v1.1+ > -- > > K

[jira] [Updated] (SPARK-40384) Do base image real in time build only when infra dockerfile is changed

[ https://issues.apache.org/jira/browse/SPARK-40384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-40384: - Fix Version/s: (was: 3.4.0) > Do base image real in time build only when infra dockerfile is

[jira] [Reopened] (SPARK-40384) Do base image real in time build only when infra dockerfile is changed

[ https://issues.apache.org/jira/browse/SPARK-40384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reopened SPARK-40384: -- > Do base image real in time build only when infra dockerfile is changed > ---

[jira] [Assigned] (SPARK-40384) Do base image real in time build only when infra dockerfile is changed

[ https://issues.apache.org/jira/browse/SPARK-40384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40384: Assignee: Yikun Jiang (was: Apache Spark) > Do base image real in time build only when i

[jira] [Assigned] (SPARK-40384) Do base image real in time build only when infra dockerfile is changed

[ https://issues.apache.org/jira/browse/SPARK-40384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40384: Assignee: Apache Spark (was: Yikun Jiang) > Do base image real in time build only when i

[jira] [Commented] (SPARK-40384) Do base image real in time build only when infra dockerfile is changed

[ https://issues.apache.org/jira/browse/SPARK-40384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17603798#comment-17603798 ] Hyukjin Kwon commented on SPARK-40384: -- Reverted at https://github.com/apache/spar

[jira] [Deleted] (SPARK-40309) Introduce sql_conf context manager for pyspark.sql

[ https://issues.apache.org/jira/browse/SPARK-40309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinrong Meng deleted SPARK-40309: - > Introduce sql_conf context manager for pyspark.sql >

[jira] [Commented] (SPARK-40287) Load Data using Spark by a single partition moves entire dataset under same location in S3

[ https://issues.apache.org/jira/browse/SPARK-40287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17603819#comment-17603819 ] Drew commented on SPARK-40287: -- Hey [~ste...@apache.org], Yes, I get the same functionali

[jira] [Commented] (SPARK-40286) Load Data from S3 deletes data source file

[ https://issues.apache.org/jira/browse/SPARK-40286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17603826#comment-17603826 ] Drew commented on SPARK-40286: -- Hi [~ste...@apache.org], Yeah, is there anything significa

[jira] [Resolved] (SPARK-40403) Negative size in error message when unsafe array is too big

[ https://issues.apache.org/jira/browse/SPARK-40403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-40403. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37852 [https://gi

[jira] [Assigned] (SPARK-40403) Negative size in error message when unsafe array is too big

[ https://issues.apache.org/jira/browse/SPARK-40403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-40403: Assignee: Bruce Robbins > Negative size in error message when unsafe array is too big > -

[jira] [Updated] (SPARK-40405) sparksql throws exception while reading by jdbc

[ https://issues.apache.org/jira/browse/SPARK-40405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-40405: - Priority: Major (was: Critical) > sparksql throws exception while reading by jdbc > ---

[jira] [Updated] (SPARK-40418) Increase default initialNumPartitions to 10

[ https://issues.apache.org/jira/browse/SPARK-40418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-40418: - Component/s: SQL > Increase default initialNumPartitions to 10 > ---

[jira] [Commented] (SPARK-40405) sparksql throws exception while reading by jdbc

[ https://issues.apache.org/jira/browse/SPARK-40405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17603861#comment-17603861 ] Hyukjin Kwon commented on SPARK-40405: -- [~ghsea]seems like it's a classpath problem

[jira] [Commented] (SPARK-40390) Spark Master UI - SSL implementation

[ https://issues.apache.org/jira/browse/SPARK-40390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17603862#comment-17603862 ] Hyukjin Kwon commented on SPARK-40390: -- Is this still an issue in Spark 3.1+? Spark

[jira] [Commented] (SPARK-40390) Spark Master UI - SSL implementation

[ https://issues.apache.org/jira/browse/SPARK-40390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17603867#comment-17603867 ] Rhajvijay Manoharan commented on SPARK-40390: - [~hyukjin.kwon] We will try u

[jira] [Created] (SPARK-40419) Integrate aggregate pandas UDFs into *.sql test cases

Haejoon Lee created SPARK-40419: --- Summary: Integrate aggregate pandas UDFs into *.sql test cases Key: SPARK-40419 URL: https://issues.apache.org/jira/browse/SPARK-40419 Project: Spark Issue Typ

[jira] [Updated] (SPARK-40419) Integrate Grouped Aggregate Pandas UDFs into *.sql test cases

[ https://issues.apache.org/jira/browse/SPARK-40419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-40419: Summary: Integrate Grouped Aggregate Pandas UDFs into *.sql test cases (was: Integrate aggregate

[jira] [Assigned] (SPARK-40419) Integrate Grouped Aggregate Pandas UDFs into *.sql test cases

[ https://issues.apache.org/jira/browse/SPARK-40419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40419: Assignee: Apache Spark > Integrate Grouped Aggregate Pandas UDFs into *.sql test cases >

[jira] [Assigned] (SPARK-40419) Integrate Grouped Aggregate Pandas UDFs into *.sql test cases

[ https://issues.apache.org/jira/browse/SPARK-40419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40419: Assignee: (was: Apache Spark) > Integrate Grouped Aggregate Pandas UDFs into *.sql te

[jira] [Commented] (SPARK-40419) Integrate Grouped Aggregate Pandas UDFs into *.sql test cases

[ https://issues.apache.org/jira/browse/SPARK-40419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17603880#comment-17603880 ] Apache Spark commented on SPARK-40419: -- User 'itholic' has created a pull request f

[jira] [Assigned] (SPARK-40414) Fix PythonArrowInput and PythonArrowOutput to be more generic to handle complicated type/data

[ https://issues.apache.org/jira/browse/SPARK-40414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-40414: Assignee: Jungtaek Lim > Fix PythonArrowInput and PythonArrowOutput to be more generic to

[jira] [Resolved] (SPARK-40414) Fix PythonArrowInput and PythonArrowOutput to be more generic to handle complicated type/data

[ https://issues.apache.org/jira/browse/SPARK-40414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-40414. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37864 [https://gi

[jira] [Updated] (SPARK-40303) The performance will be worse after codegen

[ https://issues.apache.org/jira/browse/SPARK-40303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-40303: - Attachment: TestApiBenchmark.scala TestApis.java TestParameters.java > T

[jira] [Commented] (SPARK-40303) The performance will be worse after codegen

[ https://issues.apache.org/jira/browse/SPARK-40303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17603883#comment-17603883 ] Yang Jie commented on SPARK-40303: -- I did a simple experiment to compare the following

[jira] [Comment Edited] (SPARK-40303) The performance will be worse after codegen

[ https://issues.apache.org/jira/browse/SPARK-40303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17603883#comment-17603883 ] Yang Jie edited comment on SPARK-40303 at 9/14/22 5:24 AM: --- I

[jira] [Created] (SPARK-40420) Sort message parameters in the JSON formats

Max Gekk created SPARK-40420: Summary: Sort message parameters in the JSON formats Key: SPARK-40420 URL: https://issues.apache.org/jira/browse/SPARK-40420 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-40421) Make `spearman` correlation in `DataFrame.corr` support missing values and `min_periods`

Ruifeng Zheng created SPARK-40421: - Summary: Make `spearman` correlation in `DataFrame.corr` support missing values and `min_periods` Key: SPARK-40421 URL: https://issues.apache.org/jira/browse/SPARK-40421

[jira] [Commented] (SPARK-40421) Make `spearman` correlation in `DataFrame.corr` support missing values and `min_periods`

[ https://issues.apache.org/jira/browse/SPARK-40421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17603891#comment-17603891 ] Apache Spark commented on SPARK-40421: -- User 'zhengruifeng' has created a pull requ

[jira] [Assigned] (SPARK-40421) Make `spearman` correlation in `DataFrame.corr` support missing values and `min_periods`

[ https://issues.apache.org/jira/browse/SPARK-40421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40421: Assignee: (was: Apache Spark) > Make `spearman` correlation in `DataFrame.corr` suppo

[jira] [Assigned] (SPARK-40421) Make `spearman` correlation in `DataFrame.corr` support missing values and `min_periods`

[ https://issues.apache.org/jira/browse/SPARK-40421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40421: Assignee: Apache Spark > Make `spearman` correlation in `DataFrame.corr` support missing

[jira] [Commented] (SPARK-40421) Make `spearman` correlation in `DataFrame.corr` support missing values and `min_periods`

[ https://issues.apache.org/jira/browse/SPARK-40421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17603892#comment-17603892 ] Apache Spark commented on SPARK-40421: -- User 'zhengruifeng' has created a pull requ

[jira] [Assigned] (SPARK-40420) Sort message parameters in the JSON formats

[ https://issues.apache.org/jira/browse/SPARK-40420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40420: Assignee: Max Gekk (was: Apache Spark) > Sort message parameters in the JSON formats > -

[jira] [Commented] (SPARK-40420) Sort message parameters in the JSON formats

[ https://issues.apache.org/jira/browse/SPARK-40420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17603894#comment-17603894 ] Apache Spark commented on SPARK-40420: -- User 'MaxGekk' has created a pull request f

[jira] [Assigned] (SPARK-40420) Sort message parameters in the JSON formats

[ https://issues.apache.org/jira/browse/SPARK-40420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40420: Assignee: Apache Spark (was: Max Gekk) > Sort message parameters in the JSON formats > -

[jira] [Commented] (SPARK-32563) spark-sql doesn't support insert into mixed static & dynamic partition

[ https://issues.apache.org/jira/browse/SPARK-32563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17603901#comment-17603901 ] Zhen Wang commented on SPARK-32563: --- In Spark 3.1.1 and hive 3.1.2, this issue also oc

  1   2   >