[jira] [Updated] (SPARK-40626) Do not reorder join keys in EnsureRequirements if they are not simple expressions

2022-09-30 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-40626: Description: {code:scala} sql("CREATE TABLE t1 (itemid BIGINT, eventType STRING, dt STRING) USING

[jira] [Commented] (SPARK-40509) Construct an example of applyInPandasWithState in examples directory

2022-09-30 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17611825#comment-17611825 ] Apache Spark commented on SPARK-40509: -- User 'chaoqin-li1123' has created a pull re

[jira] [Commented] (SPARK-40509) Construct an example of applyInPandasWithState in examples directory

2022-09-30 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17611826#comment-17611826 ] Apache Spark commented on SPARK-40509: -- User 'chaoqin-li1123' has created a pull re

[jira] [Created] (SPARK-40626) Do not reorder join keys in EnsureRequirements if they are not simple expressions

2022-09-30 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-40626: --- Summary: Do not reorder join keys in EnsureRequirements if they are not simple expressions Key: SPARK-40626 URL: https://issues.apache.org/jira/browse/SPARK-40626 Proje

[jira] [Commented] (SPARK-40624) A DECIMAL value with division by 0 errors in DataFrame but evaluates to NULL in SparkSQL

2022-09-30 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17611803#comment-17611803 ] Bruce Robbins commented on SPARK-40624: --- That's not a Spark API throwing that exce

[jira] [Assigned] (SPARK-40625) Add MASK_CCN and TRY_MASK_CCN functions

2022-09-30 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40625: Assignee: (was: Apache Spark) > Add MASK_CCN and TRY_MASK_CCN functions > ---

[jira] [Commented] (SPARK-40625) Add MASK_CCN and TRY_MASK_CCN functions

2022-09-30 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17611801#comment-17611801 ] Apache Spark commented on SPARK-40625: -- User 'dtenedor' has created a pull request

[jira] [Assigned] (SPARK-40625) Add MASK_CCN and TRY_MASK_CCN functions

2022-09-30 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40625: Assignee: Apache Spark > Add MASK_CCN and TRY_MASK_CCN functions > --

[jira] [Created] (SPARK-40625) Add MASK_CCN and TRY_MASK_CCN functions

2022-09-30 Thread Daniel (Jira)
Daniel created SPARK-40625: -- Summary: Add MASK_CCN and TRY_MASK_CCN functions Key: SPARK-40625 URL: https://issues.apache.org/jira/browse/SPARK-40625 Project: Spark Issue Type: Sub-task Co

[jira] [Assigned] (SPARK-40622) Result of a single task in collect() must fit in 2GB

2022-09-30 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40622: Assignee: (was: Apache Spark) > Result of a single task in collect() must fit in 2GB

[jira] [Commented] (SPARK-40622) Result of a single task in collect() must fit in 2GB

2022-09-30 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17611800#comment-17611800 ] Apache Spark commented on SPARK-40622: -- User 'liuzqt' has created a pull request fo

[jira] [Assigned] (SPARK-40622) Result of a single task in collect() must fit in 2GB

2022-09-30 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40622: Assignee: Apache Spark > Result of a single task in collect() must fit in 2GB > -

[jira] [Updated] (SPARK-40624) A DECIMAL value with division by 0 errors in DataFrame but evaluates to NULL in SparkSQL

2022-09-30 Thread xsys (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] xsys updated SPARK-40624: - Description: h3. Describe the bug Storing an invalid value (e.g. {{{}BigDecimal("1.0/0"){}}}) via {{spark-shell

[jira] [Created] (SPARK-40624) A DECIMAL value with division by 0 errors in DataFrame but evaluates to NULL in SparkSQL

2022-09-30 Thread xsys (Jira)
xsys created SPARK-40624: Summary: A DECIMAL value with division by 0 errors in DataFrame but evaluates to NULL in SparkSQL Key: SPARK-40624 URL: https://issues.apache.org/jira/browse/SPARK-40624 Project: Spa

[jira] [Created] (SPARK-40623) Add new SQL built-in functions to help with redacting data

2022-09-30 Thread Daniel (Jira)
Daniel created SPARK-40623: -- Summary: Add new SQL built-in functions to help with redacting data Key: SPARK-40623 URL: https://issues.apache.org/jira/browse/SPARK-40623 Project: Spark Issue Type: Ne

[jira] [Resolved] (SPARK-40612) On Kubernetes for long running app Spark using an invalid principal to renew the delegation token

2022-09-30 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-40612. --- Fix Version/s: 3.3.2 3.2.3 3.4.0 Resolution: Fix

[jira] [Commented] (SPARK-40569) Expose port for spark standalone mode

2022-09-30 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-40569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17611730#comment-17611730 ] Bjørn Jørgensen commented on SPARK-40569: - Like this on https://github.com/jupyt

[jira] [Commented] (SPARK-39725) Upgrade jetty-http from 9.4.46.v20220331 to 9.4.48.v20220622

2022-09-30 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-39725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17611725#comment-17611725 ] Bjørn Jørgensen commented on SPARK-39725: - Yes, for the release question I do wi

[jira] [Updated] (SPARK-40622) Result of a single task in collect() must fit in 2GB

2022-09-30 Thread Ziqi Liu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ziqi Liu updated SPARK-40622: - Description: when collecting results, data from single partition/task is serialized through byte array

[jira] [Created] (SPARK-40622) Result of a single task in collect() must fit in 2GB

2022-09-30 Thread Ziqi Liu (Jira)
Ziqi Liu created SPARK-40622: Summary: Result of a single task in collect() must fit in 2GB Key: SPARK-40622 URL: https://issues.apache.org/jira/browse/SPARK-40622 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-40540) Migrate compilation errors onto error classes

2022-09-30 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17611685#comment-17611685 ] Apache Spark commented on SPARK-40540: -- User 'MaxGekk' has created a pull request f

[jira] [Commented] (SPARK-40448) Prototype implementation

2022-09-30 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17611553#comment-17611553 ] Apache Spark commented on SPARK-40448: -- User 'beliefer' has created a pull request

[jira] [Resolved] (SPARK-40621) Implement `numeric_only` and `min_count` in `GroupBy.sum`

2022-09-30 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-40621. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38060 [https://gi

[jira] [Assigned] (SPARK-40621) Implement `numeric_only` and `min_count` in `GroupBy.sum`

2022-09-30 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-40621: Assignee: Ruifeng Zheng > Implement `numeric_only` and `min_count` in `GroupBy.sum` > ---

[jira] [Updated] (SPARK-40165) Update test plugins to latest versions

2022-09-30 Thread BingKun Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BingKun Pan updated SPARK-40165: Description: Include: * 1.scalacheck (from 1.16.0 to 1.17.0) * 2.maven-surefire-plugin (from 3.0

[jira] [Commented] (SPARK-40621) Implement `numeric_only` and `min_count` in `GroupBy.sum`

2022-09-30 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17611450#comment-17611450 ] Apache Spark commented on SPARK-40621: -- User 'zhengruifeng' has created a pull requ

[jira] [Assigned] (SPARK-40621) Implement `numeric_only` and `min_count` in `GroupBy.sum`

2022-09-30 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40621: Assignee: Apache Spark > Implement `numeric_only` and `min_count` in `GroupBy.sum` >

[jira] [Assigned] (SPARK-40621) Implement `numeric_only` and `min_count` in `GroupBy.sum`

2022-09-30 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40621: Assignee: (was: Apache Spark) > Implement `numeric_only` and `min_count` in `GroupBy.

[jira] [Updated] (SPARK-40621) Implement `numeric_only` and `min_count` in `GroupBy.sum`

2022-09-30 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-40621: -- Summary: Implement `numeric_only` and `min_count` in `GroupBy.sum` (was: Add `numeric_only` a

[jira] [Created] (SPARK-40621) Add `numeric_only` and `min_count` in `GroupBy.sum`

2022-09-30 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-40621: - Summary: Add `numeric_only` and `min_count` in `GroupBy.sum` Key: SPARK-40621 URL: https://issues.apache.org/jira/browse/SPARK-40621 Project: Spark Issue T

[jira] [Commented] (SPARK-40563) Error at where clause, when sql case executes by else branch

2022-09-30 Thread Vadim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17611409#comment-17611409 ] Vadim commented on SPARK-40563: --- [~Zing]  Our respect, thanks for the help! > Error at w

[jira] [Commented] (SPARK-40619) HivePartitionFilteringSuites teset aborted due to `java.lang.OutOfMemoryError: Metaspace`

2022-09-30 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17611403#comment-17611403 ] Apache Spark commented on SPARK-40619: -- User 'LuciferYang' has created a pull reque

[jira] [Assigned] (SPARK-40619) HivePartitionFilteringSuites teset aborted due to `java.lang.OutOfMemoryError: Metaspace`

2022-09-30 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-40619: Assignee: Yang Jie > HivePartitionFilteringSuites teset aborted due to > `java.lang.OutO

[jira] [Commented] (SPARK-40620) Deduplication of WorkerOffer build in CoarseGrainedSchedulerBackend

2022-09-30 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17611400#comment-17611400 ] Apache Spark commented on SPARK-40620: -- User 'khalidmammadov' has created a pull re

[jira] [Assigned] (SPARK-40620) Deduplication of WorkerOffer build in CoarseGrainedSchedulerBackend

2022-09-30 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40620: Assignee: (was: Apache Spark) > Deduplication of WorkerOffer build in CoarseGrainedSc

[jira] [Resolved] (SPARK-40619) HivePartitionFilteringSuites teset aborted due to `java.lang.OutOfMemoryError: Metaspace`

2022-09-30 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-40619. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 38057 [https://gi

[jira] [Assigned] (SPARK-40620) Deduplication of WorkerOffer build in CoarseGrainedSchedulerBackend

2022-09-30 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-40620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-40620: Assignee: Apache Spark > Deduplication of WorkerOffer build in CoarseGrainedSchedulerBack

[jira] [Created] (SPARK-40620) Deduplication of WorkerOffer build in CoarseGrainedSchedulerBackend

2022-09-30 Thread Khalid Mammadov (Jira)
Khalid Mammadov created SPARK-40620: --- Summary: Deduplication of WorkerOffer build in CoarseGrainedSchedulerBackend Key: SPARK-40620 URL: https://issues.apache.org/jira/browse/SPARK-40620 Project: Sp