[jira] [Updated] (SPARK-39235) Make Catalog API be compatible with 3-layer-namespace

2022-07-07 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-39235: -- Affects Version/s: 3.4.0 (was: 3.3.0) > Make Catalog API be compati

[jira] [Updated] (SPARK-39235) Make Catalog API be compatible with 3-layer-namespace

2022-07-07 Thread Ruifeng Zheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ruifeng Zheng updated SPARK-39235: -- Component/s: PySpark R > Make Catalog API be compatible with 3-layer-namespac

[jira] [Commented] (SPARK-39716) Make currentDatabase/setCurrentDatabase/listDatabases in SparkR support 3L namespace

2022-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17564114#comment-17564114 ] Apache Spark commented on SPARK-39716: -- User 'zhengruifeng' has created a pull requ

[jira] [Commented] (SPARK-39716) Make currentDatabase/setCurrentDatabase/listDatabases in SparkR support 3L namespace

2022-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17564112#comment-17564112 ] Apache Spark commented on SPARK-39716: -- User 'zhengruifeng' has created a pull requ

[jira] [Assigned] (SPARK-39716) Make currentDatabase/setCurrentDatabase/listDatabases in SparkR support 3L namespace

2022-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39716: Assignee: (was: Apache Spark) > Make currentDatabase/setCurrentDatabase/listDatabases

[jira] [Assigned] (SPARK-39716) Make currentDatabase/setCurrentDatabase/listDatabases in SparkR support 3L namespace

2022-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39716: Assignee: Apache Spark > Make currentDatabase/setCurrentDatabase/listDatabases in SparkR

[jira] [Updated] (SPARK-39715) spark UI about file size read

2022-07-07 Thread Maple (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maple updated SPARK-39715: -- Description: I run this sql  *select * from cm_dw.dwd_client_install where proc_date=20220707;* and then

[jira] [Created] (SPARK-39716) Make currentDatabase/setCurrentDatabase/listDatabases in SparkR support 3L namespace

2022-07-07 Thread Ruifeng Zheng (Jira)
Ruifeng Zheng created SPARK-39716: - Summary: Make currentDatabase/setCurrentDatabase/listDatabases in SparkR support 3L namespace Key: SPARK-39716 URL: https://issues.apache.org/jira/browse/SPARK-39716

[jira] [Updated] (SPARK-39715) spark UI about file size read

2022-07-07 Thread Maple (Jira)
Attachments: 1.png, 2.png, 3.png > > > I run this sql  > *select * from cm_dw.dwd_client_install where proc_date=20220707;* > and then see the SQL page , it shows  the file size read: 2.1MB,it is > right,the file on my storage is 2.1MB, > but The Stages tab  shows i

[jira] [Updated] (SPARK-39715) spark UI about file size read

2022-07-07 Thread Maple (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maple updated SPARK-39715: -- Description: I run this sql  *select * from cm_dw.dwd_client_install where proc_date=20220707;* and then

[jira] [Updated] (SPARK-39715) spark UI about file size read

2022-07-07 Thread Maple (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maple updated SPARK-39715: -- Description: I run this sql  *select * from cm_dw.dwd_client_install where proc_date=20220707;* and then

[jira] [Updated] (SPARK-39715) spark UI about file size read

2022-07-07 Thread Maple (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maple updated SPARK-39715: -- Description: I run this sql  *select * from cm_dw.dwd_client_install where proc_date=20220707;* and then

[jira] [Updated] (SPARK-39715) spark UI about file size read

2022-07-07 Thread Maple (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maple updated SPARK-39715: -- Description: I run this sql  *select * from cm_dw.dwd_client_install where proc_date=20220707;* and then

[jira] [Updated] (SPARK-39715) spark UI about file size read

2022-07-07 Thread Maple (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maple updated SPARK-39715: -- Description: I run this sql  *select * from cm_dw.dwd_client_install where proc_date=20220707;* and then

[jira] [Updated] (SPARK-39715) spark UI about file size read

2022-07-07 Thread Maple (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maple updated SPARK-39715: -- Description: I run this sql  *select * from cm_dw.dwd_client_install where proc_date=20220707;* and then

[jira] [Updated] (SPARK-39715) spark UI about file size read

2022-07-07 Thread Maple (Jira)
t; Attachments: 1.png, 2.png > > > I run this sql  > *select * from cm_dw.dwd_client_install where proc_date=20220707;* > and then see then SQL TAB , it shows  the file size read: 2.1MB,it is > right.{*}{*} > *!image-2022-07-08-14-24-36-555.png!* > but on states ta

[jira] [Updated] (SPARK-39715) spark UI about file size read

2022-07-07 Thread Maple (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maple updated SPARK-39715: -- Description: I run this sql  *select * from cm_dw.dwd_client_install where proc_date=20220707;* and then

[jira] [Updated] (SPARK-39715) spark UI about file size read

2022-07-07 Thread Maple (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maple updated SPARK-39715: -- Description: I run this sql  *select * from cm_dw.dwd_client_install where proc_date=20220707;* and then

[jira] [Updated] (SPARK-39715) spark UI about file size read

2022-07-07 Thread Maple (Jira)
; Attachments: 1.png > > > I run this sql  > *select * from cm_dw.dwd_client_install where proc_date=20220707;* > and then see then SQL TAB , it shows  the file size read: 2.1MB,it is > right.{*}{*} > *!image-2022-07-08-14-24-36-555.png!* > but on states tab it show

[jira] [Created] (SPARK-39715) spark UI about file size

2022-07-07 Thread Maple (Jira)
Maple created SPARK-39715: - Summary: spark UI about file size Key: SPARK-39715 URL: https://issues.apache.org/jira/browse/SPARK-39715 Project: Spark Issue Type: Bug Components: Input/Output

[jira] [Resolved] (SPARK-39713) ANSI mode: add suggestion of using try_element_at for INVALID_ARRAY_INDEX error

2022-07-07 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-39713. Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37125 [https:

[jira] [Updated] (SPARK-39620) History server page and API are using inconsistent conditions to filter running applications

2022-07-07 Thread wy (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wy updated SPARK-39620: --- Fix Version/s: (was: 3.3.0) (was: 3.2.1) > History server page and API are using inconsis

[jira] [Commented] (SPARK-39385) Translate linear regression aggregate functions for pushdown

2022-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17564077#comment-17564077 ] Apache Spark commented on SPARK-39385: -- User 'beliefer' has created a pull request

[jira] [Commented] (SPARK-39385) Translate linear regression aggregate functions for pushdown

2022-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17564076#comment-17564076 ] Apache Spark commented on SPARK-39385: -- User 'beliefer' has created a pull request

[jira] [Assigned] (SPARK-39714) Resolve pyspark mypy part tests.

2022-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39714: Assignee: (was: Apache Spark) > Resolve pyspark mypy part tests. > --

[jira] [Commented] (SPARK-39714) Resolve pyspark mypy part tests.

2022-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17564074#comment-17564074 ] Apache Spark commented on SPARK-39714: -- User 'bzhaoopenstack' has created a pull re

[jira] [Assigned] (SPARK-39714) Resolve pyspark mypy part tests.

2022-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39714: Assignee: Apache Spark > Resolve pyspark mypy part tests. > -

[jira] [Commented] (SPARK-39714) Resolve pyspark mypy part tests.

2022-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17564073#comment-17564073 ] Apache Spark commented on SPARK-39714: -- User 'bzhaoopenstack' has created a pull re

[jira] [Commented] (SPARK-39714) Resolve pyspark mypy part tests.

2022-07-07 Thread bo zhao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17564072#comment-17564072 ] bo zhao commented on SPARK-39714: - [https://github.com/apache/spark/pull/37117] I opene

[jira] [Assigned] (SPARK-39713) ANSI mode: add suggestion of using try_element_at for INVALID_ARRAY_INDEX error

2022-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39713: Assignee: Gengliang Wang (was: Apache Spark) > ANSI mode: add suggestion of using try_el

[jira] [Assigned] (SPARK-39713) ANSI mode: add suggestion of using try_element_at for INVALID_ARRAY_INDEX error

2022-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39713: Assignee: Apache Spark (was: Gengliang Wang) > ANSI mode: add suggestion of using try_el

[jira] [Commented] (SPARK-39713) ANSI mode: add suggestion of using try_element_at for INVALID_ARRAY_INDEX error

2022-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17564071#comment-17564071 ] Apache Spark commented on SPARK-39713: -- User 'gengliangwang' has created a pull req

[jira] [Created] (SPARK-39714) Resolve pyspark mypy part tests.

2022-07-07 Thread bo zhao (Jira)
bo zhao created SPARK-39714: --- Summary: Resolve pyspark mypy part tests. Key: SPARK-39714 URL: https://issues.apache.org/jira/browse/SPARK-39714 Project: Spark Issue Type: Bug Components:

[jira] [Resolved] (SPARK-38899) DS V2 supports push down datetime functions

2022-07-07 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-38899. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36663 [https://gith

[jira] [Assigned] (SPARK-38899) DS V2 supports push down datetime functions

2022-07-07 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-38899: --- Assignee: jiaan.geng > DS V2 supports push down datetime functions > --

[jira] [Updated] (SPARK-39713) ANSI mode: add suggestion of using try_element_at for INVALID_ARRAY_INDEX error

2022-07-07 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-39713: --- Description: To make it consistent with the error INVALID_ARRAY_INDEX_IN_ELEMENT_AT and  MA

[jira] [Created] (SPARK-39713) ANSI mode: add suggestion of using try_element_at for INVALID_ARRAY_INDEX error

2022-07-07 Thread Gengliang Wang (Jira)
Gengliang Wang created SPARK-39713: -- Summary: ANSI mode: add suggestion of using try_element_at for INVALID_ARRAY_INDEX error Key: SPARK-39713 URL: https://issues.apache.org/jira/browse/SPARK-39713 P

[jira] [Commented] (SPARK-39686) Disable scheduled builds that do not pass even once

2022-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17564060#comment-17564060 ] Apache Spark commented on SPARK-39686: -- User 'HyukjinKwon' has created a pull reque

[jira] [Commented] (SPARK-39686) Disable scheduled builds that do not pass even once

2022-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17564061#comment-17564061 ] Apache Spark commented on SPARK-39686: -- User 'HyukjinKwon' has created a pull reque

[jira] [Updated] (SPARK-39712) SparkR build broken in branch-3.2 with Scala 2.13

2022-07-07 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-39712: - Component/s: Build > SparkR build broken in branch-3.2 with Scala 2.13 > ---

[jira] [Created] (SPARK-39712) SparkR build broken in branch-3.2 with Scala 2.13

2022-07-07 Thread Hyukjin Kwon (Jira)
Hyukjin Kwon created SPARK-39712: Summary: SparkR build broken in branch-3.2 with Scala 2.13 Key: SPARK-39712 URL: https://issues.apache.org/jira/browse/SPARK-39712 Project: Spark Issue Type:

[jira] [Commented] (SPARK-39711) Remove redundant trait: BeforeAndAfterAll & BeforeAndAfterEach & Logging

2022-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17564058#comment-17564058 ] Apache Spark commented on SPARK-39711: -- User 'panbingkun' has created a pull reques

[jira] [Assigned] (SPARK-39711) Remove redundant trait: BeforeAndAfterAll & BeforeAndAfterEach & Logging

2022-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39711: Assignee: Apache Spark > Remove redundant trait: BeforeAndAfterAll & BeforeAndAfterEach &

[jira] [Commented] (SPARK-39711) Remove redundant trait: BeforeAndAfterAll & BeforeAndAfterEach & Logging

2022-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17564057#comment-17564057 ] Apache Spark commented on SPARK-39711: -- User 'panbingkun' has created a pull reques

[jira] [Updated] (SPARK-39711) Remove redundant trait: BeforeAndAfterAll & BeforeAndAfterEach & Logging

2022-07-07 Thread BingKun Pan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BingKun Pan updated SPARK-39711: Description: SparkFunSuite declare as follow: {code:java} abstract class SparkFunSuite extends Any

[jira] [Assigned] (SPARK-39711) Remove redundant trait: BeforeAndAfterAll & BeforeAndAfterEach & Logging

2022-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39711: Assignee: (was: Apache Spark) > Remove redundant trait: BeforeAndAfterAll & BeforeAnd

[jira] [Created] (SPARK-39711) Remove redundant trait: BeforeAndAfterAll & BeforeAndAfterEach & Logging

2022-07-07 Thread BingKun Pan (Jira)
BingKun Pan created SPARK-39711: --- Summary: Remove redundant trait: BeforeAndAfterAll & BeforeAndAfterEach & Logging Key: SPARK-39711 URL: https://issues.apache.org/jira/browse/SPARK-39711 Project: Spark

[jira] [Updated] (SPARK-38787) Possible correctness issue on stream-stream join when handling edge case

2022-07-07 Thread Josh Rosen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-38787: --- Labels: correctness (was: ) > Possible correctness issue on stream-stream join when handling edge c

[jira] [Updated] (SPARK-37643) when charVarcharAsString is true, char datatype partition table query incorrect

2022-07-07 Thread Josh Rosen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-37643: --- Labels: correctness (was: ) > when charVarcharAsString is true, char datatype partition table query

[jira] [Updated] (SPARK-39710) Support push topK through outer join

2022-07-07 Thread XiDuo You (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiDuo You updated SPARK-39710: -- Description: Supports push down local limit and local sort from TopK through other join:   - for a lef

[jira] [Resolved] (SPARK-39693) `tpcds-1g-gen` shouldn't execute If benchmark GA does not specify to execute TPCDSQueryBenchmark

2022-07-07 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-39693. -- Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37120 [https://gi

[jira] [Assigned] (SPARK-39693) `tpcds-1g-gen` shouldn't execute If benchmark GA does not specify to execute TPCDSQueryBenchmark

2022-07-07 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-39693: Assignee: Yang Jie > `tpcds-1g-gen` shouldn't execute If benchmark GA does not specify to

[jira] [Commented] (SPARK-38018) Fix ColumnVectorUtils.populate to handle CalendarIntervalType correctly

2022-07-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17563897#comment-17563897 ] Dongjoon Hyun commented on SPARK-38018: --- I added `correctness` label. > Fix Colum

[jira] [Updated] (SPARK-38018) Fix ColumnVectorUtils.populate to handle CalendarIntervalType correctly

2022-07-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-38018: -- Labels: correctness (was: ) > Fix ColumnVectorUtils.populate to handle CalendarIntervalType c

[jira] [Assigned] (SPARK-39693) `tpcds-1g-gen` shouldn't execute If benchmark GA does not specify to execute TPCDSQueryBenchmark

2022-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39693: Assignee: (was: Apache Spark) > `tpcds-1g-gen` shouldn't execute If benchmark GA does

[jira] [Assigned] (SPARK-39693) `tpcds-1g-gen` shouldn't execute If benchmark GA does not specify to execute TPCDSQueryBenchmark

2022-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39693: Assignee: Apache Spark > `tpcds-1g-gen` shouldn't execute If benchmark GA does not specif

[jira] [Commented] (SPARK-39693) `tpcds-1g-gen` shouldn't execute If benchmark GA does not specify to execute TPCDSQueryBenchmark

2022-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17563853#comment-17563853 ] Apache Spark commented on SPARK-39693: -- User 'kazuyukitanimura' has created a pull

[jira] [Resolved] (SPARK-39385) Translate linear regression aggregate functions for pushdown

2022-07-07 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-39385. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 36773 [https://gith

[jira] [Assigned] (SPARK-39385) Translate linear regression aggregate functions for pushdown

2022-07-07 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-39385: --- Assignee: jiaan.geng > Translate linear regression aggregate functions for pushdown > -

[jira] [Updated] (SPARK-38018) Fix ColumnVectorUtils.populate to handle CalendarIntervalType correctly

2022-07-07 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-38018: Fix Version/s: 3.2.2 > Fix ColumnVectorUtils.populate to handle CalendarIntervalType correctly > -

[jira] [Resolved] (SPARK-39447) Only non-broadcast query stage can propagate empty relation

2022-07-07 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-39447. - Fix Version/s: 3.2.2 3.4.0 3.3.1 Resolution: Fixed

[jira] [Reopened] (SPARK-39447) Only non-broadcast query stage can propagate empty relation

2022-07-07 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reopened SPARK-39447: - > Only non-broadcast query stage can propagate empty relation >

[jira] [Assigned] (SPARK-39447) Only non-broadcast query stage can propagate empty relation

2022-07-07 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-39447: --- Assignee: XiDuo You > Only non-broadcast query stage can propagate empty relation > ---

[jira] [Resolved] (SPARK-33753) Reduce the memory footprint and gc of the cache (hadoopJobMetadata)

2022-07-07 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-33753. - Fix Version/s: 3.4.0 Resolution: Fixed Issue resolved by pull request 37043 [https://gith

[jira] [Assigned] (SPARK-33753) Reduce the memory footprint and gc of the cache (hadoopJobMetadata)

2022-07-07 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-33753: --- Assignee: dzcxzl > Reduce the memory footprint and gc of the cache (hadoopJobMetadata) > --

[jira] [Commented] (SPARK-38910) Clean sparkStaging dir should before unregister()

2022-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17563818#comment-17563818 ] Apache Spark commented on SPARK-38910: -- User 'AngersZh' has created a pull requ

[jira] [Commented] (SPARK-38910) Clean sparkStaging dir should before unregister()

2022-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17563817#comment-17563817 ] Apache Spark commented on SPARK-38910: -- User 'AngersZh' has created a pull requ

[jira] [Commented] (SPARK-39709) The result of executeCollect and doExecute of TakeOrderedAndProjectExec should be the same

2022-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17563806#comment-17563806 ] Apache Spark commented on SPARK-39709: -- User 'wangyum' has created a pull request f

[jira] [Assigned] (SPARK-39709) The result of executeCollect and doExecute of TakeOrderedAndProjectExec should be the same

2022-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39709: Assignee: Apache Spark > The result of executeCollect and doExecute of TakeOrderedAndProj

[jira] [Commented] (SPARK-39709) The result of executeCollect and doExecute of TakeOrderedAndProjectExec should be the same

2022-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17563804#comment-17563804 ] Apache Spark commented on SPARK-39709: -- User 'wangyum' has created a pull request f

[jira] [Assigned] (SPARK-39709) The result of executeCollect and doExecute of TakeOrderedAndProjectExec should be the same

2022-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39709: Assignee: (was: Apache Spark) > The result of executeCollect and doExecute of TakeOrd

[jira] [Updated] (SPARK-39710) Support push topK through outer join

2022-07-07 Thread XiDuo You (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiDuo You updated SPARK-39710: -- Description: Supports push down local limit and local sort from TopK through other join:   - for a lef

[jira] [Updated] (SPARK-39710) Support push topK through outer join

2022-07-07 Thread XiDuo You (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiDuo You updated SPARK-39710: -- Environment: (was: Supports push down local limit and local sort from TopK through other join:   -

[jira] [Created] (SPARK-39710) Support push topK through outer join

2022-07-07 Thread XiDuo You (Jira)
XiDuo You created SPARK-39710: - Summary: Support push topK through outer join Key: SPARK-39710 URL: https://issues.apache.org/jira/browse/SPARK-39710 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-39709) The result of executeCollect and doExecute of TakeOrderedAndProjectExec should be the same

2022-07-07 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-39709: --- Summary: The result of executeCollect and doExecute of TakeOrderedAndProjectExec should be the same Key: SPARK-39709 URL: https://issues.apache.org/jira/browse/SPARK-39709

[jira] [Resolved] (SPARK-39695) Add the REGEXP_SUBSTR function

2022-07-07 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-39695. -- Resolution: Fixed Issue resolved by pull request 37101 [https://github.com/apache/spark/pull/37101] >

[jira] [Commented] (SPARK-39707) Add SQL reference for aggregate functions

2022-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17563621#comment-17563621 ] Apache Spark commented on SPARK-39707: -- User 'beliefer' has created a pull request

[jira] [Assigned] (SPARK-39707) Add SQL reference for aggregate functions

2022-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39707: Assignee: (was: Apache Spark) > Add SQL reference for aggregate functions >

[jira] [Assigned] (SPARK-39707) Add SQL reference for aggregate functions

2022-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39707: Assignee: Apache Spark > Add SQL reference for aggregate functions > ---

[jira] [Updated] (SPARK-39708) ALS Model Loading

2022-07-07 Thread zehra (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zehra updated SPARK-39708: -- Priority: Critical (was: Major) > ALS Model Loading > - > > Key: SPARK-39708

[jira] [Created] (SPARK-39708) ALS Model Loading

2022-07-07 Thread zehra (Jira)
zehra created SPARK-39708: - Summary: ALS Model Loading Key: SPARK-39708 URL: https://issues.apache.org/jira/browse/SPARK-39708 Project: Spark Issue Type: Question Components: PySpark, Spark

[jira] [Assigned] (SPARK-39706) Set missing column with defaultValue as constant in `ParquetColumnVector`

2022-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39706: Assignee: Apache Spark > Set missing column with defaultValue as constant in `ParquetColu

[jira] [Commented] (SPARK-39706) Set missing column with defaultValue as constant in `ParquetColumnVector`

2022-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17563611#comment-17563611 ] Apache Spark commented on SPARK-39706: -- User 'LuciferYang' has created a pull reque

[jira] [Assigned] (SPARK-39706) Set missing column with defaultValue as constant in `ParquetColumnVector`

2022-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-39706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39706: Assignee: (was: Apache Spark) > Set missing column with defaultValue as constant in `

[jira] [Created] (SPARK-39707) Add SQL reference for aggregate functions

2022-07-07 Thread jiaan.geng (Jira)
jiaan.geng created SPARK-39707: -- Summary: Add SQL reference for aggregate functions Key: SPARK-39707 URL: https://issues.apache.org/jira/browse/SPARK-39707 Project: Spark Issue Type: Documentat

[jira] [Commented] (SPARK-38018) Fix ColumnVectorUtils.populate to handle CalendarIntervalType correctly

2022-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17563604#comment-17563604 ] Apache Spark commented on SPARK-38018: -- User 'c21' has created a pull request for t

[jira] [Commented] (SPARK-38018) Fix ColumnVectorUtils.populate to handle CalendarIntervalType correctly

2022-07-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-38018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17563603#comment-17563603 ] Apache Spark commented on SPARK-38018: -- User 'c21' has created a pull request for t

[jira] [Commented] (SPARK-37872) [SQL] Some classes are move from org.codehaus.janino:janino to org.codehaus.janino:common-compiler after version 3.1.x

2022-07-07 Thread gabrywu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-37872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17563599#comment-17563599 ] gabrywu commented on SPARK-37872: - Yes, janino 3.0.16 is out of date, and not compatible