[jira] [Updated] (SPARK-37922) Combine to one cast if we can safely up-cast two casts

[ https://issues.apache.org/jira/browse/SPARK-37922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-37922: Summary: Combine to one cast if we can safely up-cast two casts (was: Remove useless cast) >

[jira] [Reopened] (SPARK-37933) Limit push down for parquet datasource v2

[ https://issues.apache.org/jira/browse/SPARK-37933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jackey Lee reopened SPARK-37933: The pre pr is only to optimize a part of the code, there will be other prs for this jira in the

[jira] (SPARK-37933) Limit push down for parquet datasource v2

[ https://issues.apache.org/jira/browse/SPARK-37933 ] Jackey Lee deleted comment on SPARK-37933: was (Author: jackey lee): [https://github.com/apache/spark/pull/35242] This pr is only to optimize a part of the code, there will be other prs for

[jira] [Commented] (SPARK-37929) Support cascade mode for `dropNamespace` API

[ https://issues.apache.org/jira/browse/SPARK-37929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17478357#comment-17478357 ] Apache Spark commented on SPARK-37929: -- User 'dchvn' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37929) Support cascade mode for `dropNamespace` API

[ https://issues.apache.org/jira/browse/SPARK-37929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37929: Assignee: (was: Apache Spark) > Support cascade mode for `dropNamespace` API >

[jira] [Commented] (SPARK-37929) Support cascade mode for `dropNamespace` API

[ https://issues.apache.org/jira/browse/SPARK-37929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17478356#comment-17478356 ] Apache Spark commented on SPARK-37929: -- User 'dchvn' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37929) Support cascade mode for `dropNamespace` API

[ https://issues.apache.org/jira/browse/SPARK-37929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37929: Assignee: Apache Spark > Support cascade mode for `dropNamespace` API >

[jira] [Commented] (SPARK-37769) Filter on the metadata struct

[ https://issues.apache.org/jira/browse/SPARK-37769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17478354#comment-17478354 ] Apache Spark commented on SPARK-37769: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Resolved] (SPARK-37933) Limit push down for parquet datasource v2

[ https://issues.apache.org/jira/browse/SPARK-37933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-37933. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35242

[jira] [Assigned] (SPARK-37933) Limit push down for parquet datasource v2

[ https://issues.apache.org/jira/browse/SPARK-37933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-37933: Assignee: Jackey Lee > Limit push down for parquet datasource v2 >

[jira] [Assigned] (SPARK-37896) ConstantColumnVector: a column vector with same values

[ https://issues.apache.org/jira/browse/SPARK-37896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-37896: --- Assignee: Yaohua Zhao > ConstantColumnVector: a column vector with same values >

[jira] [Resolved] (SPARK-37896) ConstantColumnVector: a column vector with same values

[ https://issues.apache.org/jira/browse/SPARK-37896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-37896. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35068

[jira] [Assigned] (SPARK-37769) Filter on the metadata struct

[ https://issues.apache.org/jira/browse/SPARK-37769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-37769: --- Assignee: Yaohua Zhao > Filter on the metadata struct > - > >

[jira] [Resolved] (SPARK-37769) Filter on the metadata struct

[ https://issues.apache.org/jira/browse/SPARK-37769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-37769. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35055

[jira] [Assigned] (SPARK-37917) Push down limit 1 for right side of left semi/anti join

[ https://issues.apache.org/jira/browse/SPARK-37917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-37917: --- Assignee: Yuming Wang > Push down limit 1 for right side of left semi/anti join >

[jira] [Resolved] (SPARK-37917) Push down limit 1 for right side of left semi/anti join

[ https://issues.apache.org/jira/browse/SPARK-37917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-37917. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35216

[jira] [Resolved] (SPARK-37580) Optimize current TaskSetManager abort logic when task failed count reach the threshold

[ https://issues.apache.org/jira/browse/SPARK-37580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] wuyi resolved SPARK-37580. -- Assignee: wangshengjie Resolution: Fixed Issue resolved by https://github.com/apache/spark/pull/34834

[jira] [Assigned] (SPARK-37094) Inline type hints for files in python/pyspark

[ https://issues.apache.org/jira/browse/SPARK-37094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-37094: Assignee: dch nguyen > Inline type hints for files in python/pyspark >

[jira] [Assigned] (SPARK-37093) Inline type hints python/pyspark/streaming

[ https://issues.apache.org/jira/browse/SPARK-37093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-37093: Assignee: dch nguyen > Inline type hints python/pyspark/streaming >

[jira] [Resolved] (SPARK-37850) Enable flake's E731 rule in PySpark

[ https://issues.apache.org/jira/browse/SPARK-37850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-37850. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35150

[jira] [Assigned] (SPARK-37850) Enable flake's E731 rule in PySpark

[ https://issues.apache.org/jira/browse/SPARK-37850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-37850: Assignee: Hyukjin Kwon > Enable flake's E731 rule in PySpark >

[jira] [Commented] (SPARK-37956) Add Java and Python examples to the Parquet encryption feature documentation

[ https://issues.apache.org/jira/browse/SPARK-37956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17478265#comment-17478265 ] Apache Spark commented on SPARK-37956: -- User 'andersonm-ibm' has created a pull request for this

[jira] [Assigned] (SPARK-37956) Add Java and Python examples to the Parquet encryption feature documentation

[ https://issues.apache.org/jira/browse/SPARK-37956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37956: Assignee: Apache Spark > Add Java and Python examples to the Parquet encryption feature

[jira] [Assigned] (SPARK-37956) Add Java and Python examples to the Parquet encryption feature documentation

[ https://issues.apache.org/jira/browse/SPARK-37956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37956: Assignee: (was: Apache Spark) > Add Java and Python examples to the Parquet

[jira] [Commented] (SPARK-37957) Deterministic flag is not handled for V2 functions

[ https://issues.apache.org/jira/browse/SPARK-37957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17478234#comment-17478234 ] Apache Spark commented on SPARK-37957: -- User 'sunchao' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37957) Deterministic flag is not handled for V2 functions

[ https://issues.apache.org/jira/browse/SPARK-37957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37957: Assignee: Apache Spark (was: Chao Sun) > Deterministic flag is not handled for V2

[jira] [Assigned] (SPARK-37957) Deterministic flag is not handled for V2 functions

[ https://issues.apache.org/jira/browse/SPARK-37957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37957: Assignee: Chao Sun (was: Apache Spark) > Deterministic flag is not handled for V2

[jira] [Updated] (SPARK-37872) [SQL] Some classes are move from org.codehaus.janino:janino to org.codehaus.janino:common-compiler after version 3.1.x

[ https://issues.apache.org/jira/browse/SPARK-37872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jin Shen updated SPARK-37872: - Issue Type: Improvement (was: Bug) > [SQL] Some classes are move from org.codehaus.janino:janino to >

[jira] [Created] (SPARK-37958) Pyspark SparkContext.AddFile() does not respect spark.files.overwrite

taylor schneider created SPARK-37958: Summary: Pyspark SparkContext.AddFile() does not respect spark.files.overwrite Key: SPARK-37958 URL: https://issues.apache.org/jira/browse/SPARK-37958

[jira] [Commented] (SPARK-37912) Add benchmark for delta binary encoding

[ https://issues.apache.org/jira/browse/SPARK-37912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17478152#comment-17478152 ] Parth Chandra commented on SPARK-37912: --- Superseded by SPARK-37928 > Add benchmark for delta

[jira] [Created] (SPARK-37957) Deterministic flag is not handled for V2 functions

Chao Sun created SPARK-37957: Summary: Deterministic flag is not handled for V2 functions Key: SPARK-37957 URL: https://issues.apache.org/jira/browse/SPARK-37957 Project: Spark Issue Type:

[jira] [Created] (SPARK-37956) Add Java and Python examples to the Parquet encryption feature documentation

Maya Anderson created SPARK-37956: - Summary: Add Java and Python examples to the Parquet encryption feature documentation Key: SPARK-37956 URL: https://issues.apache.org/jira/browse/SPARK-37956

[jira] [Commented] (SPARK-37955) PartitioningAwareFileIndex->basePath incorrectly contains the partition filters

[ https://issues.apache.org/jira/browse/SPARK-37955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17478032#comment-17478032 ] Andreas Chatzistergiou commented on SPARK-37955: I am working on a PR about this. I will

[jira] [Created] (SPARK-37955) PartitioningAwareFileIndex->basePath incorrectly contains the partition filters

Andreas Chatzistergiou created SPARK-37955: -- Summary: PartitioningAwareFileIndex->basePath incorrectly contains the partition filters Key: SPARK-37955 URL:

[jira] [Created] (SPARK-37954) old columns should not be available after select or drop

Jean Bon created SPARK-37954: Summary: old columns should not be available after select or drop Key: SPARK-37954 URL: https://issues.apache.org/jira/browse/SPARK-37954 Project: Spark Issue Type:

[jira] [Commented] (SPARK-37942) Use error classes in the compilation errors of properties

[ https://issues.apache.org/jira/browse/SPARK-37942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17477975#comment-17477975 ] roryqi commented on SPARK-37942: I will work on this. Thanks [~maxgekk] > Use error classes in the

[jira] [Commented] (SPARK-37933) Limit push down for parquet datasource v2

[ https://issues.apache.org/jira/browse/SPARK-37933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17477838#comment-17477838 ] Jackey Lee commented on SPARK-37933: [https://github.com/apache/spark/pull/35242] This pr is only

[jira] [Commented] (SPARK-37933) Limit push down for parquet datasource v2

[ https://issues.apache.org/jira/browse/SPARK-37933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17477807#comment-17477807 ] Apache Spark commented on SPARK-37933: -- User 'stczwd' has created a pull request for this issue:

[jira] [Commented] (SPARK-37933) Limit push down for parquet datasource v2

[ https://issues.apache.org/jira/browse/SPARK-37933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17477809#comment-17477809 ] Apache Spark commented on SPARK-37933: -- User 'stczwd' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37933) Limit push down for parquet datasource v2

[ https://issues.apache.org/jira/browse/SPARK-37933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37933: Assignee: (was: Apache Spark) > Limit push down for parquet datasource v2 >

[jira] [Assigned] (SPARK-37933) Limit push down for parquet datasource v2

[ https://issues.apache.org/jira/browse/SPARK-37933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37933: Assignee: Apache Spark > Limit push down for parquet datasource v2 >

[jira] [Assigned] (SPARK-37953) Improve the implement of UnevaluableAggregate

[ https://issues.apache.org/jira/browse/SPARK-37953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37953: Assignee: (was: Apache Spark) > Improve the implement of UnevaluableAggregate >

[jira] [Assigned] (SPARK-37953) Improve the implement of UnevaluableAggregate

[ https://issues.apache.org/jira/browse/SPARK-37953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37953: Assignee: Apache Spark > Improve the implement of UnevaluableAggregate >

[jira] [Commented] (SPARK-37953) Improve the implement of UnevaluableAggregate

[ https://issues.apache.org/jira/browse/SPARK-37953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17477741#comment-17477741 ] Apache Spark commented on SPARK-37953: -- User 'beliefer' has created a pull request for this issue:

[jira] [Updated] (SPARK-37953) Improve the implement of UnevaluableAggregate

[ https://issues.apache.org/jira/browse/SPARK-37953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-37953: --- Description: Currently, Spark provides UnevaluableAggregate to replace function with another

[jira] [Updated] (SPARK-37953) Improve the implement of UnevaluableAggregate

[ https://issues.apache.org/jira/browse/SPARK-37953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-37953: --- Description: Currently, Spark provides RuntimeReplaceable to replace function with another

[jira] [Created] (SPARK-37953) Improve the implement of UnevaluableAggregate

jiaan.geng created SPARK-37953: -- Summary: Improve the implement of UnevaluableAggregate Key: SPARK-37953 URL: https://issues.apache.org/jira/browse/SPARK-37953 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-37930) Fix DataFrame select subset with duplicated columns

[ https://issues.apache.org/jira/browse/SPARK-37930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37930: Assignee: Apache Spark > Fix DataFrame select subset with duplicated columns >

[jira] [Assigned] (SPARK-37930) Fix DataFrame select subset with duplicated columns

[ https://issues.apache.org/jira/browse/SPARK-37930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37930: Assignee: (was: Apache Spark) > Fix DataFrame select subset with duplicated columns

[jira] [Commented] (SPARK-37930) Fix DataFrame select subset with duplicated columns

[ https://issues.apache.org/jira/browse/SPARK-37930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17477702#comment-17477702 ] Apache Spark commented on SPARK-37930: -- User 'dchvn' has created a pull request for this issue:

[jira] [Commented] (SPARK-37930) Fix DataFrame select subset with duplicated columns

[ https://issues.apache.org/jira/browse/SPARK-37930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17477700#comment-17477700 ] Apache Spark commented on SPARK-37930: -- User 'dchvn' has created a pull request for this issue:

[jira] [Commented] (SPARK-37952) Add missing statements to ALTER TABLE document.

[ https://issues.apache.org/jira/browse/SPARK-37952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17477674#comment-17477674 ] Apache Spark commented on SPARK-37952: -- User 'yutoacts' has created a pull request for this issue:

[jira] [Commented] (SPARK-37952) Add missing statements to ALTER TABLE document.

[ https://issues.apache.org/jira/browse/SPARK-37952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17477672#comment-17477672 ] Apache Spark commented on SPARK-37952: -- User 'yutoacts' has created a pull request for this issue:

[jira] [Assigned] (SPARK-37952) Add missing statements to ALTER TABLE document.

[ https://issues.apache.org/jira/browse/SPARK-37952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37952: Assignee: Apache Spark > Add missing statements to ALTER TABLE document. >

[jira] [Assigned] (SPARK-37952) Add missing statements to ALTER TABLE document.

[ https://issues.apache.org/jira/browse/SPARK-37952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37952: Assignee: (was: Apache Spark) > Add missing statements to ALTER TABLE document. >

[jira] [Commented] (SPARK-36649) Support Trigger.AvailableNow on Kafka data source

[ https://issues.apache.org/jira/browse/SPARK-36649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17477654#comment-17477654 ] Apache Spark commented on SPARK-36649: -- User 'jerrypeng' has created a pull request for this issue:

[jira] [Assigned] (SPARK-36649) Support Trigger.AvailableNow on Kafka data source

[ https://issues.apache.org/jira/browse/SPARK-36649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36649: Assignee: (was: Apache Spark) > Support Trigger.AvailableNow on Kafka data source >

[jira] [Assigned] (SPARK-36649) Support Trigger.AvailableNow on Kafka data source

[ https://issues.apache.org/jira/browse/SPARK-36649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36649: Assignee: Apache Spark > Support Trigger.AvailableNow on Kafka data source >

[jira] [Commented] (SPARK-36649) Support Trigger.AvailableNow on Kafka data source

[ https://issues.apache.org/jira/browse/SPARK-36649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17477652#comment-17477652 ] Apache Spark commented on SPARK-36649: -- User 'jerrypeng' has created a pull request for this issue:

[jira] [Resolved] (SPARK-37949) Improve Rebalance statistics estimation

[ https://issues.apache.org/jira/browse/SPARK-37949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-37949. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35235

[jira] [Assigned] (SPARK-37949) Improve Rebalance statistics estimation

[ https://issues.apache.org/jira/browse/SPARK-37949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-37949: --- Assignee: XiDuo You > Improve Rebalance statistics estimation >

[jira] [Commented] (SPARK-37943) Use error classes in the compilation errors of grouping

[ https://issues.apache.org/jira/browse/SPARK-37943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17477650#comment-17477650 ] Saurabh Chawla commented on SPARK-37943: I can work on this [~maxgekk]  > Use error classes in

[jira] [Created] (SPARK-37952) Add missing statements to ALTER TABLE document.

Yuto Akutsu created SPARK-37952: --- Summary: Add missing statements to ALTER TABLE document. Key: SPARK-37952 URL: https://issues.apache.org/jira/browse/SPARK-37952 Project: Spark Issue Type: