[jira] [Updated] (SPARK-37287) Pull out dynamic partition and bucket sort from FileFormatWriter

[ https://issues.apache.org/jira/browse/SPARK-37287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiDuo You updated SPARK-37287: -- Description: `FileFormatWriter.write` now is used by all V1 write which includes datasource and hive

[jira] [Updated] (SPARK-37300) TaskSchedulerImpl should ignore task finished event if its task was already finished state

[ https://issues.apache.org/jira/browse/SPARK-37300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hujiahua updated SPARK-37300: - Description: `TaskSchedulerImpl` handle task finished event at `handleSuccessfulTask` and `handleFailed

[jira] [Updated] (SPARK-37300) TaskSchedulerImpl should ignore task finished event if its task was already finished state

[ https://issues.apache.org/jira/browse/SPARK-37300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hujiahua updated SPARK-37300: - Description: `TaskSchedulerImpl` handle task finished event at `handleSuccessfulTask` and `handleFailed

[jira] [Updated] (SPARK-37300) TaskSchedulerImpl should ignore task finished event if its task was already finished state

[ https://issues.apache.org/jira/browse/SPARK-37300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hujiahua updated SPARK-37300: - Description: `TaskSchedulerImpl` handle task finished event at `handleSuccessfulTask` and `handleFailed

[jira] [Updated] (SPARK-37300) TaskSchedulerImpl should ignore task finished event if its task was already finished state

[ https://issues.apache.org/jira/browse/SPARK-37300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hujiahua updated SPARK-37300: - Description: TaskSchedulerImpl in some case may handle task `handleSuccessfulTask` and `handleFailedTas

[jira] [Updated] (SPARK-37300) TaskSchedulerImpl should ignore task finished event if its task was already finished state

[ https://issues.apache.org/jira/browse/SPARK-37300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hujiahua updated SPARK-37300: - Description: TaskSchedulerImpl in some case may When a executor finished a task of some stage, the driv

[jira] [Updated] (SPARK-37300) TaskSchedulerImpl should ignore task finished event if its task was already finished state

[ https://issues.apache.org/jira/browse/SPARK-37300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hujiahua updated SPARK-37300: - Description: When a executor finished a task of some stage, the driver will receive a StatusUpdate event

[jira] [Created] (SPARK-37300) TaskSchedulerImpl should ignore task finished event if its task was already finished state

hujiahua created SPARK-37300: Summary: TaskSchedulerImpl should ignore task finished event if its task was already finished state Key: SPARK-37300 URL: https://issues.apache.org/jira/browse/SPARK-37300 Pr

[jira] [Commented] (SPARK-37298) Use unique exprId in RewriteAsOfJoin

[ https://issues.apache.org/jira/browse/SPARK-37298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17442583#comment-17442583 ] Apache Spark commented on SPARK-37298: -- User 'allisonwang-db' has created a pull re

[jira] [Commented] (SPARK-37298) Use unique exprId in RewriteAsOfJoin

[ https://issues.apache.org/jira/browse/SPARK-37298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17442582#comment-17442582 ] Apache Spark commented on SPARK-37298: -- User 'allisonwang-db' has created a pull re

[jira] [Assigned] (SPARK-37298) Use unique exprId in RewriteAsOfJoin

[ https://issues.apache.org/jira/browse/SPARK-37298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37298: Assignee: Apache Spark > Use unique exprId in RewriteAsOfJoin > -

[jira] [Assigned] (SPARK-37298) Use unique exprId in RewriteAsOfJoin

[ https://issues.apache.org/jira/browse/SPARK-37298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37298: Assignee: (was: Apache Spark) > Use unique exprId in RewriteAsOfJoin > --

[jira] [Created] (SPARK-37299) Fix Python linter failure in branch-3.1

Haejoon Lee created SPARK-37299: --- Summary: Fix Python linter failure in branch-3.1 Key: SPARK-37299 URL: https://issues.apache.org/jira/browse/SPARK-37299 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-37298) Use unique exprId in RewriteAsOfJoin

Allison Wang created SPARK-37298: Summary: Use unique exprId in RewriteAsOfJoin Key: SPARK-37298 URL: https://issues.apache.org/jira/browse/SPARK-37298 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-37292) Removes outer join if it only has DISTINCT on streamed side with alias

[ https://issues.apache.org/jira/browse/SPARK-37292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-37292: - Assignee: Yuming Wang > Removes outer join if it only has DISTINCT on streamed side wit

[jira] [Resolved] (SPARK-37292) Removes outer join if it only has DISTINCT on streamed side with alias

[ https://issues.apache.org/jira/browse/SPARK-37292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-37292. --- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34557 [https://

[jira] [Commented] (SPARK-36825) Read/write dataframes with ANSI intervals from/to parquet files

[ https://issues.apache.org/jira/browse/SPARK-36825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17442571#comment-17442571 ] Apache Spark commented on SPARK-36825: -- User 'sarutak' has created a pull request f

[jira] [Commented] (SPARK-36825) Read/write dataframes with ANSI intervals from/to parquet files

[ https://issues.apache.org/jira/browse/SPARK-36825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17442572#comment-17442572 ] Apache Spark commented on SPARK-36825: -- User 'sarutak' has created a pull request f

[jira] [Comment Edited] (SPARK-15428) Disable support for multiple streaming aggregations

[ https://issues.apache.org/jira/browse/SPARK-15428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17442292#comment-17442292 ] Hongbo edited comment on SPARK-15428 at 11/12/21, 3:56 AM: --- Is

[jira] [Assigned] (SPARK-37296) Add missing type hints in python/pyspark/util.py

[ https://issues.apache.org/jira/browse/SPARK-37296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-37296: Assignee: Takuya Ueshin > Add missing type hints in python/pyspark/util.py >

[jira] [Resolved] (SPARK-37296) Add missing type hints in python/pyspark/util.py

[ https://issues.apache.org/jira/browse/SPARK-37296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-37296. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34563 [https://gi

[jira] [Created] (SPARK-37297) Capture SQL statement executed at source in spark debug log

Sivakumar Ramaswamy created SPARK-37297: --- Summary: Capture SQL statement executed at source in spark debug log Key: SPARK-37297 URL: https://issues.apache.org/jira/browse/SPARK-37297 Project: Sp

[jira] [Resolved] (SPARK-37293) Remove explicit GC options from Scala tests

[ https://issues.apache.org/jira/browse/SPARK-37293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-37293. --- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34560 [https://

[jira] [Commented] (SPARK-37296) Add missing type hints in python/pyspark/util.py

[ https://issues.apache.org/jira/browse/SPARK-37296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17442544#comment-17442544 ] Apache Spark commented on SPARK-37296: -- User 'ueshin' has created a pull request fo

[jira] [Commented] (SPARK-37022) Use black as a formatter for the whole PySpark codebase.

[ https://issues.apache.org/jira/browse/SPARK-37022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17442541#comment-17442541 ] Apache Spark commented on SPARK-37022: -- User 'itholic' has created a pull request f

[jira] [Assigned] (SPARK-37296) Add missing type hints in python/pyspark/util.py

[ https://issues.apache.org/jira/browse/SPARK-37296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37296: Assignee: (was: Apache Spark) > Add missing type hints in python/pyspark/util.py > --

[jira] [Assigned] (SPARK-37296) Add missing type hints in python/pyspark/util.py

[ https://issues.apache.org/jira/browse/SPARK-37296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37296: Assignee: Apache Spark > Add missing type hints in python/pyspark/util.py > -

[jira] [Commented] (SPARK-37296) Add missing type hints in python/pyspark/util.py

[ https://issues.apache.org/jira/browse/SPARK-37296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17442540#comment-17442540 ] Apache Spark commented on SPARK-37296: -- User 'ueshin' has created a pull request fo

[jira] [Updated] (SPARK-37287) Pull out dynamic partition and bucket sort from FileFormatWriter

[ https://issues.apache.org/jira/browse/SPARK-37287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiDuo You updated SPARK-37287: -- Description: FileFormatWriter.write now is used by all V1 write which includes datasource and hive ta

[jira] [Updated] (SPARK-37287) Pull out dynamic partition and bucket sort from FileFormatWriter

[ https://issues.apache.org/jira/browse/SPARK-37287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] XiDuo You updated SPARK-37287: -- Description: FileFormatWriter.write now is used by all V1 write which includes datasource and hive ta

[jira] [Created] (SPARK-37296) Add missing type hints in python/pyspark/util.py

Takuya Ueshin created SPARK-37296: - Summary: Add missing type hints in python/pyspark/util.py Key: SPARK-37296 URL: https://issues.apache.org/jira/browse/SPARK-37296 Project: Spark Issue Type

[jira] [Assigned] (SPARK-37263) Add PandasAPIOnSparkAdviceWarning class

[ https://issues.apache.org/jira/browse/SPARK-37263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-37263: Assignee: Haejoon Lee > Add PandasAPIOnSparkAdviceWarning class > ---

[jira] [Resolved] (SPARK-37263) Add PandasAPIOnSparkAdviceWarning class

[ https://issues.apache.org/jira/browse/SPARK-37263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-37263. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34550 [https://gi

[jira] [Updated] (SPARK-37268) Remove unused method call in FileScanRDD

[ https://issues.apache.org/jira/browse/SPARK-37268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-37268: -- Priority: Trivial (was: Major) > Remove unused method call in FileScanRDD > -

[jira] [Updated] (SPARK-37268) Remove unused method call in FileScanRDD

[ https://issues.apache.org/jira/browse/SPARK-37268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-37268: -- Affects Version/s: 3.3.0 (was: 3.2.0) > Remove unused method call i

[jira] [Resolved] (SPARK-37268) Remove unused method call in FileScanRDD

[ https://issues.apache.org/jira/browse/SPARK-37268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-37268. --- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34545 [https://

[jira] [Assigned] (SPARK-37268) Remove unused method call in FileScanRDD

[ https://issues.apache.org/jira/browse/SPARK-37268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-37268: - Assignee: Junfan Zhang > Remove unused method call in FileScanRDD > ---

[jira] [Resolved] (SPARK-35011) Avoid Block Manager registerations when StopExecutor msg is in-flight.

[ https://issues.apache.org/jira/browse/SPARK-35011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-35011. --- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34536 [https://

[jira] [Assigned] (SPARK-35011) Avoid Block Manager registerations when StopExecutor msg is in-flight.

[ https://issues.apache.org/jira/browse/SPARK-35011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-35011: - Assignee: wuyi > Avoid Block Manager registerations when StopExecutor msg is in-flight.

[jira] [Updated] (SPARK-36845) Inline type hint files for files in python/pyspark/sql

[ https://issues.apache.org/jira/browse/SPARK-36845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-36845: -- Summary: Inline type hint files for files in python/pyspark/sql (was: Inline type hint files)

[jira] [Resolved] (SPARK-37284) Upgrade Jekyll to 4.2.1

[ https://issues.apache.org/jira/browse/SPARK-37284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-37284. --- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34552 [https://

[jira] [Created] (SPARK-37295) illegal reflective access operation has occurred; Please consider reporting this to the maintainers

Andrew Davidson created SPARK-37295: --- Summary: illegal reflective access operation has occurred; Please consider reporting this to the maintainers Key: SPARK-37295 URL: https://issues.apache.org/jira/browse/SPAR

[jira] [Updated] (SPARK-37120) Add Daily GitHub Action jobs for Java11/17

[ https://issues.apache.org/jira/browse/SPARK-37120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-37120: -- Summary: Add Daily GitHub Action jobs for Java11/17 (was: Add Java17 GitHub Action build and

[jira] [Commented] (SPARK-37270) Incorect result of filter using isNull condition

[ https://issues.apache.org/jira/browse/SPARK-37270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17442433#comment-17442433 ] Bruce Robbins commented on SPARK-37270: --- I can reproduce locally. In 3.1, the abov

[jira] [Assigned] (SPARK-37294) Check inserting of ANSI intervals into a table partitioned by the interval columns

[ https://issues.apache.org/jira/browse/SPARK-37294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37294: Assignee: Max Gekk (was: Apache Spark) > Check inserting of ANSI intervals into a table

[jira] [Assigned] (SPARK-37294) Check inserting of ANSI intervals into a table partitioned by the interval columns

[ https://issues.apache.org/jira/browse/SPARK-37294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37294: Assignee: Apache Spark (was: Max Gekk) > Check inserting of ANSI intervals into a table

[jira] [Commented] (SPARK-37294) Check inserting of ANSI intervals into a table partitioned by the interval columns

[ https://issues.apache.org/jira/browse/SPARK-37294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17442408#comment-17442408 ] Apache Spark commented on SPARK-37294: -- User 'MaxGekk' has created a pull request f

[jira] [Created] (SPARK-37294) Check inserting of ANSI intervals into a table partitioned by the interval columns

Max Gekk created SPARK-37294: Summary: Check inserting of ANSI intervals into a table partitioned by the interval columns Key: SPARK-37294 URL: https://issues.apache.org/jira/browse/SPARK-37294 Project: S

[jira] [Updated] (SPARK-37290) Exponential planning time in case of non-deterministic function

[ https://issues.apache.org/jira/browse/SPARK-37290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kaya Kupferschmidt updated SPARK-37290: --- Description: We are experiencing an exponential growth of processing time in case of

[jira] [Updated] (SPARK-37232) Upgrade ORC to 1.7.1

[ https://issues.apache.org/jira/browse/SPARK-37232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-37232: -- Parent: SPARK-33772 Issue Type: Sub-task (was: Bug) > Upgrade ORC to 1.7.1 >

[jira] [Commented] (SPARK-33772) Build and Run Spark on Java 17

[ https://issues.apache.org/jira/browse/SPARK-33772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17442368#comment-17442368 ] Dongjoon Hyun commented on SPARK-33772: --- Hi, All. This is almost done and Apache S

[jira] [Closed] (SPARK-37265) Support Java 17 in `dev/test-dependencies.sh`

[ https://issues.apache.org/jira/browse/SPARK-37265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-37265. - > Support Java 17 in `dev/test-dependencies.sh` > - > >

[jira] [Assigned] (SPARK-37293) Remove explicit GC options from Scala tests

[ https://issues.apache.org/jira/browse/SPARK-37293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-37293: - Assignee: Dongjoon Hyun > Remove explicit GC options from Scala tests > ---

[jira] [Resolved] (SPARK-33772) Build and Run Spark on Java 17

[ https://issues.apache.org/jira/browse/SPARK-33772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-33772. --- Fix Version/s: 3.3.0 Target Version/s: 3.3.0 Resolution: Fixed > Build and

[jira] [Assigned] (SPARK-33772) Build and Run Spark on Java 17

[ https://issues.apache.org/jira/browse/SPARK-33772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-33772: - Assignee: Yang Jie > Build and Run Spark on Java 17 > -- >

[jira] [Updated] (SPARK-35496) Upgrade Scala 2.13 to 2.13.7

[ https://issues.apache.org/jira/browse/SPARK-35496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-35496: -- Parent: SPARK-33772 Issue Type: Sub-task (was: Task) > Upgrade Scala 2.13 to 2.13.7 >

[jira] [Assigned] (SPARK-37293) Remove explicit GC options from Scala tests

[ https://issues.apache.org/jira/browse/SPARK-37293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37293: Assignee: (was: Apache Spark) > Remove explicit GC options from Scala tests > ---

[jira] [Assigned] (SPARK-37293) Remove explicit GC options from Scala tests

[ https://issues.apache.org/jira/browse/SPARK-37293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37293: Assignee: Apache Spark > Remove explicit GC options from Scala tests > --

[jira] [Commented] (SPARK-37293) Remove explicit GC options from Scala tests

[ https://issues.apache.org/jira/browse/SPARK-37293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17442358#comment-17442358 ] Apache Spark commented on SPARK-37293: -- User 'dongjoon-hyun' has created a pull req

[jira] [Closed] (SPARK-35557) Adapt uses of JDK 17 Internal APIs

[ https://issues.apache.org/jira/browse/SPARK-35557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-35557. - > Adapt uses of JDK 17 Internal APIs > -- > > Key: S

[jira] [Updated] (SPARK-37293) Remove explicit GC options from Scala tests

[ https://issues.apache.org/jira/browse/SPARK-37293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-37293: -- Description: At Spark 3.0, SPARK-29282 introduced the explicit GC options in Scala tests to s

[jira] [Updated] (SPARK-37293) Remove explicit GC options from Scala tests

[ https://issues.apache.org/jira/browse/SPARK-37293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-37293: -- Description: At Spark 3.0, SPARK-29282 introduced the explicit GC options in Scala tests to s

[jira] [Created] (SPARK-37293) Remove explicit GC options from Scala tests

Dongjoon Hyun created SPARK-37293: - Summary: Remove explicit GC options from Scala tests Key: SPARK-37293 URL: https://issues.apache.org/jira/browse/SPARK-37293 Project: Spark Issue Type: Sub

[jira] [Resolved] (SPARK-37267) OptimizeSkewInRebalancePartitions support optimize non-root node

[ https://issues.apache.org/jira/browse/SPARK-37267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-37267. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34542 [https://gith

[jira] [Assigned] (SPARK-37267) OptimizeSkewInRebalancePartitions support optimize non-root node

[ https://issues.apache.org/jira/browse/SPARK-37267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-37267: --- Assignee: XiDuo You > OptimizeSkewInRebalancePartitions support optimize non-root node > --

[jira] [Assigned] (SPARK-37291) PySpark SparkSession.config should respect enableHiveSupport

[ https://issues.apache.org/jira/browse/SPARK-37291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37291: Assignee: (was: Apache Spark) > PySpark SparkSession.config should respect enableHive

[jira] [Commented] (SPARK-37291) PySpark SparkSession.config should respect enableHiveSupport

[ https://issues.apache.org/jira/browse/SPARK-37291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17442303#comment-17442303 ] Apache Spark commented on SPARK-37291: -- User 'AngersZh' has created a pull requ

[jira] [Assigned] (SPARK-37291) PySpark SparkSession.config should respect enableHiveSupport

[ https://issues.apache.org/jira/browse/SPARK-37291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37291: Assignee: Apache Spark > PySpark SparkSession.config should respect enableHiveSupport > -

[jira] [Commented] (SPARK-15428) Disable support for multiple streaming aggregations

[ https://issues.apache.org/jira/browse/SPARK-15428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17442292#comment-17442292 ] Hongbo commented on SPARK-15428: Is there any plan to enable it? It's quite a big limita

[jira] [Commented] (SPARK-37292) Removes outer join if it only has DISTINCT on streamed side with alias

[ https://issues.apache.org/jira/browse/SPARK-37292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17442288#comment-17442288 ] Apache Spark commented on SPARK-37292: -- User 'wangyum' has created a pull request f

[jira] [Commented] (SPARK-37289) Refactoring:remove the unnecessary function with partitionSchemaOption

[ https://issues.apache.org/jira/browse/SPARK-37289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17442287#comment-17442287 ] Apache Spark commented on SPARK-37289: -- User 'tenglei' has created a pull request f

[jira] [Assigned] (SPARK-37289) Refactoring:remove the unnecessary function with partitionSchemaOption

[ https://issues.apache.org/jira/browse/SPARK-37289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37289: Assignee: Apache Spark > Refactoring:remove the unnecessary function with partitionSchema

[jira] [Assigned] (SPARK-37292) Removes outer join if it only has DISTINCT on streamed side with alias

[ https://issues.apache.org/jira/browse/SPARK-37292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37292: Assignee: Apache Spark > Removes outer join if it only has DISTINCT on streamed side with

[jira] [Assigned] (SPARK-37289) Refactoring:remove the unnecessary function with partitionSchemaOption

[ https://issues.apache.org/jira/browse/SPARK-37289?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37289: Assignee: (was: Apache Spark) > Refactoring:remove the unnecessary function with part

[jira] [Commented] (SPARK-37019) Add Codegen support to array higher-order-functions

[ https://issues.apache.org/jira/browse/SPARK-37019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17442286#comment-17442286 ] Apache Spark commented on SPARK-37019: -- User 'Kimahriman' has created a pull reques

[jira] [Assigned] (SPARK-37292) Removes outer join if it only has DISTINCT on streamed side with alias

[ https://issues.apache.org/jira/browse/SPARK-37292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37292: Assignee: (was: Apache Spark) > Removes outer join if it only has DISTINCT on streame

[jira] [Updated] (SPARK-37019) Add Codegen support to array higher-order-functions

[ https://issues.apache.org/jira/browse/SPARK-37019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Binford updated SPARK-37019: - Description: Currently all of the higher order functions use CodegenFallback. We can improve th

[jira] [Updated] (SPARK-37292) Removes outer join if it only has DISTINCT on streamed side with alias

[ https://issues.apache.org/jira/browse/SPARK-37292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-37292: Description: {code:scala} spark.range(200L).selectExpr("id AS a").createTempView("t1") spa

[jira] [Created] (SPARK-37292) Removes outer join if it only has DISTINCT on streamed side with alias

Yuming Wang created SPARK-37292: --- Summary: Removes outer join if it only has DISTINCT on streamed side with alias Key: SPARK-37292 URL: https://issues.apache.org/jira/browse/SPARK-37292 Project: Spark

[jira] [Created] (SPARK-37291) PySpark SparkSession.config should respect enableHiveSupport

angerszhu created SPARK-37291: - Summary: PySpark SparkSession.config should respect enableHiveSupport Key: SPARK-37291 URL: https://issues.apache.org/jira/browse/SPARK-37291 Project: Spark Issue

[jira] [Updated] (SPARK-37286) Move compileFilter and compileAggregates from JDBCRDD to JdbcDialect

[ https://issues.apache.org/jira/browse/SPARK-37286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-37286: --- Description: Currently, the method compileFilter and compileAggregates in JDBCRDD. But it is not rea

[jira] [Updated] (SPARK-37286) Move compileFilter and compileAggregates from JDBCRDD to JdbcDialect

[ https://issues.apache.org/jira/browse/SPARK-37286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jiaan.geng updated SPARK-37286: --- Summary: Move compileFilter and compileAggregates from JDBCRDD to JdbcDialect (was: Move compileAgg

[jira] [Commented] (SPARK-37258) Add Volcano support in kubernetes-client

[ https://issues.apache.org/jira/browse/SPARK-37258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17442249#comment-17442249 ] Yikun Jiang commented on SPARK-37258: - https://github.com/fabric8io/kubernetes-clien

[jira] [Updated] (SPARK-37290) Exponential planning time in case of non-deterministic function

[ https://issues.apache.org/jira/browse/SPARK-37290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kaya Kupferschmidt updated SPARK-37290: --- Description: We are experiencing an exponential growth of processing time in case of

[jira] [Created] (SPARK-37290) Exponential planning time in case of non-deterministic function

Kaya Kupferschmidt created SPARK-37290: -- Summary: Exponential planning time in case of non-deterministic function Key: SPARK-37290 URL: https://issues.apache.org/jira/browse/SPARK-37290 Project:

[jira] [Created] (SPARK-37289) Refactoring:remove the unnecessary function with partitionSchemaOption

tenglei created SPARK-37289: --- Summary: Refactoring:remove the unnecessary function with partitionSchemaOption Key: SPARK-37289 URL: https://issues.apache.org/jira/browse/SPARK-37289 Project: Spark

[jira] [Commented] (SPARK-37288) Backport update pyspark.since annotation to 3.1 and 3.2

[ https://issues.apache.org/jira/browse/SPARK-37288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17442208#comment-17442208 ] Apache Spark commented on SPARK-37288: -- User 'zero323' has created a pull request f

[jira] [Assigned] (SPARK-37288) Backport update pyspark.since annotation to 3.1 and 3.2

[ https://issues.apache.org/jira/browse/SPARK-37288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37288: Assignee: (was: Apache Spark) > Backport update pyspark.since annotation to 3.1 and 3

[jira] [Assigned] (SPARK-37288) Backport update pyspark.since annotation to 3.1 and 3.2

[ https://issues.apache.org/jira/browse/SPARK-37288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37288: Assignee: Apache Spark > Backport update pyspark.since annotation to 3.1 and 3.2 > --

[jira] [Commented] (SPARK-37288) Backport update pyspark.since annotation to 3.1 and 3.2

[ https://issues.apache.org/jira/browse/SPARK-37288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17442207#comment-17442207 ] Apache Spark commented on SPARK-37288: -- User 'zero323' has created a pull request f

[jira] [Updated] (SPARK-37288) Backport update pyspark.since annotation to 3.1 and 3.2

[ https://issues.apache.org/jira/browse/SPARK-37288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-37288: --- Summary: Backport update pyspark.since annotation to 3.1 and 3.2 (was: Backport upd

[jira] [Created] (SPARK-37288) Backport update pyspakr.since annotation to 3.1 and 3.2

Maciej Szymkiewicz created SPARK-37288: -- Summary: Backport update pyspakr.since annotation to 3.1 and 3.2 Key: SPARK-37288 URL: https://issues.apache.org/jira/browse/SPARK-37288 Project: Spark

[jira] [Commented] (SPARK-37286) Move compileAggregates from JDBCRDD to JdbcDialect

[ https://issues.apache.org/jira/browse/SPARK-37286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17442196#comment-17442196 ] Apache Spark commented on SPARK-37286: -- User 'beliefer' has created a pull request

[jira] [Created] (SPARK-37287) Pull out dynamic partition and bucket sort from FileFormatWriter

XiDuo You created SPARK-37287: - Summary: Pull out dynamic partition and bucket sort from FileFormatWriter Key: SPARK-37287 URL: https://issues.apache.org/jira/browse/SPARK-37287 Project: Spark I

[jira] [Assigned] (SPARK-37286) Move compileAggregates from JDBCRDD to JdbcDialect

[ https://issues.apache.org/jira/browse/SPARK-37286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37286: Assignee: (was: Apache Spark) > Move compileAggregates from JDBCRDD to JdbcDialect >

[jira] [Assigned] (SPARK-37286) Move compileAggregates from JDBCRDD to JdbcDialect

[ https://issues.apache.org/jira/browse/SPARK-37286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37286: Assignee: Apache Spark > Move compileAggregates from JDBCRDD to JdbcDialect > ---

[jira] [Created] (SPARK-37286) Move compileAggregates from JDBCRDD to JdbcDialect

jiaan.geng created SPARK-37286: -- Summary: Move compileAggregates from JDBCRDD to JdbcDialect Key: SPARK-37286 URL: https://issues.apache.org/jira/browse/SPARK-37286 Project: Spark Issue Type: Im

[jira] [Commented] (SPARK-37197) Behaviour inconsistency between pandas and pandas API on Spark

[ https://issues.apache.org/jira/browse/SPARK-37197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17442189#comment-17442189 ] Yikun Jiang commented on SPARK-37197: - [~hyukjin.kwon] Thanks for cc, I will try to

[jira] [Resolved] (SPARK-37262) Not log empty aggregate and group by in JDBCScan

[ https://issues.apache.org/jira/browse/SPARK-37262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-37262. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34540 [https://gith

[jira] [Assigned] (SPARK-37262) Not log empty aggregate and group by in JDBCScan

[ https://issues.apache.org/jira/browse/SPARK-37262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-37262: --- Assignee: Huaxin Gao > Not log empty aggregate and group by in JDBCScan > -

  1   2   >