[jira] [Created] (SPARK-37017) Reduce the scope of synchronized to prevent deadlock.

Zhixiong Chen created SPARK-37017: - Summary: Reduce the scope of synchronized to prevent deadlock. Key: SPARK-37017 URL: https://issues.apache.org/jira/browse/SPARK-37017 Project: Spark Issue

[jira] [Commented] (SPARK-37017) Reduce the scope of synchronized to prevent deadlock.

[ https://issues.apache.org/jira/browse/SPARK-37017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17429160#comment-17429160 ] Zhixiong Chen commented on SPARK-37017: --- I have created a pull request for this is

[jira] [Commented] (SPARK-37017) Reduce the scope of synchronized to prevent deadlock.

[ https://issues.apache.org/jira/browse/SPARK-37017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17429161#comment-17429161 ] Zhixiong Chen commented on SPARK-37017: --- I have created a pull request for this is

[jira] [Issue Comment Deleted] (SPARK-37017) Reduce the scope of synchronized to prevent deadlock.

[ https://issues.apache.org/jira/browse/SPARK-37017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhixiong Chen updated SPARK-37017: -- Comment: was deleted (was: I have created a pull request for this issue: [https://github.com/a

[jira] [Issue Comment Deleted] (SPARK-37017) Reduce the scope of synchronized to prevent deadlock.

[ https://issues.apache.org/jira/browse/SPARK-37017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhixiong Chen updated SPARK-37017: -- Comment: was deleted (was: I have created a pull request for this issue: [https://github.com/

[jira] [Assigned] (SPARK-37017) Reduce the scope of synchronized to prevent deadlock.

[ https://issues.apache.org/jira/browse/SPARK-37017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37017: Assignee: Apache Spark > Reduce the scope of synchronized to prevent deadlock. >

[jira] [Commented] (SPARK-37014) Inline type hints for python/pyspark/streaming/context.py

[ https://issues.apache.org/jira/browse/SPARK-37014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17429179#comment-17429179 ] Apache Spark commented on SPARK-37014: -- User 'dchvn' has created a pull request for

[jira] [Assigned] (SPARK-37017) Reduce the scope of synchronized to prevent deadlock.

[ https://issues.apache.org/jira/browse/SPARK-37017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37017: Assignee: (was: Apache Spark) > Reduce the scope of synchronized to prevent deadlock.

[jira] [Commented] (SPARK-37017) Reduce the scope of synchronized to prevent deadlock.

[ https://issues.apache.org/jira/browse/SPARK-37017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17429178#comment-17429178 ] Apache Spark commented on SPARK-37017: -- User 'chenzhx' has created a pull request f

[jira] [Assigned] (SPARK-37014) Inline type hints for python/pyspark/streaming/context.py

[ https://issues.apache.org/jira/browse/SPARK-37014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37014: Assignee: (was: Apache Spark) > Inline type hints for python/pyspark/streaming/contex

[jira] [Assigned] (SPARK-37014) Inline type hints for python/pyspark/streaming/context.py

[ https://issues.apache.org/jira/browse/SPARK-37014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37014: Assignee: Apache Spark > Inline type hints for python/pyspark/streaming/context.py >

[jira] [Commented] (SPARK-37017) Reduce the scope of synchronized to prevent deadlock.

[ https://issues.apache.org/jira/browse/SPARK-37017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17429162#comment-17429162 ] Zhixiong Chen commented on SPARK-37017: --- I have created a pull request for this is

[jira] [Updated] (SPARK-37016) Publicise UpperCaseCharStream

[ https://issues.apache.org/jira/browse/SPARK-37016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dohongdayi updated SPARK-37016: --- Description: Many Spark extension projects are copying `UpperCaseCharStream` because it is private

[jira] [Created] (SPARK-37018) Spark SQL should support create function with Aggregator

jiaan.geng created SPARK-37018: -- Summary: Spark SQL should support create function with Aggregator Key: SPARK-37018 URL: https://issues.apache.org/jira/browse/SPARK-37018 Project: Spark Issue Ty

[jira] [Commented] (SPARK-37018) Spark SQL should support create function with Aggregator

[ https://issues.apache.org/jira/browse/SPARK-37018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17429198#comment-17429198 ] jiaan.geng commented on SPARK-37018: I'm working. > Spark SQL should support create

[jira] [Resolved] (SPARK-36987) Add Doc about FROM statement

[ https://issues.apache.org/jira/browse/SPARK-36987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu resolved SPARK-36987. --- Resolution: Not A Problem > Add Doc about FROM statement > > >

[jira] [Created] (SPARK-37019) Add Codegen support to ArrayTransform

Adam Binford created SPARK-37019: Summary: Add Codegen support to ArrayTransform Key: SPARK-37019 URL: https://issues.apache.org/jira/browse/SPARK-37019 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-37019) Add Codegen support to ArrayTransform

[ https://issues.apache.org/jira/browse/SPARK-37019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37019: Assignee: Apache Spark > Add Codegen support to ArrayTransform >

[jira] [Assigned] (SPARK-37019) Add Codegen support to ArrayTransform

[ https://issues.apache.org/jira/browse/SPARK-37019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37019: Assignee: (was: Apache Spark) > Add Codegen support to ArrayTransform > -

[jira] [Commented] (SPARK-37019) Add Codegen support to ArrayTransform

[ https://issues.apache.org/jira/browse/SPARK-37019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17429254#comment-17429254 ] Apache Spark commented on SPARK-37019: -- User 'Kimahriman' has created a pull reques

[jira] [Assigned] (SPARK-35926) Support YearMonthIntervalType in width-bucket function

[ https://issues.apache.org/jira/browse/SPARK-35926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-35926: Assignee: PengLei > Support YearMonthIntervalType in width-bucket function >

[jira] [Resolved] (SPARK-35926) Support YearMonthIntervalType in width-bucket function

[ https://issues.apache.org/jira/browse/SPARK-35926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-35926. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 33132 [https://github.com

[jira] [Commented] (SPARK-36276) Update maven-checkstyle-plugin to 3.1.2 and checkstyle to 8.43

[ https://issues.apache.org/jira/browse/SPARK-36276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17429341#comment-17429341 ] Apache Spark commented on SPARK-36276: -- User 'sarutak' has created a pull request f

[jira] [Commented] (SPARK-36276) Update maven-checkstyle-plugin to 3.1.2 and checkstyle to 8.43

[ https://issues.apache.org/jira/browse/SPARK-36276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17429343#comment-17429343 ] Apache Spark commented on SPARK-36276: -- User 'sarutak' has created a pull request f

[jira] [Created] (SPARK-37020) Limit push down in DS V2

Huaxin Gao created SPARK-37020: -- Summary: Limit push down in DS V2 Key: SPARK-37020 URL: https://issues.apache.org/jira/browse/SPARK-37020 Project: Spark Issue Type: Sub-task Component

[jira] [Created] (SPARK-37021) JDBC option "sessionInitStatement" does not execute set sql statement when resolving a table

Valery Meleshkin created SPARK-37021: Summary: JDBC option "sessionInitStatement" does not execute set sql statement when resolving a table Key: SPARK-37021 URL: https://issues.apache.org/jira/browse/SPARK-370

[jira] [Updated] (SPARK-37021) JDBC option "sessionInitStatement" does not execute set sql statement when resolving a table

[ https://issues.apache.org/jira/browse/SPARK-37021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Valery Meleshkin updated SPARK-37021: - Description: If {{sessionInitStatement}} is required to grant permissions or resolve an

[jira] [Commented] (SPARK-37020) Limit push down in DS V2

[ https://issues.apache.org/jira/browse/SPARK-37020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17429395#comment-17429395 ] Apache Spark commented on SPARK-37020: -- User 'huaxingao' has created a pull request

[jira] [Assigned] (SPARK-36989) Migrate type hint data tests

[ https://issues.apache.org/jira/browse/SPARK-36989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36989: Assignee: (was: Apache Spark) > Migrate type hint data tests > --

[jira] [Assigned] (SPARK-36989) Migrate type hint data tests

[ https://issues.apache.org/jira/browse/SPARK-36989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36989: Assignee: Apache Spark > Migrate type hint data tests > > >

[jira] [Commented] (SPARK-36989) Migrate type hint data tests

[ https://issues.apache.org/jira/browse/SPARK-36989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17429394#comment-17429394 ] Apache Spark commented on SPARK-36989: -- User 'zero323' has created a pull request f

[jira] [Assigned] (SPARK-37020) Limit push down in DS V2

[ https://issues.apache.org/jira/browse/SPARK-37020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37020: Assignee: Apache Spark > Limit push down in DS V2 > > >

[jira] [Commented] (SPARK-37020) Limit push down in DS V2

[ https://issues.apache.org/jira/browse/SPARK-37020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17429396#comment-17429396 ] Apache Spark commented on SPARK-37020: -- User 'huaxingao' has created a pull request

[jira] [Assigned] (SPARK-37020) Limit push down in DS V2

[ https://issues.apache.org/jira/browse/SPARK-37020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37020: Assignee: (was: Apache Spark) > Limit push down in DS V2 > >

[jira] [Resolved] (SPARK-36991) Inline type hints for spark/python/pyspark/sql/streaming.py

[ https://issues.apache.org/jira/browse/SPARK-36991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-36991. --- Fix Version/s: 3.3.0 Assignee: Xinrong Meng Resolution: Fixed Issue resolved

[jira] [Resolved] (SPARK-36910) Inline type hints for python/pyspark/sql/types.py

[ https://issues.apache.org/jira/browse/SPARK-36910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-36910. --- Fix Version/s: 3.3.0 Assignee: Xinrong Meng Resolution: Fixed Issue resolved

[jira] [Created] (SPARK-37022) Use black as a formatter for the whole PySpark codebase.

Maciej Szymkiewicz created SPARK-37022: -- Summary: Use black as a formatter for the whole PySpark codebase. Key: SPARK-37022 URL: https://issues.apache.org/jira/browse/SPARK-37022 Project: Spark

[jira] [Updated] (SPARK-37022) Use black as a formatter for the whole PySpark codebase.

[ https://issues.apache.org/jira/browse/SPARK-37022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-37022: --- Attachment: pyproject.toml > Use black as a formatter for the whole PySpark codebase

[jira] [Updated] (SPARK-37022) Use black as a formatter for the whole PySpark codebase.

[ https://issues.apache.org/jira/browse/SPARK-37022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-37022: --- Attachment: black-diff-stats.txt > Use black as a formatter for the whole PySpark co

[jira] [Commented] (SPARK-37022) Use black as a formatter for the whole PySpark codebase.

[ https://issues.apache.org/jira/browse/SPARK-37022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17429477#comment-17429477 ] Maciej Szymkiewicz commented on SPARK-37022: The attached files show git dif

[jira] [Updated] (SPARK-37022) Use black as a formatter for the whole PySpark codebase.

[ https://issues.apache.org/jira/browse/SPARK-37022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-37022: --- Attachment: (was: black-diff-stats.txt) > Use black as a formatter for the whole

[jira] [Updated] (SPARK-37022) Use black as a formatter for the whole PySpark codebase.

[ https://issues.apache.org/jira/browse/SPARK-37022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-37022: --- Attachment: black-diff-stats.txt > Use black as a formatter for the whole PySpark co

[jira] [Comment Edited] (SPARK-37022) Use black as a formatter for the whole PySpark codebase.

[ https://issues.apache.org/jira/browse/SPARK-37022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17429477#comment-17429477 ] Maciej Szymkiewicz edited comment on SPARK-37022 at 10/15/21, 8:45 PM: ---

[jira] [Commented] (SPARK-37022) Use black as a formatter for the whole PySpark codebase.

[ https://issues.apache.org/jira/browse/SPARK-37022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17429482#comment-17429482 ] Maciej Szymkiewicz commented on SPARK-37022: cc [~hyukjin.kwon] > Use black

[jira] [Commented] (SPARK-37022) Use black as a formatter for the whole PySpark codebase.

[ https://issues.apache.org/jira/browse/SPARK-37022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17429491#comment-17429491 ] Apache Spark commented on SPARK-37022: -- User 'zero323' has created a pull request f

[jira] [Assigned] (SPARK-37022) Use black as a formatter for the whole PySpark codebase.

[ https://issues.apache.org/jira/browse/SPARK-37022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37022: Assignee: Apache Spark > Use black as a formatter for the whole PySpark codebase. > -

[jira] [Assigned] (SPARK-37022) Use black as a formatter for the whole PySpark codebase.

[ https://issues.apache.org/jira/browse/SPARK-37022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37022: Assignee: (was: Apache Spark) > Use black as a formatter for the whole PySpark codeba

[jira] [Updated] (SPARK-37017) Reduce the scope of synchronized to prevent deadlock.

[ https://issues.apache.org/jira/browse/SPARK-37017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-37017: - Issue Type: Bug (was: Improvement) > Reduce the scope of synchronized to prevent deadlock. > --

[jira] [Assigned] (SPARK-34960) Aggregate (Min/Max/Count) push down for ORC

[ https://issues.apache.org/jira/browse/SPARK-34960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34960: Assignee: Apache Spark > Aggregate (Min/Max/Count) push down for ORC > --

[jira] [Assigned] (SPARK-34960) Aggregate (Min/Max/Count) push down for ORC

[ https://issues.apache.org/jira/browse/SPARK-34960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34960: Assignee: (was: Apache Spark) > Aggregate (Min/Max/Count) push down for ORC > ---

[jira] [Commented] (SPARK-34960) Aggregate (Min/Max/Count) push down for ORC

[ https://issues.apache.org/jira/browse/SPARK-34960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17429513#comment-17429513 ] Apache Spark commented on SPARK-34960: -- User 'c21' has created a pull request for t

[jira] [Commented] (SPARK-36230) hasnans for Series of Decimal(`NaN`)

[ https://issues.apache.org/jira/browse/SPARK-36230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17429514#comment-17429514 ] Yikun Jiang commented on SPARK-36230: - working on this > hasnans for Series of Deci

[jira] [Commented] (SPARK-36231) Support arithmetic operations of Series containing Decimal(np.nan)

[ https://issues.apache.org/jira/browse/SPARK-36231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17429515#comment-17429515 ] Yikun Jiang commented on SPARK-36231: - working on this > Support arithmetic operati

[jira] [Assigned] (SPARK-36230) hasnans for Series of Decimal(`NaN`)

[ https://issues.apache.org/jira/browse/SPARK-36230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36230: Assignee: Apache Spark > hasnans for Series of Decimal(`NaN`) > -

[jira] [Assigned] (SPARK-36232) Support creating a ps.Series/Index with `Decimal('NaN')` with Arrow disabled

[ https://issues.apache.org/jira/browse/SPARK-36232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36232: Assignee: Apache Spark > Support creating a ps.Series/Index with `Decimal('NaN')` with Ar

[jira] [Assigned] (SPARK-36230) hasnans for Series of Decimal(`NaN`)

[ https://issues.apache.org/jira/browse/SPARK-36230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36230: Assignee: (was: Apache Spark) > hasnans for Series of Decimal(`NaN`) > --

[jira] [Commented] (SPARK-36230) hasnans for Series of Decimal(`NaN`)

[ https://issues.apache.org/jira/browse/SPARK-36230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17429517#comment-17429517 ] Apache Spark commented on SPARK-36230: -- User 'Yikun' has created a pull request for

[jira] [Assigned] (SPARK-36232) Support creating a ps.Series/Index with `Decimal('NaN')` with Arrow disabled

[ https://issues.apache.org/jira/browse/SPARK-36232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36232: Assignee: (was: Apache Spark) > Support creating a ps.Series/Index with `Decimal('NaN

[jira] [Commented] (SPARK-36232) Support creating a ps.Series/Index with `Decimal('NaN')` with Arrow disabled

[ https://issues.apache.org/jira/browse/SPARK-36232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17429518#comment-17429518 ] Apache Spark commented on SPARK-36232: -- User 'Yikun' has created a pull request for

[jira] [Assigned] (SPARK-36230) hasnans for Series of Decimal(`NaN`)

[ https://issues.apache.org/jira/browse/SPARK-36230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36230: Assignee: (was: Apache Spark) > hasnans for Series of Decimal(`NaN`) > --

[jira] [Commented] (SPARK-36232) Support creating a ps.Series/Index with `Decimal('NaN')` with Arrow disabled

[ https://issues.apache.org/jira/browse/SPARK-36232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17429519#comment-17429519 ] Apache Spark commented on SPARK-36232: -- User 'Yikun' has created a pull request for

[jira] [Commented] (SPARK-37004) Job cancellation causes py4j errors on Jupyter due to pinned thread mode

[ https://issues.apache.org/jira/browse/SPARK-37004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17429523#comment-17429523 ] Hyukjin Kwon commented on SPARK-37004: -- Just for other people who face this issue f

[jira] [Created] (SPARK-37023) Avoid fetching merge status when shuffleMergeEnabled is false for a shuffleDependency during retry

Ye Zhou created SPARK-37023: --- Summary: Avoid fetching merge status when shuffleMergeEnabled is false for a shuffleDependency during retry Key: SPARK-37023 URL: https://issues.apache.org/jira/browse/SPARK-37023