[jira] [Commented] (SPARK-37989) Push down limit through Aggregate if it is group only

[ https://issues.apache.org/jira/browse/SPARK-37989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17480882#comment-17480882 ] Yuming Wang commented on SPARK-37989: - Benchmark 3 {noformat} import org.apache.spar

[jira] [Commented] (SPARK-37987) Flaky Test: StreamingAggregationSuite.changing schema of state when restarting query - state format version 1

[ https://issues.apache.org/jira/browse/SPARK-37987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17480877#comment-17480877 ] Apache Spark commented on SPARK-37987: -- User 'HeartSaVioR' has created a pull reque

[jira] [Created] (SPARK-37994) Unable to build spark3.2 with -Dhadoop.version=3.1.4

Vu Tan created SPARK-37994: -- Summary: Unable to build spark3.2 with -Dhadoop.version=3.1.4 Key: SPARK-37994 URL: https://issues.apache.org/jira/browse/SPARK-37994 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-37989) Push down limit through Aggregate if it is group only

[ https://issues.apache.org/jira/browse/SPARK-37989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17480855#comment-17480855 ] Yuming Wang commented on SPARK-37989: - Benchmark 2 {code:scala} import org.apache.s

[jira] [Commented] (SPARK-37989) Push down limit through Aggregate if it is group only

[ https://issues.apache.org/jira/browse/SPARK-37989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17480839#comment-17480839 ] Yuming Wang commented on SPARK-37989: - Benchmark 1 {code:scala} import org.apache.s

[jira] [Resolved] (SPARK-37990) Support TimestampNTZ in RowToColumnConverter

[ https://issues.apache.org/jira/browse/SPARK-37990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-37990. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35288 [https://gi

[jira] [Commented] (SPARK-37980) Extend METADATA column to support row indices for file based data sources

[ https://issues.apache.org/jira/browse/SPARK-37980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17480811#comment-17480811 ] Wenchen Fan commented on SPARK-37980: - I think it's going to be a useful feature to

[jira] [Assigned] (SPARK-37984) Avoid calculating all outstanding requests to improve performance.

[ https://issues.apache.org/jira/browse/SPARK-37984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-37984: Assignee: weixiuli > Avoid calculating all outstanding requests to improve performance. >

[jira] [Resolved] (SPARK-37984) Avoid calculating all outstanding requests to improve performance.

[ https://issues.apache.org/jira/browse/SPARK-37984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-37984. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35276 [https://gi

[jira] [Commented] (SPARK-37993) Avoid multiple calls to configuration parameter values

[ https://issues.apache.org/jira/browse/SPARK-37993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17480781#comment-17480781 ] Apache Spark commented on SPARK-37993: -- User 'weixiuli' has created a pull request

[jira] [Commented] (SPARK-37993) Avoid multiple calls to configuration parameter values

[ https://issues.apache.org/jira/browse/SPARK-37993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17480780#comment-17480780 ] Apache Spark commented on SPARK-37993: -- User 'weixiuli' has created a pull request

[jira] [Assigned] (SPARK-37993) Avoid multiple calls to configuration parameter values

[ https://issues.apache.org/jira/browse/SPARK-37993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37993: Assignee: (was: Apache Spark) > Avoid multiple calls to configuration parameter value

[jira] [Assigned] (SPARK-37993) Avoid multiple calls to configuration parameter values

[ https://issues.apache.org/jira/browse/SPARK-37993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37993: Assignee: Apache Spark > Avoid multiple calls to configuration parameter values > ---

[jira] [Updated] (SPARK-37993) Avoid multiple calls to configuration parameter values

[ https://issues.apache.org/jira/browse/SPARK-37993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] weixiuli updated SPARK-37993: - Description: Avoid multiple calls to configuration parameter values (was: Avoid multiple calls to conf

[jira] [Created] (SPARK-37993) Avoid multiple calls to conf parameter values

weixiuli created SPARK-37993: Summary: Avoid multiple calls to conf parameter values Key: SPARK-37993 URL: https://issues.apache.org/jira/browse/SPARK-37993 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-37991) Improve test case inside SQL\Catalyst\Catalog

[ https://issues.apache.org/jira/browse/SPARK-37991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37991: Assignee: Apache Spark > Improve test case inside SQL\Catalyst\Catalog >

[jira] [Assigned] (SPARK-37991) Improve test case inside SQL\Catalyst\Catalog

[ https://issues.apache.org/jira/browse/SPARK-37991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37991: Assignee: (was: Apache Spark) > Improve test case inside SQL\Catalyst\Catalog > -

[jira] [Resolved] (SPARK-37992) Restore mypy version check in dev/lint-python

[ https://issues.apache.org/jira/browse/SPARK-37992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-37992. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35294 [https://gi

[jira] [Assigned] (SPARK-37992) Restore mypy version check in dev/lint-python

[ https://issues.apache.org/jira/browse/SPARK-37992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-37992: Assignee: Maciej Szymkiewicz > Restore mypy version check in dev/lint-python > --

[jira] [Resolved] (SPARK-37188) pyspark.pandas histogram accepts the title option but does not add a title to the plot

[ https://issues.apache.org/jira/browse/SPARK-37188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-37188. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35282 [https://gi

[jira] [Assigned] (SPARK-37188) pyspark.pandas histogram accepts the title option but does not add a title to the plot

[ https://issues.apache.org/jira/browse/SPARK-37188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-37188: Assignee: Yikun Jiang > pyspark.pandas histogram accepts the title option but does not ad

[jira] [Resolved] (SPARK-37408) Inline type hints for python/pyspark/ml/image.py

[ https://issues.apache.org/jira/browse/SPARK-37408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-37408. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35280 [https://gi

[jira] [Resolved] (SPARK-37985) Fix flaky test SPARK-37578

[ https://issues.apache.org/jira/browse/SPARK-37985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-37985. -- Fix Version/s: 3.3 Assignee: angerszhu Resolution: Fixed Fixed in https://gith

[jira] [Updated] (SPARK-37985) Fix flaky test SPARK-37578

[ https://issues.apache.org/jira/browse/SPARK-37985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-37985: - Fix Version/s: 3.3.0 (was: 3.3) > Fix flaky test SPARK-37578 > --

[jira] [Comment Edited] (SPARK-31726) Make spark.files available in driver with cluster deploy mode on kubernetes

[ https://issues.apache.org/jira/browse/SPARK-31726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17480756#comment-17480756 ] koert kuipers edited comment on SPARK-31726 at 1/23/22, 11:22 PM:

[jira] [Commented] (SPARK-31726) Make spark.files available in driver with cluster deploy mode on kubernetes

[ https://issues.apache.org/jira/browse/SPARK-31726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17480756#comment-17480756 ] koert kuipers commented on SPARK-31726: --- [~beregon87] about --jars, are you seeing

[jira] [Commented] (SPARK-37981) Deletes columns with all Null as default.

[ https://issues.apache.org/jira/browse/SPARK-37981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17480737#comment-17480737 ] Bjørn Jørgensen commented on SPARK-37981: - Please note that the PR that I have o

[jira] [Commented] (SPARK-37981) Deletes columns with all Null as default.

[ https://issues.apache.org/jira/browse/SPARK-37981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17480733#comment-17480733 ] Apache Spark commented on SPARK-37981: -- User 'bjornjorgensen' has created a pull re

[jira] [Commented] (SPARK-37981) Deletes columns with all Null as default.

[ https://issues.apache.org/jira/browse/SPARK-37981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17480734#comment-17480734 ] Apache Spark commented on SPARK-37981: -- User 'bjornjorgensen' has created a pull re

[jira] [Commented] (SPARK-37936) Use error classes in the parsing errors of intervals

[ https://issues.apache.org/jira/browse/SPARK-37936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17480715#comment-17480715 ] Apache Spark commented on SPARK-37936: -- User 'senthh' has created a pull request fo

[jira] [Assigned] (SPARK-37936) Use error classes in the parsing errors of intervals

[ https://issues.apache.org/jira/browse/SPARK-37936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37936: Assignee: Apache Spark > Use error classes in the parsing errors of intervals > -

[jira] [Commented] (SPARK-37936) Use error classes in the parsing errors of intervals

[ https://issues.apache.org/jira/browse/SPARK-37936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17480714#comment-17480714 ] Apache Spark commented on SPARK-37936: -- User 'senthh' has created a pull request fo

[jira] [Assigned] (SPARK-37936) Use error classes in the parsing errors of intervals

[ https://issues.apache.org/jira/browse/SPARK-37936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37936: Assignee: (was: Apache Spark) > Use error classes in the parsing errors of intervals

[jira] [Commented] (SPARK-37992) Restore mypy version check in dev/lint-python

[ https://issues.apache.org/jira/browse/SPARK-37992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17480711#comment-17480711 ] Apache Spark commented on SPARK-37992: -- User 'zero323' has created a pull request f

[jira] [Assigned] (SPARK-37992) Restore mypy version check in dev/lint-python

[ https://issues.apache.org/jira/browse/SPARK-37992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37992: Assignee: (was: Apache Spark) > Restore mypy version check in dev/lint-python > -

[jira] [Commented] (SPARK-37992) Restore mypy version check in dev/lint-python

[ https://issues.apache.org/jira/browse/SPARK-37992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17480710#comment-17480710 ] Apache Spark commented on SPARK-37992: -- User 'zero323' has created a pull request f

[jira] [Assigned] (SPARK-37992) Restore mypy version check in dev/lint-python

[ https://issues.apache.org/jira/browse/SPARK-37992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37992: Assignee: Apache Spark > Restore mypy version check in dev/lint-python >

[jira] [Created] (SPARK-37992) Restore mypy version check in dev/lint-python

Maciej Szymkiewicz created SPARK-37992: -- Summary: Restore mypy version check in dev/lint-python Key: SPARK-37992 URL: https://issues.apache.org/jira/browse/SPARK-37992 Project: Spark Iss

[jira] [Commented] (SPARK-37935) Migrate onto error classes

[ https://issues.apache.org/jira/browse/SPARK-37935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17480683#comment-17480683 ] Apache Spark commented on SPARK-37935: -- User 'khalidmammadov' has created a pull re

[jira] [Assigned] (SPARK-37941) Use error classes in the compilation errors of casting

[ https://issues.apache.org/jira/browse/SPARK-37941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37941: Assignee: (was: Apache Spark) > Use error classes in the compilation errors of castin

[jira] [Assigned] (SPARK-37941) Use error classes in the compilation errors of casting

[ https://issues.apache.org/jira/browse/SPARK-37941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37941: Assignee: Apache Spark > Use error classes in the compilation errors of casting > ---

[jira] [Commented] (SPARK-37941) Use error classes in the compilation errors of casting

[ https://issues.apache.org/jira/browse/SPARK-37941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17480669#comment-17480669 ] Apache Spark commented on SPARK-37941: -- User 'ivoson' has created a pull request fo

[jira] [Assigned] (SPARK-37935) Migrate onto error classes

[ https://issues.apache.org/jira/browse/SPARK-37935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37935: Assignee: Apache Spark (was: Max Gekk) > Migrate onto error classes > --

[jira] [Assigned] (SPARK-37935) Migrate onto error classes

[ https://issues.apache.org/jira/browse/SPARK-37935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37935: Assignee: Max Gekk (was: Apache Spark) > Migrate onto error classes > --

[jira] [Commented] (SPARK-37935) Migrate onto error classes

[ https://issues.apache.org/jira/browse/SPARK-37935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17480657#comment-17480657 ] Apache Spark commented on SPARK-37935: -- User 'khalidmammadov' has created a pull re

[jira] [Commented] (SPARK-37935) Migrate onto error classes

[ https://issues.apache.org/jira/browse/SPARK-37935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17480658#comment-17480658 ] Apache Spark commented on SPARK-37935: -- User 'khalidmammadov' has created a pull re

[jira] [Created] (SPARK-37991) Improve test case inside SQL\Catalyst\Catalog

Khalid Mammadov created SPARK-37991: --- Summary: Improve test case inside SQL\Catalyst\Catalog Key: SPARK-37991 URL: https://issues.apache.org/jira/browse/SPARK-37991 Project: Spark Issue Typ

[jira] [Updated] (SPARK-37985) Fix flaky test SPARK-37578

[ https://issues.apache.org/jira/browse/SPARK-37985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-37985: - Priority: Minor (was: Major) > Fix flaky test SPARK-37578 > -- > >

[jira] [Updated] (SPARK-37985) Fix flaky test SPARK-37578

[ https://issues.apache.org/jira/browse/SPARK-37985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-37985: - Component/s: Tests > Fix flaky test SPARK-37578 > -- > >

[jira] [Updated] (SPARK-37984) Avoid calculating all outstanding requests to improve performance.

[ https://issues.apache.org/jira/browse/SPARK-37984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-37984: - Priority: Trivial (was: Major) > Avoid calculating all outstanding requests to improve performa

[jira] [Commented] (SPARK-37865) Spark should not dedup the groupingExpressions when the first child of Union has duplicate columns

[ https://issues.apache.org/jira/browse/SPARK-37865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17480634#comment-17480634 ] Apache Spark commented on SPARK-37865: -- User 'chasingegg' has created a pull reques

[jira] [Commented] (SPARK-37865) Spark should not dedup the groupingExpressions when the first child of Union has duplicate columns

[ https://issues.apache.org/jira/browse/SPARK-37865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17480632#comment-17480632 ] Apache Spark commented on SPARK-37865: -- User 'chasingegg' has created a pull reques

[jira] [Assigned] (SPARK-37397) Inline type hints for python/pyspark/ml/base.py

[ https://issues.apache.org/jira/browse/SPARK-37397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37397: Assignee: (was: Apache Spark) > Inline type hints for python/pyspark/ml/base.py > ---

[jira] [Commented] (SPARK-37397) Inline type hints for python/pyspark/ml/base.py

[ https://issues.apache.org/jira/browse/SPARK-37397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17480619#comment-17480619 ] Apache Spark commented on SPARK-37397: -- User 'zero323' has created a pull request f

[jira] [Assigned] (SPARK-37397) Inline type hints for python/pyspark/ml/base.py

[ https://issues.apache.org/jira/browse/SPARK-37397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37397: Assignee: Apache Spark > Inline type hints for python/pyspark/ml/base.py > --

[jira] [Assigned] (SPARK-37990) Support TimestampNTZ in RowToColumnConverter

[ https://issues.apache.org/jira/browse/SPARK-37990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37990: Assignee: Gengliang Wang (was: Apache Spark) > Support TimestampNTZ in RowToColumnConver

[jira] [Assigned] (SPARK-37990) Support TimestampNTZ in RowToColumnConverter

[ https://issues.apache.org/jira/browse/SPARK-37990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-37990: Assignee: Apache Spark (was: Gengliang Wang) > Support TimestampNTZ in RowToColumnConver

[jira] [Commented] (SPARK-37990) Support TimestampNTZ in RowToColumnConverter

[ https://issues.apache.org/jira/browse/SPARK-37990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17480577#comment-17480577 ] Apache Spark commented on SPARK-37990: -- User 'gengliangwang' has created a pull req

[jira] [Created] (SPARK-37990) Support TimestampNTZ in RowToColumnConverter

Gengliang Wang created SPARK-37990: -- Summary: Support TimestampNTZ in RowToColumnConverter Key: SPARK-37990 URL: https://issues.apache.org/jira/browse/SPARK-37990 Project: Spark Issue Type: