[jira] [Commented] (SPARK-36839) Add daily build with Hadoop 2 profile in GitHub Actions build

2021-10-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17426509#comment-17426509 ] Apache Spark commented on SPARK-36839: -- User 'HyukjinKwon' has created a pull reque

[jira] [Commented] (SPARK-36839) Add daily build with Hadoop 2 profile in GitHub Actions build

2021-10-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17426510#comment-17426510 ] Apache Spark commented on SPARK-36839: -- User 'HyukjinKwon' has created a pull reque

[jira] [Resolved] (SPARK-36897) Replace collections.namedtuple() by typing.NamedTuple

2021-10-08 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-36897. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34228 [https://gi

[jira] [Assigned] (SPARK-36897) Replace collections.namedtuple() by typing.NamedTuple

2021-10-08 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-36897: Assignee: Xinrong Meng > Replace collections.namedtuple() by typing.NamedTuple >

[jira] [Commented] (SPARK-36958) Reading of legacy timestamps from Parquet confusing in Spark 3, related config values don't seem working

2021-10-08 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17426495#comment-17426495 ] Hyukjin Kwon commented on SPARK-36958: -- [~dgoldenberg] doesn't it work if you set i

[jira] [Commented] (SPARK-36921) The DIV function should support ANSI intervals

2021-10-08 Thread PengLei (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17426470#comment-17426470 ] PengLei commented on SPARK-36921: - working on this > The DIV function should support AN

[jira] [Commented] (SPARK-36922) The SIGN/SIGNUM functions should support ANSI intervals

2021-10-08 Thread PengLei (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17426471#comment-17426471 ] PengLei commented on SPARK-36922: - working on this > The SIGN/SIGNUM functions should s

[jira] [Resolved] (SPARK-36238) Spark UI load event timeline too slow for huge stage

2021-10-08 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu resolved SPARK-36238. --- Resolution: Duplicate > Spark UI load event timeline too slow for huge stage >

[jira] [Commented] (SPARK-36962) Make HiveSerDe.serdeMap extensible

2021-10-08 Thread Yann Byron (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17426458#comment-17426458 ] Yann Byron commented on SPARK-36962: [~cloud_fan] can you have a look at it? > Make

[jira] [Commented] (SPARK-36963) Add max_by/min_by to sql.functions

2021-10-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17426456#comment-17426456 ] Apache Spark commented on SPARK-36963: -- User 'zhengruifeng' has created a pull requ

[jira] [Commented] (SPARK-36963) Add max_by/min_by to sql.functions

2021-10-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17426455#comment-17426455 ] Apache Spark commented on SPARK-36963: -- User 'zhengruifeng' has created a pull requ

[jira] [Assigned] (SPARK-36963) Add max_by/min_by to sql.functions

2021-10-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36963: Assignee: (was: Apache Spark) > Add max_by/min_by to sql.functions >

[jira] [Assigned] (SPARK-36963) Add max_by/min_by to sql.functions

2021-10-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36963: Assignee: Apache Spark > Add max_by/min_by to sql.functions > ---

[jira] [Created] (SPARK-36963) Add max_by/min_by to sql.functions

2021-10-08 Thread zhengruifeng (Jira)
zhengruifeng created SPARK-36963: Summary: Add max_by/min_by to sql.functions Key: SPARK-36963 URL: https://issues.apache.org/jira/browse/SPARK-36963 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-36962) Make HiveSerDe.serdeMap extensible

2021-10-08 Thread Yann Byron (Jira)
Yann Byron created SPARK-36962: -- Summary: Make HiveSerDe.serdeMap extensible Key: SPARK-36962 URL: https://issues.apache.org/jira/browse/SPARK-36962 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-36238) Spark UI load event timeline too slow for huge stage

2021-10-08 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17426441#comment-17426441 ] angerszhu commented on SPARK-36238: --- [~thejdeep]Seems so > Spark UI load event timel

[jira] [Commented] (SPARK-35428) Spark history Server to S3 doesn't show incomplete applications

2021-10-08 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17426440#comment-17426440 ] Jungtaek Lim commented on SPARK-35428: -- I think this is based on the characteristic

[jira] [Updated] (SPARK-36955) when we change the complex column's comment of hive table , it should succeed but fails

2021-10-08 Thread Liu Shuo (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liu Shuo updated SPARK-36955: - Description: 1. create a hive table {code:java} CREATE TABLE tbl STORED AS PARQUET AS SELECT '1st' AS k

[jira] [Assigned] (SPARK-36944) Remove unused python/pyspark/sql/__init__.pyi

2021-10-08 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz reassigned SPARK-36944: -- Assignee: dch nguyen > Remove unused python/pyspark/sql/__init__.pyi > --

[jira] [Resolved] (SPARK-36944) Remove unused python/pyspark/sql/__init__.pyi

2021-10-08 Thread Maciej Szymkiewicz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz resolved SPARK-36944. Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34203

[jira] [Assigned] (SPARK-36897) Replace collections.namedtuple() by typing.NamedTuple

2021-10-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36897: Assignee: Apache Spark > Replace collections.namedtuple() by typing.NamedTuple >

[jira] [Assigned] (SPARK-36897) Replace collections.namedtuple() by typing.NamedTuple

2021-10-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36897: Assignee: (was: Apache Spark) > Replace collections.namedtuple() by typing.NamedTuple

[jira] [Commented] (SPARK-36897) Replace collections.namedtuple() by typing.NamedTuple

2021-10-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17426410#comment-17426410 ] Apache Spark commented on SPARK-36897: -- User 'ueshin' has created a pull request fo

[jira] [Commented] (SPARK-36897) Replace collections.namedtuple() by typing.NamedTuple

2021-10-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17426409#comment-17426409 ] Apache Spark commented on SPARK-36897: -- User 'ueshin' has created a pull request fo

[jira] [Assigned] (SPARK-36961) Use PEP526 style variable type hints

2021-10-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36961: Assignee: Apache Spark > Use PEP526 style variable type hints > -

[jira] [Assigned] (SPARK-36961) Use PEP526 style variable type hints

2021-10-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36961: Assignee: (was: Apache Spark) > Use PEP526 style variable type hints > --

[jira] [Commented] (SPARK-36961) Use PEP526 style variable type hints

2021-10-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17426394#comment-17426394 ] Apache Spark commented on SPARK-36961: -- User 'ueshin' has created a pull request fo

[jira] [Updated] (SPARK-36961) Use PEP526 style variable type hints

2021-10-08 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-36961: -- Description: Now that we have started using newer Python syntax in the code base. We should us

[jira] [Updated] (SPARK-36961) Use PEP526 style variable type hints

2021-10-08 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-36961: -- Summary: Use PEP526 style variable type hints (was: Use PEP526 style variable type hints in p

[jira] [Updated] (SPARK-36961) Use PEP526 style variable type hints in pandas-on-Spark

2021-10-08 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-36961: -- Description: Now that we have started using newer Python syntax in the code base. We should us

[jira] [Created] (SPARK-36961) Use PEP526 style variable type hints in pandas-on-Spark

2021-10-08 Thread Takuya Ueshin (Jira)
Takuya Ueshin created SPARK-36961: - Summary: Use PEP526 style variable type hints in pandas-on-Spark Key: SPARK-36961 URL: https://issues.apache.org/jira/browse/SPARK-36961 Project: Spark Iss

[jira] [Commented] (SPARK-36951) Inline type hints for python/pyspark/sql/column.py

2021-10-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17426386#comment-17426386 ] Apache Spark commented on SPARK-36951: -- User 'xinrong-databricks' has created a pul

[jira] [Assigned] (SPARK-36951) Inline type hints for python/pyspark/sql/column.py

2021-10-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36951: Assignee: (was: Apache Spark) > Inline type hints for python/pyspark/sql/column.py >

[jira] [Commented] (SPARK-36951) Inline type hints for python/pyspark/sql/column.py

2021-10-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17426385#comment-17426385 ] Apache Spark commented on SPARK-36951: -- User 'xinrong-databricks' has created a pul

[jira] [Assigned] (SPARK-36951) Inline type hints for python/pyspark/sql/column.py

2021-10-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36951: Assignee: Apache Spark > Inline type hints for python/pyspark/sql/column.py > ---

[jira] [Assigned] (SPARK-36885) Inline type hints for python/pyspark/sql/dataframe.py

2021-10-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36885: Assignee: Apache Spark > Inline type hints for python/pyspark/sql/dataframe.py >

[jira] [Commented] (SPARK-36885) Inline type hints for python/pyspark/sql/dataframe.py

2021-10-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17426380#comment-17426380 ] Apache Spark commented on SPARK-36885: -- User 'ueshin' has created a pull request fo

[jira] [Assigned] (SPARK-36885) Inline type hints for python/pyspark/sql/dataframe.py

2021-10-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36885: Assignee: Apache Spark > Inline type hints for python/pyspark/sql/dataframe.py >

[jira] [Assigned] (SPARK-36885) Inline type hints for python/pyspark/sql/dataframe.py

2021-10-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36885: Assignee: (was: Apache Spark) > Inline type hints for python/pyspark/sql/dataframe.py

[jira] [Assigned] (SPARK-36960) Pushdown filters with ANSI interval values to ORC

2021-10-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36960: Assignee: Kousuke Saruta (was: Apache Spark) > Pushdown filters with ANSI interval value

[jira] [Assigned] (SPARK-36960) Pushdown filters with ANSI interval values to ORC

2021-10-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36960: Assignee: Apache Spark (was: Kousuke Saruta) > Pushdown filters with ANSI interval value

[jira] [Commented] (SPARK-36960) Pushdown filters with ANSI interval values to ORC

2021-10-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17426354#comment-17426354 ] Apache Spark commented on SPARK-36960: -- User 'sarutak' has created a pull request f

[jira] [Created] (SPARK-36960) Pushdown filters with ANSI interval values to ORC

2021-10-08 Thread Kousuke Saruta (Jira)
Kousuke Saruta created SPARK-36960: -- Summary: Pushdown filters with ANSI interval values to ORC Key: SPARK-36960 URL: https://issues.apache.org/jira/browse/SPARK-36960 Project: Spark Issue T

[jira] [Assigned] (SPARK-36913) Implement createIndex and IndexExists in JDBC (MySQL dialect)

2021-10-08 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh reassigned SPARK-36913: --- Assignee: Huaxin Gao > Implement createIndex and IndexExists in JDBC (MySQL dialect) >

[jira] [Resolved] (SPARK-36913) Implement createIndex and IndexExists in JDBC (MySQL dialect)

2021-10-08 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-36913. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34164 [https://gith

[jira] [Created] (SPARK-36959) Renamed columns of parquet tables become NULL

2021-10-08 Thread Jira
Tóth Andor created SPARK-36959: -- Summary: Renamed columns of parquet tables become NULL Key: SPARK-36959 URL: https://issues.apache.org/jira/browse/SPARK-36959 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-36238) Spark UI load event timeline too slow for huge stage

2021-10-08 Thread Thejdeep Gudivada (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17426269#comment-17426269 ] Thejdeep Gudivada commented on SPARK-36238: --- This should be fixed by SPARK-352

[jira] [Commented] (SPARK-36936) spark-hadoop-cloud broken on release and only published via 3rd party repositories

2021-10-08 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17426255#comment-17426255 ] Chao Sun commented on SPARK-36936: -- [~colin.williams] Spark 3.2.0 is not released yet -

[jira] [Created] (SPARK-36958) Reading of legacy timestamps from Parquet confusing in Spark 3, related config values don't seem working

2021-10-08 Thread Dmitry Goldenberg (Jira)
Dmitry Goldenberg created SPARK-36958: - Summary: Reading of legacy timestamps from Parquet confusing in Spark 3, related config values don't seem working Key: SPARK-36958 URL: https://issues.apache.org/jira/br

[jira] [Commented] (SPARK-36957) Add product function to SQL function

2021-10-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17426180#comment-17426180 ] Apache Spark commented on SPARK-36957: -- User 'AngersZh' has created a pull requ

[jira] [Assigned] (SPARK-36957) Add product function to SQL function

2021-10-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36957: Assignee: (was: Apache Spark) > Add product function to SQL function > --

[jira] [Commented] (SPARK-36957) Add product function to SQL function

2021-10-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17426178#comment-17426178 ] Apache Spark commented on SPARK-36957: -- User 'AngersZh' has created a pull requ

[jira] [Assigned] (SPARK-36957) Add product function to SQL function

2021-10-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36957: Assignee: Apache Spark > Add product function to SQL function > -

[jira] [Commented] (SPARK-36024) Switch the datasource example due to the depreciation of the dataset

2021-10-08 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17426166#comment-17426166 ] Steve Loughran commented on SPARK-36024: Amazon are being very nice here and kee

[jira] [Commented] (SPARK-36761) spark-examples_2.12-3.0.2.jar DFSReadWriteTest S3A Implementation

2021-10-08 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17426163#comment-17426163 ] Steve Loughran commented on SPARK-36761: something in the code has got the defau

[jira] [Commented] (SPARK-35428) Spark history Server to S3 doesn't show incomplete applications

2021-10-08 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17426159#comment-17426159 ] Steve Loughran commented on SPARK-35428: # please stop using s3n; that connector

[jira] [Commented] (SPARK-36529) Decouple CPU with IO work in vectorized Parquet reader

2021-10-08 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17426158#comment-17426158 ] Steve Loughran commented on SPARK-36529: If you look at HADOOP-11867 / https://g

[jira] [Commented] (SPARK-36766) Spark SQL DDL does not recognize fs.s3.impl implied filesystem in LOCATION tag

2021-10-08 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17426152#comment-17426152 ] Steve Loughran commented on SPARK-36766: I can see why you'd want to do this (co

[jira] [Resolved] (SPARK-36717) Wrong order of variable initialization may lead to incorrect behavior

2021-10-08 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-36717. -- Fix Version/s: 3.3.0 3.0.4 3.1.3 3.2.0

[jira] [Assigned] (SPARK-36717) Wrong order of variable initialization may lead to incorrect behavior

2021-10-08 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-36717: Assignee: Jianmeng Li > Wrong order of variable initialization may lead to incorrect beha

[jira] [Updated] (SPARK-36717) Wrong order of variable initialization may lead to incorrect behavior

2021-10-08 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-36717: - Fix Version/s: (was: 3.1.3) Target Version/s: (was: 3.1.2) > Wrong order of variabl

[jira] [Updated] (SPARK-36717) Wrong order of variable initialization may lead to incorrect behavior

2021-10-08 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-36717: - Priority: Minor (was: Major) > Wrong order of variable initialization may lead to incorrect beh

[jira] [Assigned] (SPARK-36929) Remove Unused Method EliminateSubqueryAliasesSuite#assertEquivalent

2021-10-08 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-36929: Assignee: leesf > Remove Unused Method EliminateSubqueryAliasesSuite#assertEquivalent > -

[jira] [Resolved] (SPARK-36929) Remove Unused Method EliminateSubqueryAliasesSuite#assertEquivalent

2021-10-08 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-36929. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34175 [https://gi

[jira] [Updated] (SPARK-36929) Remove Unused Method EliminateSubqueryAliasesSuite#assertEquivalent

2021-10-08 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-36929: - Priority: Trivial (was: Minor) > Remove Unused Method EliminateSubqueryAliasesSuite#assertEquiv

[jira] [Commented] (SPARK-33678) Numerical product aggregation

2021-10-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17426118#comment-17426118 ] Apache Spark commented on SPARK-33678: -- User 'AngersZh' has created a pull requ

[jira] [Created] (SPARK-36957) Add product function to SQL function

2021-10-08 Thread angerszhu (Jira)
angerszhu created SPARK-36957: - Summary: Add product function to SQL function Key: SPARK-36957 URL: https://issues.apache.org/jira/browse/SPARK-36957 Project: Spark Issue Type: Task Com

[jira] [Commented] (SPARK-36952) Inline type hints for python/pyspark/resource/information.py and python/pyspark/resource/profile.py

2021-10-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17426083#comment-17426083 ] Apache Spark commented on SPARK-36952: -- User 'dchvn' has created a pull request for

[jira] [Assigned] (SPARK-36952) Inline type hints for python/pyspark/resource/information.py and python/pyspark/resource/profile.py

2021-10-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36952: Assignee: Apache Spark > Inline type hints for python/pyspark/resource/information.py and

[jira] [Assigned] (SPARK-36952) Inline type hints for python/pyspark/resource/information.py and python/pyspark/resource/profile.py

2021-10-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36952: Assignee: (was: Apache Spark) > Inline type hints for python/pyspark/resource/informa

[jira] [Commented] (SPARK-36952) Inline type hints for python/pyspark/resource/information.py and python/pyspark/resource/profile.py

2021-10-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17426082#comment-17426082 ] Apache Spark commented on SPARK-36952: -- User 'dchvn' has created a pull request for

[jira] [Commented] (SPARK-36903) oom exception occurred during code generation due to a large number of case when branches

2021-10-08 Thread JacobZheng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17426075#comment-17426075 ] JacobZheng commented on SPARK-36903: Yes. The same issue occurs on Spark 3.1. Actual

[jira] [Assigned] (SPARK-36956) model prediction in .mllib avoid conversion to breeze vector

2021-10-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36956: Assignee: (was: Apache Spark) > model prediction in .mllib avoid conversion to breeze

[jira] [Commented] (SPARK-36956) model prediction in .mllib avoid conversion to breeze vector

2021-10-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17426068#comment-17426068 ] Apache Spark commented on SPARK-36956: -- User 'zhengruifeng' has created a pull requ

[jira] [Assigned] (SPARK-36956) model prediction in .mllib avoid conversion to breeze vector

2021-10-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36956: Assignee: Apache Spark > model prediction in .mllib avoid conversion to breeze vector > -

[jira] [Created] (SPARK-36956) model prediction in .mllib avoid conversion to breeze vector

2021-10-08 Thread zhengruifeng (Jira)
zhengruifeng created SPARK-36956: Summary: model prediction in .mllib avoid conversion to breeze vector Key: SPARK-36956 URL: https://issues.apache.org/jira/browse/SPARK-36956 Project: Spark

[jira] [Resolved] (SPARK-36868) Migrate CreateFunctionStatement to v2 command framework

2021-10-08 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-36868. - Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34121 [https://gith

[jira] [Assigned] (SPARK-36868) Migrate CreateFunctionStatement to v2 command framework

2021-10-08 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-36868: --- Assignee: Huaxin Gao > Migrate CreateFunctionStatement to v2 command framework > --

[jira] [Assigned] (SPARK-36931) Read/write dataframes with ANSI intervals from/to ORC files

2021-10-08 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk reassigned SPARK-36931: Assignee: Kousuke Saruta > Read/write dataframes with ANSI intervals from/to ORC files >

[jira] [Resolved] (SPARK-36931) Read/write dataframes with ANSI intervals from/to ORC files

2021-10-08 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-36931. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34184 [https://github.com

[jira] [Resolved] (SPARK-36948) Check CREATE TABLE with ANSI intervals using Hive external catalog and Parquet

2021-10-08 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-36948. -- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 34215 [https://github.com

[jira] [Commented] (SPARK-36954) Fast fail with explicit err msg when calling withWatermark on non-streaming dataset

2021-10-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17426017#comment-17426017 ] Apache Spark commented on SPARK-36954: -- User 'ivoson' has created a pull request fo

[jira] [Assigned] (SPARK-36954) Fast fail with explicit err msg when calling withWatermark on non-streaming dataset

2021-10-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36954: Assignee: Apache Spark > Fast fail with explicit err msg when calling withWatermark on no

[jira] [Assigned] (SPARK-36954) Fast fail with explicit err msg when calling withWatermark on non-streaming dataset

2021-10-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-36954: Assignee: (was: Apache Spark) > Fast fail with explicit err msg when calling withWate

[jira] [Commented] (SPARK-36954) Fast fail with explicit err msg when calling withWatermark on non-streaming dataset

2021-10-08 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17426016#comment-17426016 ] Apache Spark commented on SPARK-36954: -- User 'ivoson' has created a pull request fo

[jira] [Created] (SPARK-36955) when we change the complex column's comment of hive table , it should succeed but fails

2021-10-08 Thread Liu Qilei (Jira)
Liu Qilei created SPARK-36955: - Summary: when we change the complex column's comment of hive table , it should succeed but fails Key: SPARK-36955 URL: https://issues.apache.org/jira/browse/SPARK-36955 Pro

[jira] [Updated] (SPARK-36954) Fast fail with explicit err msg when calling withWatermark on non-streaming dataset

2021-10-08 Thread huangtengfei (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] huangtengfei updated SPARK-36954: - Description: [Dataset.withWatermark|https://github.com/apache/spark/blob/v3.2.0-rc7/sql/core/src

[jira] [Updated] (SPARK-36954) Fast fail with explicit err msg when calling withWatermark on non-streaming dataset

2021-10-08 Thread huangtengfei (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] huangtengfei updated SPARK-36954: - Environment: (was: [Dataset.withWatermark|https://github.com/apache/spark/blob/v3.2.0-rc7/sq

[jira] [Created] (SPARK-36954) Fast fail with explicit err msg when calling withWatermark on non-streaming dataset

2021-10-08 Thread huangtengfei (Jira)
huangtengfei created SPARK-36954: Summary: Fast fail with explicit err msg when calling withWatermark on non-streaming dataset Key: SPARK-36954 URL: https://issues.apache.org/jira/browse/SPARK-36954 P