[jira] [Updated] (SPARK-35707) optimize sparse GEMM by skipping bound checking

2021-06-09 Thread zhengruifeng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng updated SPARK-35707: - Summary: optimize sparse GEMM by skipping bound checking (was: optimize sparse GEMM by skip bou

[jira] [Created] (SPARK-35707) optimize sparse GEMM by skip bound checking

2021-06-09 Thread zhengruifeng (Jira)
zhengruifeng created SPARK-35707: Summary: optimize sparse GEMM by skip bound checking Key: SPARK-35707 URL: https://issues.apache.org/jira/browse/SPARK-35707 Project: Spark Issue Type: Sub-t

[jira] [Commented] (SPARK-35706) Consider making the ':' in STRUCT data type definition optional

2021-06-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17360592#comment-17360592 ] Apache Spark commented on SPARK-35706: -- User 'jerqi' has created a pull request for

[jira] [Assigned] (SPARK-35706) Consider making the ':' in STRUCT data type definition optional

2021-06-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35706: Assignee: (was: Apache Spark) > Consider making the ':' in STRUCT data type definitio

[jira] [Commented] (SPARK-35706) Consider making the ':' in STRUCT data type definition optional

2021-06-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17360590#comment-17360590 ] Apache Spark commented on SPARK-35706: -- User 'jerqi' has created a pull request for

[jira] [Assigned] (SPARK-35706) Consider making the ':' in STRUCT data type definition optional

2021-06-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35706: Assignee: Apache Spark > Consider making the ':' in STRUCT data type definition optional

[jira] [Commented] (SPARK-34524) simplify v2 partition commands resolution

2021-06-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17360587#comment-17360587 ] Apache Spark commented on SPARK-34524: -- User 'imback82' has created a pull request

[jira] [Commented] (SPARK-34524) simplify v2 partition commands resolution

2021-06-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17360586#comment-17360586 ] Apache Spark commented on SPARK-34524: -- User 'imback82' has created a pull request

[jira] [Assigned] (SPARK-34320) Migrate ALTER TABLE drop columns command to the new resolution framework

2021-06-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34320: Assignee: (was: Apache Spark) > Migrate ALTER TABLE drop columns command to the new r

[jira] [Commented] (SPARK-34320) Migrate ALTER TABLE drop columns command to the new resolution framework

2021-06-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17360578#comment-17360578 ] Apache Spark commented on SPARK-34320: -- User 'imback82' has created a pull request

[jira] [Commented] (SPARK-34320) Migrate ALTER TABLE drop columns command to the new resolution framework

2021-06-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17360577#comment-17360577 ] Apache Spark commented on SPARK-34320: -- User 'imback82' has created a pull request

[jira] [Assigned] (SPARK-34320) Migrate ALTER TABLE drop columns command to the new resolution framework

2021-06-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-34320: Assignee: Apache Spark > Migrate ALTER TABLE drop columns command to the new resolution f

[jira] [Resolved] (SPARK-35474) Enable disallow_untyped_defs mypy check for pyspark.pandas.indexing.

2021-06-09 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-35474. --- Fix Version/s: 3.2.0 Assignee: Kevin Su Resolution: Fixed Issue resolved by

[jira] [Updated] (SPARK-35706) Consider making the ':' in STRUCT data type definition optional

2021-06-09 Thread roryqi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] roryqi updated SPARK-35706: --- Description: The STRUCT type syntax is defined like this: STRUCT(fieldNmae: fileType [NOT NULL][COMMENT str

[jira] [Created] (SPARK-35706) Consider making the ':' in STRUCT data type definition optional

2021-06-09 Thread roryqi (Jira)
roryqi created SPARK-35706: -- Summary: Consider making the ':' in STRUCT data type definition optional Key: SPARK-35706 URL: https://issues.apache.org/jira/browse/SPARK-35706 Project: Spark Issue Ty

[jira] [Resolved] (SPARK-35679) Overflow on converting valid Timestamp to Microseconds

2021-06-09 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk resolved SPARK-35679. -- Fix Version/s: 3.0.3 3.1.3 3.2.0 Resolution: Fixed Issue

[jira] [Assigned] (SPARK-35683) Fix Index.difference to avoid collect 'other' to driver side

2021-06-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35683: Assignee: Apache Spark (was: Haejoon Lee) > Fix Index.difference to avoid collect 'other

[jira] [Commented] (SPARK-35683) Fix Index.difference to avoid collect 'other' to driver side

2021-06-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17360554#comment-17360554 ] Apache Spark commented on SPARK-35683: -- User 'itholic' has created a pull request f

[jira] [Assigned] (SPARK-35683) Fix Index.difference to avoid collect 'other' to driver side

2021-06-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35683: Assignee: Haejoon Lee (was: Apache Spark) > Fix Index.difference to avoid collect 'other

[jira] [Resolved] (SPARK-35675) EnsureRequirements remove shuffle should respect PartitioningCollection

2021-06-09 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao resolved SPARK-35675. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32815 [https://github.com

[jira] [Assigned] (SPARK-35675) EnsureRequirements remove shuffle should respect PartitioningCollection

2021-06-09 Thread Kent Yao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao reassigned SPARK-35675: Assignee: XiDuo You > EnsureRequirements remove shuffle should respect PartitioningCollection > -

[jira] [Commented] (SPARK-35283) Support query some DDL with CTES

2021-06-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17360511#comment-17360511 ] Apache Spark commented on SPARK-35283: -- User 'beliefer' has created a pull request

[jira] [Commented] (SPARK-35283) Support query some DDL with CTES

2021-06-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17360512#comment-17360512 ] Apache Spark commented on SPARK-35283: -- User 'beliefer' has created a pull request

[jira] [Commented] (SPARK-35130) Construct day-time interval column from integral fields

2021-06-09 Thread copperybean (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17360484#comment-17360484 ] copperybean commented on SPARK-35130: - Hi [~maxgekk], could you review this [link P

[jira] [Comment Edited] (SPARK-35130) Construct day-time interval column from integral fields

2021-06-09 Thread copperybean (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17360484#comment-17360484 ] copperybean edited comment on SPARK-35130 at 6/10/21, 1:45 AM: ---

[jira] [Resolved] (SPARK-35705) Adjust pandas-on-spark `test_groupby_multiindex_columns` test for different pandas versions

2021-06-09 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-35705. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32851 [https://gi

[jira] [Assigned] (SPARK-34651) Improve ZSTD support

2021-06-09 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-34651: - Assignee: Dongjoon Hyun > Improve ZSTD support > > >

[jira] [Updated] (SPARK-35688) GeneratePredicate eliminate will fail in some case

2021-06-09 Thread todd.chen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] todd.chen updated SPARK-35688: -- Docs Text: code:  ```scala val sparkSession = SparkSession .builder() .config(sparkConf) .getOrCreat

[jira] [Commented] (SPARK-35699) Improve error message when creating k8s pod failed.

2021-06-09 Thread Kevin Su (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17360477#comment-17360477 ] Kevin Su commented on SPARK-35699: -- I'm working on it > Improve error message when cre

[jira] [Assigned] (SPARK-35705) Adjust pandas-on-spark `test_groupby_multiindex_columns` test for different pandas versions

2021-06-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35705: Assignee: (was: Apache Spark) > Adjust pandas-on-spark `test_groupby_multiindex_colum

[jira] [Assigned] (SPARK-35705) Adjust pandas-on-spark `test_groupby_multiindex_columns` test for different pandas versions

2021-06-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35705: Assignee: Apache Spark > Adjust pandas-on-spark `test_groupby_multiindex_columns` test fo

[jira] [Commented] (SPARK-35705) Adjust pandas-on-spark `test_groupby_multiindex_columns` test for different pandas versions

2021-06-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17360459#comment-17360459 ] Apache Spark commented on SPARK-35705: -- User 'xinrong-databricks' has created a pul

[jira] [Assigned] (SPARK-35705) Adjust pandas-on-spark `test_groupby_multiindex_columns` test for different pandas versions

2021-06-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35705: Assignee: Apache Spark > Adjust pandas-on-spark `test_groupby_multiindex_columns` test fo

[jira] [Created] (SPARK-35705) Adjust pandas-on-spark `test_groupby_multiindex_columns` test for different pandas versions

2021-06-09 Thread Xinrong Meng (Jira)
Xinrong Meng created SPARK-35705: Summary: Adjust pandas-on-spark `test_groupby_multiindex_columns` test for different pandas versions Key: SPARK-35705 URL: https://issues.apache.org/jira/browse/SPARK-35705

[jira] [Resolved] (SPARK-35601) Complete arithmetic operators involving bool literals, Series, and Index

2021-06-09 Thread Takuya Ueshin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-35601. --- Fix Version/s: 3.2.0 Assignee: Xinrong Meng Resolution: Fixed Issue resolved

[jira] [Commented] (SPARK-34920) Introduce SQLSTATE and ERRORCODE to SQL Exception

2021-06-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17360352#comment-17360352 ] Apache Spark commented on SPARK-34920: -- User 'karenfeng' has created a pull request

[jira] [Commented] (SPARK-34806) Helper class for batch Dataset.observe()

2021-06-09 Thread Enrico Minack (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17360325#comment-17360325 ] Enrico Minack commented on SPARK-34806: --- Thanks for the input [~cloud_fan] , I hav

[jira] [Commented] (SPARK-35704) Support fields by the day-time interval type

2021-06-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17360320#comment-17360320 ] Apache Spark commented on SPARK-35704: -- User 'MaxGekk' has created a pull request f

[jira] [Assigned] (SPARK-35704) Support fields by the day-time interval type

2021-06-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35704: Assignee: Apache Spark (was: Max Gekk) > Support fields by the day-time interval type >

[jira] [Assigned] (SPARK-35704) Support fields by the day-time interval type

2021-06-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35704: Assignee: Max Gekk (was: Apache Spark) > Support fields by the day-time interval type >

[jira] [Updated] (SPARK-35703) Remove HashClusteredDistribution

2021-06-09 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated SPARK-35703: - Description: Currently Spark has {{HashClusteredDistribution}} and {{ClusteredDistribution}}. The only d

[jira] [Created] (SPARK-35704) Support fields by the day-time interval type

2021-06-09 Thread Max Gekk (Jira)
Max Gekk created SPARK-35704: Summary: Support fields by the day-time interval type Key: SPARK-35704 URL: https://issues.apache.org/jira/browse/SPARK-35704 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-35680) Support fields by year-month interval type

2021-06-09 Thread Max Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Max Gekk updated SPARK-35680: - Summary: Support fields by year-month interval type (was: Support units by year-month interval type) >

[jira] [Created] (SPARK-35703) Remove HashClusteredDistribution

2021-06-09 Thread Chao Sun (Jira)
Chao Sun created SPARK-35703: Summary: Remove HashClusteredDistribution Key: SPARK-35703 URL: https://issues.apache.org/jira/browse/SPARK-35703 Project: Spark Issue Type: Improvement Co

[jira] [Commented] (SPARK-35194) Improve readability of NestingColumnAliasing

2021-06-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17360298#comment-17360298 ] Apache Spark commented on SPARK-35194: -- User 'sarutak' has created a pull request f

[jira] [Commented] (SPARK-35194) Improve readability of NestingColumnAliasing

2021-06-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17360297#comment-17360297 ] Apache Spark commented on SPARK-35194: -- User 'sarutak' has created a pull request f

[jira] [Commented] (SPARK-32709) Write Hive ORC/Parquet bucketed table with hivehash (for Hive 1,2)

2021-06-09 Thread Cheng Su (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17360287#comment-17360287 ] Cheng Su commented on SPARK-32709: -- [~spedamallu] - yes I am still working on it. It's

[jira] [Resolved] (SPARK-35698) Support casting of timestamp without time zone to strings

2021-06-09 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-35698. Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32846 [https:

[jira] [Assigned] (SPARK-35698) Support casting of timestamp without time zone to strings

2021-06-09 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang reassigned SPARK-35698: -- Assignee: Gengliang Wang > Support casting of timestamp without time zone to strings

[jira] [Assigned] (SPARK-35616) Make astype data-type-based

2021-06-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35616: Assignee: (was: Apache Spark) > Make astype data-type-based > ---

[jira] [Assigned] (SPARK-35616) Make astype data-type-based

2021-06-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35616: Assignee: Apache Spark > Make astype data-type-based > --- > >

[jira] [Commented] (SPARK-35616) Make astype data-type-based

2021-06-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17360269#comment-17360269 ] Apache Spark commented on SPARK-35616: -- User 'xinrong-databricks' has created a pul

[jira] [Assigned] (SPARK-35701) Contention on SQLConf.sqlConfEntries and SQLConf.staticConfKeys

2021-06-09 Thread Jira
[ https://issues.apache.org/jira/browse/SPARK-35701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hövell reassigned SPARK-35701: - Assignee: Haiyang Sun > Contention on SQLConf.sqlConfEntries and SQLConf.static

[jira] [Created] (SPARK-35702) Kubernetes: Allow different namespaces for Driver and Executors

2021-06-09 Thread Christian Thiel (Jira)
Christian Thiel created SPARK-35702: --- Summary: Kubernetes: Allow different namespaces for Driver and Executors Key: SPARK-35702 URL: https://issues.apache.org/jira/browse/SPARK-35702 Project: Spark

[jira] [Assigned] (SPARK-34382) ANSI SQL: LATERAL derived table(T491)

2021-06-09 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-34382: --- Assignee: Allison Wang > ANSI SQL: LATERAL derived table(T491) > --

[jira] [Resolved] (SPARK-34382) ANSI SQL: LATERAL derived table(T491)

2021-06-09 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-34382. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32303 [https://gith

[jira] [Commented] (SPARK-32709) Write Hive ORC/Parquet bucketed table with hivehash (for Hive 1,2)

2021-06-09 Thread Shashank Pedamallu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17360179#comment-17360179 ] Shashank Pedamallu commented on SPARK-32709: [~chengsu], Is this actively be

[jira] [Assigned] (SPARK-35698) Support casting of timestamp without time zone to strings

2021-06-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35698: Assignee: (was: Apache Spark) > Support casting of timestamp without time zone to str

[jira] [Assigned] (SPARK-35698) Support casting of timestamp without time zone to strings

2021-06-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35698: Assignee: Apache Spark > Support casting of timestamp without time zone to strings >

[jira] [Commented] (SPARK-35698) Support casting of timestamp without time zone to strings

2021-06-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17360177#comment-17360177 ] Apache Spark commented on SPARK-35698: -- User 'gengliangwang' has created a pull req

[jira] [Resolved] (SPARK-35423) The output of PCA is inconsistent

2021-06-09 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-35423. -- Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32734 [https://gi

[jira] [Updated] (SPARK-35423) The output of PCA is inconsistent

2021-06-09 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen updated SPARK-35423: - Priority: Minor (was: Major) > The output of PCA is inconsistent >

[jira] [Assigned] (SPARK-35423) The output of PCA is inconsistent

2021-06-09 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35423?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-35423: Assignee: shahid > The output of PCA is inconsistent > -

[jira] [Resolved] (SPARK-35697) Test TimestampWithoutTZType as ordered and atomic type

2021-06-09 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-35697. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32843 [https://gith

[jira] [Commented] (SPARK-35691) addFile/addJar/addDirectory should put CanonicalFile

2021-06-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17360118#comment-17360118 ] Apache Spark commented on SPARK-35691: -- User 'pingsutw' has created a pull request

[jira] [Commented] (SPARK-35691) addFile/addJar/addDirectory should put CanonicalFile

2021-06-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17360117#comment-17360117 ] Apache Spark commented on SPARK-35691: -- User 'pingsutw' has created a pull request

[jira] [Created] (SPARK-35701) Contention on SQLConf.sqlConfEntries and SQLConf.staticConfKeys

2021-06-09 Thread Haiyang Sun (Jira)
Haiyang Sun created SPARK-35701: --- Summary: Contention on SQLConf.sqlConfEntries and SQLConf.staticConfKeys Key: SPARK-35701 URL: https://issues.apache.org/jira/browse/SPARK-35701 Project: Spark

[jira] [Assigned] (SPARK-35691) addFile/addJar/addDirectory should put CanonicalFile

2021-06-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35691: Assignee: Apache Spark > addFile/addJar/addDirectory should put CanonicalFile > -

[jira] [Assigned] (SPARK-35691) addFile/addJar/addDirectory should put CanonicalFile

2021-06-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35691: Assignee: (was: Apache Spark) > addFile/addJar/addDirectory should put CanonicalFile

[jira] [Commented] (SPARK-35625) Spark on k8s zombie executors

2021-06-09 Thread Liran (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17360111#comment-17360111 ] Liran commented on SPARK-35625: --- We will do the update soon to verify if it was fixed, in

[jira] [Resolved] (SPARK-35661) Allow deserialized off-heap memory entry

2021-06-09 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-35661. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32800 [https://gith

[jira] [Assigned] (SPARK-35661) Allow deserialized off-heap memory entry

2021-06-09 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-35661: --- Assignee: Wenchen Fan > Allow deserialized off-heap memory entry >

[jira] [Updated] (SPARK-35700) spark.sql.orc.filterPushdown not working with Spark 3.1.1 for tables with varchar data type

2021-06-09 Thread Arghya Saha (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arghya Saha updated SPARK-35700: Description: We are not able to upgrade to Spark 3.1.1 from Spark 2.4.x as the join on varchar co

[jira] [Resolved] (SPARK-35693) Add plan check for stream-stream join unit test

2021-06-09 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-35693. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32836 [https://gith

[jira] [Updated] (SPARK-35700) spark.sql.orc.filterPushdown not working with Spark 3.1.1 for tables with varchar data type

2021-06-09 Thread Arghya Saha (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arghya Saha updated SPARK-35700: Description: We are not able to upgrade to Spark 3.1.1 from Spark 2.4.x as the join on varchar co

[jira] [Updated] (SPARK-35700) spark.sql.orc.filterPushdown not working with Spark 3.1.1 for tables with varchar data type

2021-06-09 Thread Arghya Saha (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arghya Saha updated SPARK-35700: Description: We are not able to upgrade to Spark 3.1.1 from Spark 2.4.x as the join on varchar co

[jira] [Created] (SPARK-35700) spark.sql.orc.filterPushdown not working with Spark 3.1.1 for tables with varchar(12) data type

2021-06-09 Thread Arghya Saha (Jira)
Arghya Saha created SPARK-35700: --- Summary: spark.sql.orc.filterPushdown not working with Spark 3.1.1 for tables with varchar(12) data type Key: SPARK-35700 URL: https://issues.apache.org/jira/browse/SPARK-35700

[jira] [Updated] (SPARK-35700) spark.sql.orc.filterPushdown not working with Spark 3.1.1 for tables with varchar data type

2021-06-09 Thread Arghya Saha (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arghya Saha updated SPARK-35700: Environment: Spark 3.1.1 on K8S (was: **Spark 3.1.1 on K8S**) > spark.sql.orc.filterPushdown not

[jira] [Updated] (SPARK-35700) spark.sql.orc.filterPushdown not working with Spark 3.1.1 for tables with varchar data type

2021-06-09 Thread Arghya Saha (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arghya Saha updated SPARK-35700: Summary: spark.sql.orc.filterPushdown not working with Spark 3.1.1 for tables with varchar data ty

[jira] [Created] (SPARK-35699) Improve error message when creating k8s pod failed.

2021-06-09 Thread Kevin Su (Jira)
Kevin Su created SPARK-35699: Summary: Improve error message when creating k8s pod failed. Key: SPARK-35699 URL: https://issues.apache.org/jira/browse/SPARK-35699 Project: Spark Issue Type: Impro

[jira] [Commented] (SPARK-29626) notEqual() should return true when the one is null, the other is not null

2021-06-09 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17360071#comment-17360071 ] Wenchen Fan commented on SPARK-29626: - Note: once there is a null value, both `eq` a

[jira] [Commented] (SPARK-29626) notEqual() should return true when the one is null, the other is not null

2021-06-09 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17360070#comment-17360070 ] Wenchen Fan commented on SPARK-29626: - Yea this is the expected behavior. Perhaps we

[jira] [Commented] (SPARK-35697) Test TimestampWithoutTZType as ordered and atomic type

2021-06-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17360065#comment-17360065 ] Apache Spark commented on SPARK-35697: -- User 'gengliangwang' has created a pull req

[jira] [Assigned] (SPARK-35697) Test TimestampWithoutTZType as ordered and atomic type

2021-06-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35697: Assignee: (was: Apache Spark) > Test TimestampWithoutTZType as ordered and atomic typ

[jira] [Assigned] (SPARK-35697) Test TimestampWithoutTZType as ordered and atomic type

2021-06-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35697: Assignee: Apache Spark > Test TimestampWithoutTZType as ordered and atomic type > ---

[jira] [Assigned] (SPARK-35697) Test TimestampWithoutTZType as ordered and atomic type

2021-06-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35697: Assignee: Apache Spark > Test TimestampWithoutTZType as ordered and atomic type > ---

[jira] [Assigned] (SPARK-35390) Handle type coercion when resolving V2 functions

2021-06-09 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-35390: --- Assignee: Chao Sun > Handle type coercion when resolving V2 functions > --

[jira] [Resolved] (SPARK-35390) Handle type coercion when resolving V2 functions

2021-06-09 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-35390. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32764 [https://gith

[jira] [Resolved] (SPARK-35650) Coalesce small output files through AQE

2021-06-09 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-35650. - Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32781 [https://gith

[jira] [Assigned] (SPARK-35650) Coalesce small output files through AQE

2021-06-09 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-35650: --- Assignee: Yuming Wang > Coalesce small output files through AQE > -

[jira] [Created] (SPARK-35698) Support casting of timestamp without time zone to strings

2021-06-09 Thread Gengliang Wang (Jira)
Gengliang Wang created SPARK-35698: -- Summary: Support casting of timestamp without time zone to strings Key: SPARK-35698 URL: https://issues.apache.org/jira/browse/SPARK-35698 Project: Spark

[jira] [Created] (SPARK-35697) Test TimestampWithoutTZType as ordered and atomic type

2021-06-09 Thread Gengliang Wang (Jira)
Gengliang Wang created SPARK-35697: -- Summary: Test TimestampWithoutTZType as ordered and atomic type Key: SPARK-35697 URL: https://issues.apache.org/jira/browse/SPARK-35697 Project: Spark Is

[jira] [Resolved] (SPARK-35694) Increase the default JVM stack size of SBT/Maven

2021-06-09 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-35694. Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32838 [https:

[jira] [Assigned] (SPARK-35694) Increase the default JVM stack size of SBT/Maven

2021-06-09 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang reassigned SPARK-35694: -- Assignee: Gengliang Wang > Increase the default JVM stack size of SBT/Maven > ---

[jira] [Comment Edited] (SPARK-35667) spark.speculation causes incorrect query results with TRANSFORM

2021-06-09 Thread yuanxm (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17359710#comment-17359710 ] yuanxm edited comment on SPARK-35667 at 6/9/21, 11:36 AM: -- Trie

[jira] [Assigned] (SPARK-35674) Test timestamp without time zone in UDF

2021-06-09 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang reassigned SPARK-35674: -- Assignee: Gengliang Wang > Test timestamp without time zone in UDF >

[jira] [Resolved] (SPARK-35674) Test timestamp without time zone in UDF

2021-06-09 Thread Gengliang Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-35674. Fix Version/s: 3.2.0 Resolution: Fixed Issue resolved by pull request 32840 [https:

[jira] [Commented] (SPARK-29626) notEqual() should return true when the one is null, the other is not null

2021-06-09 Thread dc-heros (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17359900#comment-17359900 ] dc-heros commented on SPARK-29626: -- [~cloud_fan] could you give your opinion?   > not

[jira] [Assigned] (SPARK-35673) Spark fails on unrecognized hint in subquery

2021-06-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35673: Assignee: Apache Spark > Spark fails on unrecognized hint in subquery > -

[jira] [Assigned] (SPARK-35673) Spark fails on unrecognized hint in subquery

2021-06-09 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-35673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-35673: Assignee: (was: Apache Spark) > Spark fails on unrecognized hint in subquery > --

  1   2   >