[jira] [Resolved] (SPARK-33306) TimezoneID is needed when there cast from Date to String

2020-10-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-33306. --- Fix Version/s: 3.0.2 3.1.0 Resolution: Fixed Issue resolved by

[jira] [Updated] (SPARK-33290) REFRESH TABLE should invalidate cache even though the table itself may not be cached

2020-10-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33290: -- Fix Version/s: 2.4.8 > REFRESH TABLE should invalidate cache even though the table itself may

[jira] [Updated] (SPARK-29574) spark with user provided hadoop doesn't work on kubernetes

2020-10-31 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh updated SPARK-29574: Fix Version/s: 2.4.8 > spark with user provided hadoop doesn't work on kubernetes >

[jira] [Updated] (SPARK-33311) Improve semantics for REFRESH TABLE

2020-10-31 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated SPARK-33311: - Description: Currently, the semantics for {{REFRESH TABLE t}} is not well defined for view (let's say

[jira] [Updated] (SPARK-33311) Improve semantics for REFRESH TABLE

2020-10-31 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated SPARK-33311: - Description: Currently, the semantics for {{REFRESH TABLE t}} is not well defined for view (let's say

[jira] [Created] (SPARK-33311) Improve semantics for REFRESH TABLE

2020-10-31 Thread Chao Sun (Jira)
Chao Sun created SPARK-33311: Summary: Improve semantics for REFRESH TABLE Key: SPARK-33311 URL: https://issues.apache.org/jira/browse/SPARK-33311 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-33259) Joining 3 streams results in incorrect output

2020-10-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17224170#comment-17224170 ] Dongjoon Hyun commented on SPARK-33259: --- According to the JIRA report, I added `correctness` label

[jira] [Updated] (SPARK-33259) Joining 3 streams results in incorrect output

2020-10-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33259: -- Labels: correctness (was: ) > Joining 3 streams results in incorrect output >

[jira] [Updated] (SPARK-33290) REFRESH TABLE should invalidate cache even though the table itself may not be cached

2020-10-31 Thread Chao Sun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated SPARK-33290: - Affects Version/s: 2.4.7 > REFRESH TABLE should invalidate cache even though the table itself may not

[jira] [Commented] (SPARK-33290) REFRESH TABLE should invalidate cache even though the table itself may not be cached

2020-10-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17224168#comment-17224168 ] Apache Spark commented on SPARK-33290: -- User 'sunchao' has created a pull request for this issue:

[jira] [Updated] (SPARK-33306) TimezoneID is needed when there cast from Date to String

2020-10-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33306: -- Description: A simple way to reproduce this is  {code} spark-shell --conf 

[jira] [Assigned] (SPARK-33306) TimezoneID is needed when there cast from Date to String

2020-10-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-33306: - Assignee: EdisonWang > TimezoneID is needed when there cast from Date to String >

[jira] [Updated] (SPARK-33306) TimezoneID is needed when there cast from Date to String

2020-10-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-33306: -- Affects Version/s: 3.1.0 3.0.1 > TimezoneID is needed when there cast

[jira] [Commented] (SPARK-29574) spark with user provided hadoop doesn't work on kubernetes

2020-10-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17224165#comment-17224165 ] Apache Spark commented on SPARK-29574: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Closed] (SPARK-33271) Hadoop Free Build Setup for Spark 2.4.7 on Kubernetes

2020-10-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-33271. - > Hadoop Free Build Setup for Spark 2.4.7 on Kubernetes >

[jira] [Resolved] (SPARK-33271) Hadoop Free Build Setup for Spark 2.4.7 on Kubernetes

2020-10-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-33271. --- Resolution: Duplicate > Hadoop Free Build Setup for Spark 2.4.7 on Kubernetes >

[jira] [Assigned] (SPARK-33293) Refactor WriteToDataSourceV2Exec and reduce code duplication

2020-10-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-33293: - Assignee: Chao Sun > Refactor WriteToDataSourceV2Exec and reduce code duplication >

[jira] [Resolved] (SPARK-33293) Refactor WriteToDataSourceV2Exec and reduce code duplication

2020-10-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-33293. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 30193

[jira] [Assigned] (SPARK-33290) REFRESH TABLE should invalidate cache even though the table itself may not be cached

2020-10-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-33290: - Assignee: Chao Sun > REFRESH TABLE should invalidate cache even though the table

[jira] [Resolved] (SPARK-33290) REFRESH TABLE should invalidate cache even though the table itself may not be cached

2020-10-31 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-33290. --- Fix Version/s: 3.0.2 3.1.0 Resolution: Fixed Issue resolved by

[jira] [Updated] (SPARK-33310) Relax pyspark typing for sql str functions

2020-10-31 Thread Daniel Himmelstein (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Himmelstein updated SPARK-33310: --- Description: Several pyspark.sql.functions have overly strict typing, in that the

[jira] [Commented] (SPARK-33310) Relax pyspark typing for sql str functions

2020-10-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17224074#comment-17224074 ] Apache Spark commented on SPARK-33310: -- User 'dhimmel' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33310) Relax pyspark typing for sql str functions

2020-10-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33310: Assignee: Apache Spark > Relax pyspark typing for sql str functions >

[jira] [Commented] (SPARK-33310) Relax pyspark typing for sql str functions

2020-10-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17224072#comment-17224072 ] Apache Spark commented on SPARK-33310: -- User 'dhimmel' has created a pull request for this issue:

[jira] [Assigned] (SPARK-33310) Relax pyspark typing for sql str functions

2020-10-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33310: Assignee: (was: Apache Spark) > Relax pyspark typing for sql str functions >

[jira] [Created] (SPARK-33310) Relax pyspark typing for sql str functions

2020-10-31 Thread Daniel Himmelstein (Jira)
Daniel Himmelstein created SPARK-33310: -- Summary: Relax pyspark typing for sql str functions Key: SPARK-33310 URL: https://issues.apache.org/jira/browse/SPARK-33310 Project: Spark Issue

[jira] [Updated] (SPARK-33303) Deduplicate deterministic PythonUDF calls

2020-10-31 Thread Peter Toth (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Toth updated SPARK-33303: --- Description: We run into an issue where a customer created a column with an expensive PythonUDF

[jira] [Updated] (SPARK-33303) Deduplicate deterministic PythonUDF calls

2020-10-31 Thread Peter Toth (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Toth updated SPARK-33303: --- Summary: Deduplicate deterministic PythonUDF calls (was: Deduplicate deterministic UDF calls) >

[jira] [Commented] (SPARK-33306) TimezoneID is needed when there cast from Date to String

2020-10-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17224029#comment-17224029 ] Apache Spark commented on SPARK-33306: -- User 'WangGuangxin' has created a pull request for this

[jira] [Updated] (SPARK-33306) TimezoneID is needed when there cast from Date to String

2020-10-31 Thread EdisonWang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] EdisonWang updated SPARK-33306: --- Description: A simple way to reproduce this is  ``` spark-shell --conf 

[jira] [Assigned] (SPARK-33306) TimezoneID is needed when there cast from Date to String

2020-10-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33306: Assignee: (was: Apache Spark) > TimezoneID is needed when there cast from Date to

[jira] [Assigned] (SPARK-33306) TimezoneID is needed when there cast from Date to String

2020-10-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-33306: Assignee: Apache Spark > TimezoneID is needed when there cast from Date to String >

[jira] [Commented] (SPARK-33306) TimezoneID is needed when there cast from Date to String

2020-10-31 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17224028#comment-17224028 ] Apache Spark commented on SPARK-33306: -- User 'WangGuangxin' has created a pull request for this

[jira] [Updated] (SPARK-33306) TimezoneID is needed when there cast from Date to String

2020-10-31 Thread EdisonWang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-33306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] EdisonWang updated SPARK-33306: --- Description: A simple way to reproduce this is  ``` spark-shell --conf 

[jira] [Commented] (SPARK-29606) Improve EliminateOuterJoin performance

2020-10-31 Thread Asif (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17224007#comment-17224007 ] Asif commented on SPARK-29606: -- Have proposed following PR which completely solves the issue.