[jira] [Commented] (SPARK-30989) TABLE.COLUMN reference doesn't work with new columns created by UDF

2020-03-20 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17063779#comment-17063779 ] L. C. Hsieh commented on SPARK-30989: - The analyzed query plan of {{df2}}: {code} == Analyzed

[jira] [Commented] (SPARK-31210) An issue for Spark SQL LIKE-with-ESCAPE clause

2020-03-20 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17063743#comment-17063743 ] jiaan.geng commented on SPARK-31210: spark-sql> create table test_table_like2 (subject string) ROW

[jira] [Commented] (SPARK-31210) An issue for Spark SQL LIKE-with-ESCAPE clause

2020-03-20 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17063744#comment-17063744 ] jiaan.geng commented on SPARK-31210: spark-sql> select * from test_table_like2 where subject like

[jira] [Commented] (SPARK-31210) An issue for Spark SQL LIKE-with-ESCAPE clause

2020-03-20 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17063742#comment-17063742 ] jiaan.geng commented on SPARK-31210: spark-sql> create or replace temporary view test_table_like as

[jira] [Comment Edited] (SPARK-31210) An issue for Spark SQL LIKE-with-ESCAPE clause

2020-03-20 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17063738#comment-17063738 ] jiaan.geng edited comment on SPARK-31210 at 3/21/20, 3:13 AM: -- You not need

[jira] [Commented] (SPARK-31210) An issue for Spark SQL LIKE-with-ESCAPE clause

2020-03-20 Thread jiaan.geng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17063738#comment-17063738 ] jiaan.geng commented on SPARK-31210: You not need use '^' as escaped char. spark-sql> select '100

[jira] [Commented] (SPARK-31183) Incompatible Avro dates/timestamps with Spark 2.4

2020-03-20 Thread koert kuipers (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17063725#comment-17063725 ] koert kuipers commented on SPARK-31183: --- hi! all spark tests pass for me except hive and avro.

[jira] [Commented] (SPARK-31210) An issue for Spark SQL LIKE-with-ESCAPE clause

2020-03-20 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17063715#comment-17063715 ] Yuming Wang commented on SPARK-31210: - cc  [~beliefer] > An issue for Spark SQL LIKE-with-ESCAPE

[jira] [Created] (SPARK-31210) An issue for Spark SQL LIKE-with-ESCAPE clause

2020-03-20 Thread Mingli Rui (Jira)
Mingli Rui created SPARK-31210: -- Summary: An issue for Spark SQL LIKE-with-ESCAPE clause Key: SPARK-31210 URL: https://issues.apache.org/jira/browse/SPARK-31210 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-31209) Not compatible with new version of scalatest (3.1.0 and above)

2020-03-20 Thread Timothy Zhang (Jira)
Timothy Zhang created SPARK-31209: - Summary: Not compatible with new version of scalatest (3.1.0 and above) Key: SPARK-31209 URL: https://issues.apache.org/jira/browse/SPARK-31209 Project: Spark

[jira] [Updated] (SPARK-20629) Copy shuffle data when nodes are being shut down

2020-03-20 Thread Holden Karau (Jira)
[ https://issues.apache.org/jira/browse/SPARK-20629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Holden Karau updated SPARK-20629: - Summary: Copy shuffle data when nodes are being shut down (was: Copy shuffle data when nodes

[jira] [Created] (SPARK-31208) Expose the ability for user to cleanup shuffle files

2020-03-20 Thread Holden Karau (Jira)
Holden Karau created SPARK-31208: Summary: Expose the ability for user to cleanup shuffle files Key: SPARK-31208 URL: https://issues.apache.org/jira/browse/SPARK-31208 Project: Spark Issue

[jira] [Updated] (SPARK-31208) Expose the ability for user to cleanup shuffle files

2020-03-20 Thread Holden Karau (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Holden Karau updated SPARK-31208: - Affects Version/s: 3.0.0 > Expose the ability for user to cleanup shuffle files >

[jira] [Created] (SPARK-31207) Ensure the total number of blocks to fetch equals to the sum of local/hostLocal/remote blocks

2020-03-20 Thread Xingbo Jiang (Jira)
Xingbo Jiang created SPARK-31207: Summary: Ensure the total number of blocks to fetch equals to the sum of local/hostLocal/remote blocks Key: SPARK-31207 URL: https://issues.apache.org/jira/browse/SPARK-31207

[jira] [Commented] (SPARK-31139) Fileformat datasources (ORC, Json) case sensitivity regressions

2020-03-20 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17063489#comment-17063489 ] L. C. Hsieh commented on SPARK-31139: - Thanks [~dongjoon] > Fileformat datasources (ORC, Json) case

[jira] [Resolved] (SPARK-31139) Fileformat datasources (ORC, Json) case sensitivity regressions

2020-03-20 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-31139. --- Resolution: Invalid I close this blocker issue as `Invalid` because this is not a

[jira] [Commented] (SPARK-31139) Fileformat datasources (ORC, Json) case sensitivity regressions

2020-03-20 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17063479#comment-17063479 ] Dongjoon Hyun commented on SPARK-31139: --- Hi, [~smilegator] and [~viirya] and [~kimtkyeom]. This

[jira] [Resolved] (SPARK-30773) Support NativeBlas for level-1 routines

2020-03-20 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-30773. -- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27546

[jira] [Assigned] (SPARK-30773) Support NativeBlas for level-1 routines

2020-03-20 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-30773: Assignee: Yan Ma > Support NativeBlas for level-1 routines >

[jira] [Assigned] (SPARK-31138) Add ANOVASelector

2020-03-20 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-31138: Assignee: Huaxin Gao > Add ANOVASelector > - > > Key:

[jira] [Resolved] (SPARK-31138) Add ANOVASelector

2020-03-20 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-31138. -- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 27895

[jira] [Commented] (SPARK-31204) HiveResult compatibility for DatasourceV2 command

2020-03-20 Thread Terry Kim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17063434#comment-17063434 ] Terry Kim commented on SPARK-31204: --- Yes, I will take a look. > HiveResult compatibility for

[jira] [Created] (SPARK-31206) AQE will use the same SubqueryExec even if subqueryReuseEnabled=false

2020-03-20 Thread wuyi (Jira)
wuyi created SPARK-31206: Summary: AQE will use the same SubqueryExec even if subqueryReuseEnabled=false Key: SPARK-31206 URL: https://issues.apache.org/jira/browse/SPARK-31206 Project: Spark Issue

[jira] [Commented] (SPARK-31204) HiveResult compatibility for DatasourceV2 command

2020-03-20 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17063397#comment-17063397 ] Wenchen Fan commented on SPARK-31204: - Hi [~imback82] can you take a look? I think we miss to handle

[jira] [Assigned] (SPARK-31189) Fix errors and missing parts for datetime pattern document

2020-03-20 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-31189: --- Assignee: Kent Yao > Fix errors and missing parts for datetime pattern document >

[jira] [Resolved] (SPARK-31189) Fix errors and missing parts for datetime pattern document

2020-03-20 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-31189. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27956

[jira] [Updated] (SPARK-31165) Multiple wrong references in Dockerfile for k8s

2020-03-20 Thread Nikolay Dimolarov (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikolay Dimolarov updated SPARK-31165: -- Description: I am currently trying to follow the k8s instructions for Spark: 

[jira] [Updated] (SPARK-31147) forbid CHAR type in non-Hive-Serde tables

2020-03-20 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-31147: Summary: forbid CHAR type in non-Hive-Serde tables (was: forbid CHAR/VARCHAR type in non-Hive

[jira] [Commented] (SPARK-31191) Spark SQL and hive metastore are incompatible

2020-03-20 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17063314#comment-17063314 ] Yuming Wang commented on SPARK-31191: - Please set spark.sql.hive.metastore.version to 2.3.0.

[jira] [Updated] (SPARK-31191) Spark SQL and hive metastore are incompatible

2020-03-20 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-31191: Fix Version/s: (was: 2.3.0) > Spark SQL and hive metastore are incompatible >

[jira] [Created] (SPARK-31205) support string literal as the second argument of date_add/date_sub functions

2020-03-20 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-31205: --- Summary: support string literal as the second argument of date_add/date_sub functions Key: SPARK-31205 URL: https://issues.apache.org/jira/browse/SPARK-31205 Project:

[jira] [Created] (SPARK-31204) HiveResult compatibility for DatasourceV2 command

2020-03-20 Thread Juliusz Sompolski (Jira)
Juliusz Sompolski created SPARK-31204: - Summary: HiveResult compatibility for DatasourceV2 command Key: SPARK-31204 URL: https://issues.apache.org/jira/browse/SPARK-31204 Project: Spark

[jira] [Commented] (SPARK-31203) Upgrade derby to 10.14.2.0 from 10.12.1.1

2020-03-20 Thread Udbhav Agrawal (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17063278#comment-17063278 ] Udbhav Agrawal commented on SPARK-31203: I will raise PR for the same > Upgrade derby to

[jira] [Created] (SPARK-31203) Upgrade derby to 10.14.2.0 from 10.12.1.1

2020-03-20 Thread Udbhav Agrawal (Jira)
Udbhav Agrawal created SPARK-31203: -- Summary: Upgrade derby to 10.14.2.0 from 10.12.1.1 Key: SPARK-31203 URL: https://issues.apache.org/jira/browse/SPARK-31203 Project: Spark Issue Type:

[jira] [Created] (SPARK-31202) Improve SizeEstimator for AppendOnlyMap

2020-03-20 Thread liupengcheng (Jira)
liupengcheng created SPARK-31202: Summary: Improve SizeEstimator for AppendOnlyMap Key: SPARK-31202 URL: https://issues.apache.org/jira/browse/SPARK-31202 Project: Spark Issue Type:

[jira] [Created] (SPARK-31201) add an individual config for skewed partition threshold

2020-03-20 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-31201: --- Summary: add an individual config for skewed partition threshold Key: SPARK-31201 URL: https://issues.apache.org/jira/browse/SPARK-31201 Project: Spark Issue

[jira] [Updated] (SPARK-31200) Docker image build fails with Mirror sync in progress? errors.

2020-03-20 Thread Prashant Sharma (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Sharma updated SPARK-31200: Description: Following errors would appear, in spite of trying to switch between various

[jira] [Created] (SPARK-31200) Docker image build fails with Mirror sync in progress? errors.

2020-03-20 Thread Prashant Sharma (Jira)
Prashant Sharma created SPARK-31200: --- Summary: Docker image build fails with Mirror sync in progress? errors. Key: SPARK-31200 URL: https://issues.apache.org/jira/browse/SPARK-31200 Project: Spark

[jira] [Commented] (SPARK-31139) Fileformat datasources (ORC, Json) case sensitivity regressions

2020-03-20 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17063160#comment-17063160 ] L. C. Hsieh commented on SPARK-31139: - Thanks for pinging me. I will look at this in this weekend,

[jira] [Resolved] (SPARK-30951) Potential data loss for legacy applications after switch to proleptic Gregorian calendar

2020-03-20 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-30951. - Fix Version/s: 3.0.0 Assignee: Maxim Gekk Resolution: Fixed I'm closing it as

[jira] [Resolved] (SPARK-31183) Incompatible Avro dates/timestamps with Spark 2.4

2020-03-20 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-31183. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27953

[jira] [Assigned] (SPARK-31183) Incompatible Avro dates/timestamps with Spark 2.4

2020-03-20 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-31183: --- Assignee: Maxim Gekk > Incompatible Avro dates/timestamps with Spark 2.4 >

[jira] [Resolved] (SPARK-31195) Reuse days rebase functions of DateTimeUtils in DaysWritable

2020-03-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31195. -- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27962

[jira] [Assigned] (SPARK-31195) Reuse days rebase functions of DateTimeUtils in DaysWritable

2020-03-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-31195: Assignee: Maxim Gekk > Reuse days rebase functions of DateTimeUtils in DaysWritable >

[jira] [Commented] (SPARK-31139) Fileformat datasources (ORC, Json) case sensitivity regressions

2020-03-20 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17063127#comment-17063127 ] Xiao Li commented on SPARK-31139: - ping [~viirya] [~dongjoon] > Fileformat datasources (ORC, Json) case