[jira] [Updated] (SPARK-30034) Use Apache Hive 2.3 dependency by default

[ https://issues.apache.org/jira/browse/SPARK-30034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30034: -- Labels: correctness release-notes (was: release-notes) > Use Apache Hive 2.3 dependency by de

[jira] [Updated] (SPARK-30034) Use Apache Hive 2.3 dependency by default

[ https://issues.apache.org/jira/browse/SPARK-30034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30034: -- Description: Some correctness issue like SPARK-25193 is fixed by `hive-2.3` only. > Use Apach

[jira] [Created] (SPARK-31145) spark2.4.0 read hbase2.1.0 Error

HarSenZhao created SPARK-31145: -- Summary: spark2.4.0 read hbase2.1.0 Error Key: SPARK-31145 URL: https://issues.apache.org/jira/browse/SPARK-31145 Project: Spark Issue Type: Bug Compon

[jira] [Updated] (SPARK-25193) insert overwrite doesn't throw exception when drop old data fails

[ https://issues.apache.org/jira/browse/SPARK-25193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25193: -- Affects Version/s: (was: 2.2.0) 1.6.3 2.0.2

[jira] [Updated] (SPARK-31144) Wrap java.lang.Error with an exception for QueryExecutionListener.onFailure

[ https://issues.apache.org/jira/browse/SPARK-31144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-31144: - Issue Type: Bug (was: Test) > Wrap java.lang.Error with an exception for QueryExecutionListener

[jira] [Created] (SPARK-31144) Wrap java.lang.Error with an exception for QueryExecutionListener.onFailure

Shixiong Zhu created SPARK-31144: Summary: Wrap java.lang.Error with an exception for QueryExecutionListener.onFailure Key: SPARK-31144 URL: https://issues.apache.org/jira/browse/SPARK-31144 Project:

[jira] [Created] (SPARK-31143) Spark 2.4.4 count distinct query much slower than Spark 1.6.2 and Hive 1.2.1

Kevin Ma created SPARK-31143: - Summary: Spark 2.4.4 count distinct query much slower than Spark 1.6.2 and Hive 1.2.1 Key: SPARK-31143 URL: https://issues.apache.org/jira/browse/SPARK-31143 Project: Spark

[jira] [Commented] (SPARK-31136) Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax

[ https://issues.apache.org/jira/browse/SPARK-31136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17058445#comment-17058445 ] Wenchen Fan commented on SPARK-31136: - 1. I think it's OK, and it happens a lot of t

[jira] [Commented] (SPARK-31136) Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax

[ https://issues.apache.org/jira/browse/SPARK-31136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17058438#comment-17058438 ] Dongjoon Hyun commented on SPARK-31136: --- Yes! That sounds reasonable to me. The re

[jira] [Commented] (SPARK-31136) Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax

[ https://issues.apache.org/jira/browse/SPARK-31136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17058427#comment-17058427 ] Wenchen Fan commented on SPARK-31136: - The problem is that, we don't really support

[jira] [Commented] (SPARK-25987) StackOverflowError when executing many operations on a table with many columns

[ https://issues.apache.org/jira/browse/SPARK-25987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17058424#comment-17058424 ] L. C. Hsieh commented on SPARK-25987: - Thanks [~kiszk], [~kabhwan] and [~dongjoon] f

[jira] [Comment Edited] (SPARK-31136) Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax

[ https://issues.apache.org/jira/browse/SPARK-31136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17058414#comment-17058414 ] Dongjoon Hyun edited comment on SPARK-31136 at 3/13/20, 4:35 AM: -

[jira] [Commented] (SPARK-31136) Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax

[ https://issues.apache.org/jira/browse/SPARK-31136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17058413#comment-17058413 ] Dongjoon Hyun commented on SPARK-31136: --- Also, [~kabhwan]'s comment is also valid.

[jira] [Commented] (SPARK-31136) Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax

[ https://issues.apache.org/jira/browse/SPARK-31136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17058414#comment-17058414 ] Dongjoon Hyun commented on SPARK-31136: --- [~cloud_fan]. Please see the second examp

[jira] [Commented] (SPARK-31136) Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax

[ https://issues.apache.org/jira/browse/SPARK-31136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17058412#comment-17058412 ] Wenchen Fan commented on SPARK-31136: - Can we define the standard of correctness iss

[jira] [Updated] (SPARK-31136) Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax

[ https://issues.apache.org/jira/browse/SPARK-31136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-31136: -- Description: We need to consider the behavior change of SPARK-30098 . This is a placeholder to

[jira] [Commented] (SPARK-31136) Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax

[ https://issues.apache.org/jira/browse/SPARK-31136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17058411#comment-17058411 ] Dongjoon Hyun commented on SPARK-31136: --- Hi, [~marmbrus]. I marked this as a corre

[jira] [Updated] (SPARK-31136) Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax

[ https://issues.apache.org/jira/browse/SPARK-31136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-31136: -- Description: We need to consider the behavior change of SPARK-30098 . This is a placeholder to

[jira] [Updated] (SPARK-31136) Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax

[ https://issues.apache.org/jira/browse/SPARK-31136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-31136: -- Labels: correctness (was: ) > Revert SPARK-30098 Use default datasource as provider for CREAT

[jira] [Created] (SPARK-31142) Remove useless conf set in pyspark context

Jackey Lee created SPARK-31142: -- Summary: Remove useless conf set in pyspark context Key: SPARK-31142 URL: https://issues.apache.org/jira/browse/SPARK-31142 Project: Spark Issue Type: Improvemen

[jira] [Created] (SPARK-31141) Add version information to the configuration of Dstreams

jiaan.geng created SPARK-31141: -- Summary: Add version information to the configuration of Dstreams Key: SPARK-31141 URL: https://issues.apache.org/jira/browse/SPARK-31141 Project: Spark Issue Ty

[jira] [Commented] (SPARK-30839) Add version information for Spark configuration

[ https://issues.apache.org/jira/browse/SPARK-30839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17058396#comment-17058396 ] jiaan.geng commented on SPARK-30839: There should be two or three tickets left > Ad

[jira] [Updated] (SPARK-31140) Support Quick sample in RDD

[ https://issues.apache.org/jira/browse/SPARK-31140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] deshanxiao updated SPARK-31140: --- Description: RDD.sample use the function of *filter* to pick up the data we need. It means that if

[jira] [Updated] (SPARK-31140) Support Quick sample in RDD

[ https://issues.apache.org/jira/browse/SPARK-31140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] deshanxiao updated SPARK-31140: --- Description: RDD.sample use the function of *filter* to pick up the data we need. It means that if

[jira] [Created] (SPARK-31140) Support Quick sample in RDD

deshanxiao created SPARK-31140: -- Summary: Support Quick sample in RDD Key: SPARK-31140 URL: https://issues.apache.org/jira/browse/SPARK-31140 Project: Spark Issue Type: Improvement Com

[jira] [Resolved] (SPARK-31128) Fix Uncaught TypeError in streaming statistics page

[ https://issues.apache.org/jira/browse/SPARK-31128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-31128. Fix Version/s: 3.0.0 Resolution: Fixed The issue is resolved in https://github.com/

[jira] [Comment Edited] (SPARK-31136) Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax

[ https://issues.apache.org/jira/browse/SPARK-31136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17058380#comment-17058380 ] Jungtaek Lim edited comment on SPARK-31136 at 3/13/20, 3:01 AM: --

[jira] [Commented] (SPARK-31136) Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax

[ https://issues.apache.org/jira/browse/SPARK-31136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17058380#comment-17058380 ] Jungtaek Lim commented on SPARK-31136: -- https://github.com/apache/spark/blob/master

[jira] [Resolved] (SPARK-30874) Support Postgres Kerberos login in JDBC connector

[ https://issues.apache.org/jira/browse/SPARK-30874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin resolved SPARK-30874. Fix Version/s: 3.1.0 Assignee: Gabor Somogyi Resolution: Fi

[jira] [Commented] (SPARK-25457) IntegralDivide (div) should not always return long

[ https://issues.apache.org/jira/browse/SPARK-25457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17058347#comment-17058347 ] Hyukjin Kwon commented on SPARK-25457: -- Resolved as Won't Fix. See PR description a

[jira] [Resolved] (SPARK-25457) IntegralDivide (div) should not always return long

[ https://issues.apache.org/jira/browse/SPARK-25457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-25457. -- Resolution: Won't Fix > IntegralDivide (div) should not always return long > -

[jira] [Updated] (SPARK-25457) IntegralDivide (div) should not always return long

[ https://issues.apache.org/jira/browse/SPARK-25457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-25457: - Fix Version/s: (was: 3.0.0) > IntegralDivide (div) should not always return long > -

[jira] [Reopened] (SPARK-25457) IntegralDivide (div) should not always return long

[ https://issues.apache.org/jira/browse/SPARK-25457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reopened SPARK-25457: -- Assignee: (was: Marco Gaido) Reverted at SPARK-31090 > IntegralDivide (div) should not

[jira] [Resolved] (SPARK-31090) Revert SPARK-25457 IntegralDivide returns data type of the operands

[ https://issues.apache.org/jira/browse/SPARK-31090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-31090. -- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27835 [https://gi

[jira] [Comment Edited] (SPARK-31136) Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax

[ https://issues.apache.org/jira/browse/SPARK-31136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17058333#comment-17058333 ] Jungtaek Lim edited comment on SPARK-31136 at 3/13/20, 1:34 AM: --

[jira] [Commented] (SPARK-31136) Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax

[ https://issues.apache.org/jira/browse/SPARK-31136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17058333#comment-17058333 ] Jungtaek Lim commented on SPARK-31136: -- This reminds me about my previous PR: [htt

[jira] [Updated] (SPARK-31139) Fileformat datasources (ORC, Json) case sensitivity regressions

[ https://issues.apache.org/jira/browse/SPARK-31139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tae-kyeom, Kim updated SPARK-31139: --- Description: In addition to https://issues.apache.org/jira/browse/SPARK-31116 Not only parq

[jira] [Updated] (SPARK-31139) Fileformat datasources (ORC, Json) case sensitivity regressions

[ https://issues.apache.org/jira/browse/SPARK-31139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tae-kyeom, Kim updated SPARK-31139: --- Attachment: FileBasedDataSourceSuite.scala.diff > Fileformat datasources (ORC, Json) case se

[jira] [Created] (SPARK-31139) Fileformat datasources (ORC, Json) case sensitivity regressions

Tae-kyeom, Kim created SPARK-31139: -- Summary: Fileformat datasources (ORC, Json) case sensitivity regressions Key: SPARK-31139 URL: https://issues.apache.org/jira/browse/SPARK-31139 Project: Spark

[jira] [Created] (SPARK-31138) Add ANOVASelector

Huaxin Gao created SPARK-31138: -- Summary: Add ANOVASelector Key: SPARK-31138 URL: https://issues.apache.org/jira/browse/SPARK-31138 Project: Spark Issue Type: Sub-task Components: ML

[jira] [Commented] (SPARK-31136) Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax

[ https://issues.apache.org/jira/browse/SPARK-31136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17058287#comment-17058287 ] Michael Armbrust commented on SPARK-31136: -- How hard would it be to add support

[jira] [Commented] (SPARK-31136) Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax

[ https://issues.apache.org/jira/browse/SPARK-31136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17058275#comment-17058275 ] Xiao Li commented on SPARK-31136: - Yes. The impact of this change is big. As [~dongjoon]

[jira] [Assigned] (SPARK-31129) IntervalBenchmark and DateTimeBenchmark fails to run

[ https://issues.apache.org/jira/browse/SPARK-31129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-31129: - Assignee: Kent Yao > IntervalBenchmark and DateTimeBenchmark fails to run > ---

[jira] [Resolved] (SPARK-31129) IntervalBenchmark and DateTimeBenchmark fails to run

[ https://issues.apache.org/jira/browse/SPARK-31129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-31129. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27885 [https://

[jira] [Created] (SPARK-31137) Opportunity to simplify execution plan when passing empty dataframes to subtract()

S Daniel Zafar created SPARK-31137: -- Summary: Opportunity to simplify execution plan when passing empty dataframes to subtract() Key: SPARK-31137 URL: https://issues.apache.org/jira/browse/SPARK-31137

[jira] [Updated] (SPARK-31135) Upgrdade docker-client version to 8.14.1

[ https://issues.apache.org/jira/browse/SPARK-31135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-31135: -- Summary: Upgrdade docker-client version to 8.14.1 (was: Upgrdade docker-client version) > Up

[jira] [Updated] (SPARK-31136) Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax

[ https://issues.apache.org/jira/browse/SPARK-31136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-31136: -- Description: We need to consider the behavior change of SPARK-30098 . This is a placeholder to

[jira] [Updated] (SPARK-31136) Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax

[ https://issues.apache.org/jira/browse/SPARK-31136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-31136: -- Priority: Blocker (was: Major) > Revert SPARK-30098 Use default datasource as provider for CR

[jira] [Updated] (SPARK-31136) Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax

[ https://issues.apache.org/jira/browse/SPARK-31136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-31136: -- Description: We need to consider the behavior change of SPARK-30098 . This is a placeholder to

[jira] [Commented] (SPARK-31136) Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax

[ https://issues.apache.org/jira/browse/SPARK-31136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17058194#comment-17058194 ] Dongjoon Hyun commented on SPARK-31136: --- [~marmbrus]. I updated the PR description

[jira] [Updated] (SPARK-31136) Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax

[ https://issues.apache.org/jira/browse/SPARK-31136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-31136: -- Description: We need to consider the behavior change of SPARK-30098 . This is a placeholder to

[jira] [Updated] (SPARK-31128) Fix Uncaught TypeError in streaming statistics page

[ https://issues.apache.org/jira/browse/SPARK-31128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-31128: --- Issue Type: Bug (was: Improvement) > Fix Uncaught TypeError in streaming statistics page >

[jira] [Commented] (SPARK-25987) StackOverflowError when executing many operations on a table with many columns

[ https://issues.apache.org/jira/browse/SPARK-25987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17058181#comment-17058181 ] Dongjoon Hyun commented on SPARK-25987: --- Thank you so much for the analysis, [~kab

[jira] [Commented] (SPARK-25987) StackOverflowError when executing many operations on a table with many columns

[ https://issues.apache.org/jira/browse/SPARK-25987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17058174#comment-17058174 ] Kazuaki Ishizaki commented on SPARK-25987: -- Sorry for coming here late since I

[jira] [Updated] (SPARK-30098) Use default datasource as provider for CREATE TABLE syntax

[ https://issues.apache.org/jira/browse/SPARK-30098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30098: -- Labels: release-notes (was: ) > Use default datasource as provider for CREATE TABLE syntax >

[jira] [Commented] (SPARK-31136) Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax

[ https://issues.apache.org/jira/browse/SPARK-31136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17058125#comment-17058125 ] Michael Armbrust commented on SPARK-31136: -- What was the default before, hive s

[jira] [Commented] (SPARK-31136) Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax

[ https://issues.apache.org/jira/browse/SPARK-31136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17058099#comment-17058099 ] Dongjoon Hyun commented on SPARK-31136: --- cc [~cloud_fan] and [~smilegator] > Reve

[jira] [Updated] (SPARK-31136) Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax

[ https://issues.apache.org/jira/browse/SPARK-31136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-31136: -- Description: We need to consider the behavior change of SPARK-30098 . This is a placeholder to

[jira] [Created] (SPARK-31136) Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax

Dongjoon Hyun created SPARK-31136: - Summary: Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax Key: SPARK-31136 URL: https://issues.apache.org/jira/browse/SPARK-31136 Proje

[jira] [Resolved] (SPARK-31131) Remove the unnecessary config spark.sql.legacy.timeParser.enabled

[ https://issues.apache.org/jira/browse/SPARK-31131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-31131. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27889 [https://

[jira] [Assigned] (SPARK-31131) Remove the unnecessary config spark.sql.legacy.timeParser.enabled

[ https://issues.apache.org/jira/browse/SPARK-31131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-31131: - Assignee: Kent Yao > Remove the unnecessary config spark.sql.legacy.timeParser.enabled

[jira] [Updated] (SPARK-31135) Upgrdade docker-client version

[ https://issues.apache.org/jira/browse/SPARK-31135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Somogyi updated SPARK-31135: -- Component/s: SQL Build > Upgrdade docker-client version > ---

[jira] [Created] (SPARK-31135) Upgrdade docker-client version

Gabor Somogyi created SPARK-31135: - Summary: Upgrdade docker-client version Key: SPARK-31135 URL: https://issues.apache.org/jira/browse/SPARK-31135 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-31134) optimize skew join after shuffle partitions are coalesced

Wenchen Fan created SPARK-31134: --- Summary: optimize skew join after shuffle partitions are coalesced Key: SPARK-31134 URL: https://issues.apache.org/jira/browse/SPARK-31134 Project: Spark Issue

[jira] [Resolved] (SPARK-31130) Use the same version of `commons-io` in SBT

[ https://issues.apache.org/jira/browse/SPARK-31130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-31130. --- Fix Version/s: 3.0.0 2.4.6 Resolution: Fixed Issue resolved by pul

[jira] [Created] (SPARK-31133) fix sql ref doc for DML

Wenchen Fan created SPARK-31133: --- Summary: fix sql ref doc for DML Key: SPARK-31133 URL: https://issues.apache.org/jira/browse/SPARK-31133 Project: Spark Issue Type: Documentation Com

[jira] [Resolved] (SPARK-31124) change the default value of minPartitionNum in AQE

[ https://issues.apache.org/jira/browse/SPARK-31124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-31124. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27879 [https://gith

[jira] [Updated] (SPARK-31132) Optimized Logical Plan cast('' as timestamp) is null

[ https://issues.apache.org/jira/browse/SPARK-31132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] duyf updated SPARK-31132: - Summary: Optimized Logical Plan cast('' as timestamp) is null (was: Optimized Logical Plan (receive_time<>'')

[jira] [Updated] (SPARK-31132) Optimized Logical Plan (receive_time<>'') is null

[ https://issues.apache.org/jira/browse/SPARK-31132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] duyf updated SPARK-31132: - Summary: Optimized Logical Plan (receive_time<>'') is null (was: Optimized Logical Plan cast('' as timestamp

[jira] [Updated] (SPARK-31132) Optimized Logical Plan cast('' as timestamp) is null

[ https://issues.apache.org/jira/browse/SPARK-31132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] duyf updated SPARK-31132: - Description: The type of receive_time is timestamp，The value is  '2020-02-10' SQL： select case when receive_ti

[jira] [Resolved] (SPARK-31066) Disable useless and uncleaned hive SessionState initialization parts

[ https://issues.apache.org/jira/browse/SPARK-31066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-31066. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27827 [https://gith

[jira] [Assigned] (SPARK-31066) Disable useless and uncleaned hive SessionState initialization parts

[ https://issues.apache.org/jira/browse/SPARK-31066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-31066: --- Assignee: Kent Yao > Disable useless and uncleaned hive SessionState initialization parts >

[jira] [Updated] (SPARK-31132) Optimized Logical Plan cast('' as timestamp) is null

[ https://issues.apache.org/jira/browse/SPARK-31132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] duyf updated SPARK-31132: - Description: receive_time:timestamp='2020-02-10'； SQL： select case when receive_time<>'' then 1 else 0 end as

[jira] [Updated] (SPARK-31132) Optimized Logical Plan cast('' as timestamp) is null

[ https://issues.apache.org/jira/browse/SPARK-31132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] duyf updated SPARK-31132: - Summary: Optimized Logical Plan cast('' as timestamp) is null (was: receive_time<>'' Optimized Logical Plan i

[jira] [Updated] (SPARK-31132) receive_time<>'' Optimized Logical Plan is null

[ https://issues.apache.org/jira/browse/SPARK-31132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] duyf updated SPARK-31132: - Attachment: image-2020-03-12-18-34-22-571.png > receive_time<>'' Optimized Logical Plan is null > -

[jira] [Updated] (SPARK-31132) receive_time<>'' Optimized Logical Plan is null

[ https://issues.apache.org/jira/browse/SPARK-31132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] duyf updated SPARK-31132: - Description: receive_time='2020-02-10'； SQL： select case when receive_time<>'' then 1 else 0 end as tt from

[jira] [Updated] (SPARK-31132) receive_time<>'' Optimized Logical Plan is null

[ https://issues.apache.org/jira/browse/SPARK-31132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] duyf updated SPARK-31132: - Attachment: (was: image-2020-03-12-18-33-39-841.png) > receive_time<>'' Optimized Logical Plan is null > --

[jira] [Updated] (SPARK-31132) receive_time<>'' Optimized Logical Plan is null

[ https://issues.apache.org/jira/browse/SPARK-31132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] duyf updated SPARK-31132: - Attachment: image-2020-03-12-18-33-39-841.png > receive_time<>'' Optimized Logical Plan is null > -

[jira] [Created] (SPARK-31132) receive_time<>'' Optimized Logical Plan is null

duyf created SPARK-31132: Summary: receive_time<>'' Optimized Logical Plan is null Key: SPARK-31132 URL: https://issues.apache.org/jira/browse/SPARK-31132 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-29295) Duplicate result when dropping partition of an external table and then overwriting

[ https://issues.apache.org/jira/browse/SPARK-29295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-29295: -- Fix Version/s: 2.4.6 > Duplicate result when dropping partition of an external table and then

[jira] [Commented] (SPARK-29295) Duplicate result when dropping partition of an external table and then overwriting

[ https://issues.apache.org/jira/browse/SPARK-29295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17057772#comment-17057772 ] Dongjoon Hyun commented on SPARK-29295: --- This lands to `branch-2.4` via https://gi

[jira] [Commented] (SPARK-31116) PrquetRowConverter does not follow case sensitivity

[ https://issues.apache.org/jira/browse/SPARK-31116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17057752#comment-17057752 ] Dongjoon Hyun commented on SPARK-31116: --- Thank you, [~kimtkyeom]. Since I also con

[jira] [Updated] (SPARK-31116) PrquetRowConverter does not follow case sensitivity

[ https://issues.apache.org/jira/browse/SPARK-31116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-31116: -- Priority: Blocker (was: Critical) > PrquetRowConverter does not follow case sensitivity > ---

[jira] [Updated] (SPARK-31116) PrquetRowConverter does not follow case sensitivity

[ https://issues.apache.org/jira/browse/SPARK-31116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-31116: -- Description: After upgrading spark versrion to 3.0.0-SNAPSHOT. Selecting parquet columns got

[jira] [Updated] (SPARK-31116) PrquetRowConverter does not follow case sensitivity

[ https://issues.apache.org/jira/browse/SPARK-31116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-31116: -- Target Version/s: 3.0.0 > PrquetRowConverter does not follow case sensitivity > --

[jira] [Created] (SPARK-31131) Remove the unnecessary config spark.sql.legacy.timeParser.enabled

Kent Yao created SPARK-31131: Summary: Remove the unnecessary config spark.sql.legacy.timeParser.enabled Key: SPARK-31131 URL: https://issues.apache.org/jira/browse/SPARK-31131 Project: Spark I

[jira] [Updated] (SPARK-31116) PrquetRowConverter does not follow case sensitivity

[ https://issues.apache.org/jira/browse/SPARK-31116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-31116: -- Priority: Critical (was: Blocker) > PrquetRowConverter does not follow case sensitivity > ---

[jira] [Commented] (SPARK-25987) StackOverflowError when executing many operations on a table with many columns

[ https://issues.apache.org/jira/browse/SPARK-25987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17057713#comment-17057713 ] Jungtaek Lim commented on SPARK-25987: -- The root cause is the way of how "flowAnaly

[jira] [Updated] (SPARK-31130) Use the same version of `commons-io` in SBT

[ https://issues.apache.org/jira/browse/SPARK-31130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-31130: -- Affects Version/s: 2.4.5 Description: Maven always uses 2.4 version while SBT uses 2

[jira] [Updated] (SPARK-31123) Drop does not work after join with aliases

[ https://issues.apache.org/jira/browse/SPARK-31123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikel San Vicente updated SPARK-31123: -- Priority: Major (was: Minor) > Drop does not work after join with aliases > -

[jira] [Updated] (SPARK-31130) Use the same version of `commons-io` in SBT

[ https://issues.apache.org/jira/browse/SPARK-31130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-31130: -- Description: Maven always uses 2.4 version while SBT uses 2.5 at `hadoop-3.2` profile. {code}

[jira] [Assigned] (SPARK-31130) Use the same version of `commons-io` in SBT

[ https://issues.apache.org/jira/browse/SPARK-31130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-31130: - Assignee: Dongjoon Hyun > Use the same version of `commons-io` in SBT > ---

[jira] [Updated] (SPARK-31130) Use the same version of `commons-io` in SBT

[ https://issues.apache.org/jira/browse/SPARK-31130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-31130: -- Priority: Minor (was: Major) > Use the same version of `commons-io` in SBT >

[jira] [Commented] (SPARK-30132) Scala 2.13 compile errors from Hadoop LocalFileSystem subclasses

[ https://issues.apache.org/jira/browse/SPARK-30132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17057665#comment-17057665 ] Dongjoon Hyun commented on SPARK-30132: --- This issue is waiting for Scala 2.13.2 (s

[jira] [Created] (SPARK-31130) Use the same version of `commons-io` in SBT

Dongjoon Hyun created SPARK-31130: - Summary: Use the same version of `commons-io` in SBT Key: SPARK-31130 URL: https://issues.apache.org/jira/browse/SPARK-31130 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-30565) Regression in the ORC benchmark

[ https://issues.apache.org/jira/browse/SPARK-30565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17057647#comment-17057647 ] Peter Toth commented on SPARK-30565: [~cloud_fan], agreed, native ORC is more import