[jira] [Updated] (SPARK-31152) Issue with Spark Context Initialization i.e. SparkSession.builder.getOrCreate()

[ https://issues.apache.org/jira/browse/SPARK-31152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohith Bhattaram updated SPARK-31152: - Description: Issue :  I am trying to initialize spark context using below code :   

[jira] [Created] (SPARK-31152) Issue with Spark Context Initialization

Rohith Bhattaram created SPARK-31152: Summary: Issue with Spark Context Initialization Key: SPARK-31152 URL: https://issues.apache.org/jira/browse/SPARK-31152 Project: Spark Issue Type:

[jira] [Updated] (SPARK-31152) Issue with Spark Context Initialization i.e. SparkSession.builder.getOrCreate()

[ https://issues.apache.org/jira/browse/SPARK-31152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohith Bhattaram updated SPARK-31152: - Summary: Issue with Spark Context Initialization i.e.

[jira] [Assigned] (SPARK-31151) Reorganize the migration guide of SQL

[ https://issues.apache.org/jira/browse/SPARK-31151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reassigned SPARK-31151: --- Assignee: Xiao Li > Reorganize the migration guide of SQL >

[jira] [Assigned] (SPARK-28588) Build a SQL reference doc

[ https://issues.apache.org/jira/browse/SPARK-28588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reassigned SPARK-28588: --- Assignee: (was: Xiao Li) > Build a SQL reference doc > - > >

[jira] [Assigned] (SPARK-28588) Build a SQL reference doc

[ https://issues.apache.org/jira/browse/SPARK-28588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reassigned SPARK-28588: --- Assignee: Xiao Li > Build a SQL reference doc > - > > Key:

[jira] [Created] (SPARK-31151) Reorganize the migration guide of SQL

Xiao Li created SPARK-31151: --- Summary: Reorganize the migration guide of SQL Key: SPARK-31151 URL: https://issues.apache.org/jira/browse/SPARK-31151 Project: Spark Issue Type: Documentation

[jira] [Resolved] (SPARK-31144) Wrap java.lang.Error with an exception for QueryExecutionListener.onFailure

[ https://issues.apache.org/jira/browse/SPARK-31144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-31144. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 27907

[jira] [Assigned] (SPARK-31144) Wrap java.lang.Error with an exception for QueryExecutionListener.onFailure

[ https://issues.apache.org/jira/browse/SPARK-31144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-31144: - Assignee: Shixiong Zhu > Wrap java.lang.Error with an exception for

[jira] [Assigned] (SPARK-30654) Update Docs Bootstrap to 4.4.1

[ https://issues.apache.org/jira/browse/SPARK-30654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang reassigned SPARK-30654: -- Assignee: Dale Clarke > Update Docs Bootstrap to 4.4.1 >

[jira] [Resolved] (SPARK-30654) Update Docs Bootstrap to 4.4.1

[ https://issues.apache.org/jira/browse/SPARK-30654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-30654. Target Version/s: 3.1.0 Resolution: Fixed The issue is resolved in

[jira] [Commented] (SPARK-30951) Potential data loss for legacy applications after switch to proleptic Gregorian calendar

[ https://issues.apache.org/jira/browse/SPARK-30951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17059080#comment-17059080 ] Dongjoon Hyun commented on SPARK-30951: --- I marked as `correctness` since the query result is

[jira] [Updated] (SPARK-30951) Potential data loss for legacy applications after switch to proleptic Gregorian calendar

[ https://issues.apache.org/jira/browse/SPARK-30951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30951: -- Labels: correctness (was: ) > Potential data loss for legacy applications after switch to

[jira] [Updated] (SPARK-31144) Wrap java.lang.Error with an exception for QueryExecutionListener.onFailure

[ https://issues.apache.org/jira/browse/SPARK-31144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-31144: -- Fix Version/s: 2.4.6 > Wrap java.lang.Error with an exception for

[jira] [Commented] (SPARK-31144) Wrap java.lang.Error with an exception for QueryExecutionListener.onFailure

[ https://issues.apache.org/jira/browse/SPARK-31144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17059074#comment-17059074 ] Dongjoon Hyun commented on SPARK-31144: --- This landed at `branch-2.4` via

[jira] [Resolved] (SPARK-31004) Show message for empty Streaming Queries instead of empty timelines and histograms.

[ https://issues.apache.org/jira/browse/SPARK-31004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang resolved SPARK-31004. Fix Version/s: 3.0.0 Resolution: Fixed This issue is resolved in

[jira] [Updated] (SPARK-31150) Parsing seconds fraction with variable length for timestamp

[ https://issues.apache.org/jira/browse/SPARK-31150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-31150: - Issue Type: Improvement (was: Bug) > Parsing seconds fraction with variable length for timestamp >

[jira] [Updated] (SPARK-31144) Wrap java.lang.Error with an exception for QueryExecutionListener.onFailure

[ https://issues.apache.org/jira/browse/SPARK-31144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-31144: -- Parent: SPARK-31085 Issue Type: Sub-task (was: Bug) > Wrap java.lang.Error with an

[jira] [Updated] (SPARK-31150) Parsing seconds fraction with variable length for timestamp

[ https://issues.apache.org/jira/browse/SPARK-31150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kent Yao updated SPARK-31150: - Description: This JIRA is to support parsing timestamp values with variable length second fraction

[jira] [Created] (SPARK-31150) Parsing seconds fraction with variable length for timestamp

Kent Yao created SPARK-31150: Summary: Parsing seconds fraction with variable length for timestamp Key: SPARK-31150 URL: https://issues.apache.org/jira/browse/SPARK-31150 Project: Spark Issue

[jira] [Issue Comment Deleted] (SPARK-31137) Opportunity to simplify execution plan when passing empty dataframes to subtract()

[ https://issues.apache.org/jira/browse/SPARK-31137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] S Daniel Zafar updated SPARK-31137: --- Comment: was deleted (was: Moving this to Databricks internal board.) > Opportunity to

[jira] [Resolved] (SPARK-31137) Opportunity to simplify execution plan when passing empty dataframes to subtract()

[ https://issues.apache.org/jira/browse/SPARK-31137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] S Daniel Zafar resolved SPARK-31137. Resolution: Won't Do moving to Databricks internal board. > Opportunity to simplify

[jira] [Commented] (SPARK-31137) Opportunity to simplify execution plan when passing empty dataframes to subtract()

[ https://issues.apache.org/jira/browse/SPARK-31137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17058872#comment-17058872 ] S Daniel Zafar commented on SPARK-31137: Moving this to Databricks internal board. >

[jira] [Commented] (SPARK-31149) PySpark job not killing Spark Daemon processes after the executor is killed due to OOM

[ https://issues.apache.org/jira/browse/SPARK-31149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17058853#comment-17058853 ] Arsenii Venherak commented on SPARK-31149: -- Created PR

[jira] [Updated] (SPARK-31149) PySpark job not killing Spark Daemon processes after the executor is killed due to OOM

[ https://issues.apache.org/jira/browse/SPARK-31149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arsenii Venherak updated SPARK-31149: - Description: {code:java} 2020-03-10 10:15:00,257 INFO

[jira] [Created] (SPARK-31149) PySpark job not killing Spark Daemon processes after the executor is killed due to OOM

Arsenii Venherak created SPARK-31149: Summary: PySpark job not killing Spark Daemon processes after the executor is killed due to OOM Key: SPARK-31149 URL: https://issues.apache.org/jira/browse/SPARK-31149

[jira] [Updated] (SPARK-31148) Switching inner join statement yield different result

[ https://issues.apache.org/jira/browse/SPARK-31148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tanapol Nearunchorn updated SPARK-31148: Summary: Switching inner join statement yield different result (was: Physical

[jira] [Updated] (SPARK-31148) Physical Plan and Optimized Logical Plan are mismatch

[ https://issues.apache.org/jira/browse/SPARK-31148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tanapol Nearunchorn updated SPARK-31148: Description: Given this query   {code:java} select r.id from

[jira] [Updated] (SPARK-31148) Physical Plan and Optimized Logical Plan are mismatch

[ https://issues.apache.org/jira/browse/SPARK-31148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tanapol Nearunchorn updated SPARK-31148: Description: Given this query   {code:java} select r.id from

[jira] [Created] (SPARK-31148) Physical Plan and Optimized Logical Plan are mismatch

Tanapol Nearunchorn created SPARK-31148: --- Summary: Physical Plan and Optimized Logical Plan are mismatch Key: SPARK-31148 URL: https://issues.apache.org/jira/browse/SPARK-31148 Project: Spark

[jira] [Commented] (SPARK-31143) Spark 2.4.4 count distinct query much slower than Spark 1.6.2 and Hive 1.2.1

[ https://issues.apache.org/jira/browse/SPARK-31143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17058573#comment-17058573 ] Jungtaek Lim commented on SPARK-31143: -- [~shijiezhiai] Could you please leave the information about

[jira] [Commented] (SPARK-31136) Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax

[ https://issues.apache.org/jira/browse/SPARK-31136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17058544#comment-17058544 ] Wenchen Fan commented on SPARK-31136: - Created https://github.com/apache/spark/pull/27902 > Revert

[jira] [Created] (SPARK-31147) forbid CHAR/VARCHAR type in non-Hive tables

Wenchen Fan created SPARK-31147: --- Summary: forbid CHAR/VARCHAR type in non-Hive tables Key: SPARK-31147 URL: https://issues.apache.org/jira/browse/SPARK-31147 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-4298) The spark-submit cannot read Main-Class from Manifest.

[ https://issues.apache.org/jira/browse/SPARK-4298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17058539#comment-17058539 ] Hannah Chua commented on SPARK-4298: Please assist my issue writing spark output to cassandra on

[jira] [Updated] (SPARK-4298) The spark-submit cannot read Main-Class from Manifest.

[ https://issues.apache.org/jira/browse/SPARK-4298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hannah Chua updated SPARK-4298: --- Attachment: Hannah 6598199927.png > The spark-submit cannot read Main-Class from Manifest. >

[jira] [Updated] (SPARK-4298) The spark-submit cannot read Main-Class from Manifest.

[ https://issues.apache.org/jira/browse/SPARK-4298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hannah Chua updated SPARK-4298: --- Attachment: Screenshot 2020-03-13 at 4.24.02 PM.png > The spark-submit cannot read Main-Class from

[jira] [Created] (SPARK-31146) Leverage the helper method for aliasing in all SQL expressions

Hyukjin Kwon created SPARK-31146: Summary: Leverage the helper method for aliasing in all SQL expressions Key: SPARK-31146 URL: https://issues.apache.org/jira/browse/SPARK-31146 Project: Spark

[jira] [Updated] (SPARK-28551) CTAS SQL with LOCATION properties won't check location path, cause data under this location lose

[ https://issues.apache.org/jira/browse/SPARK-28551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28551: -- Description: When we run SQL like  {code:java} CRETE TABLE TBL LOCATION 'PATH_URI' AS SELECT

[jira] [Updated] (SPARK-28551) CTAS SQL with LOCATION properties won't check location path, cause data under this location lose

[ https://issues.apache.org/jira/browse/SPARK-28551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28551: -- Affects Version/s: (was: 3.0.0) > CTAS SQL with LOCATION properties won't check location

[jira] [Updated] (SPARK-28551) CTAS SQL with LOCATION properties won't check location path, cause data under this location lose

[ https://issues.apache.org/jira/browse/SPARK-28551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28551: -- Description: When we run SQL like  {code:java} CRETE TABLE TBL LOCATION 'PATH_URI' AS SELECT

[jira] [Updated] (SPARK-28551) CTAS SQL with LOCATION properties won't check location path, cause data under this location lose

[ https://issues.apache.org/jira/browse/SPARK-28551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28551: -- Issue Type: Bug (was: Wish) > CTAS SQL with LOCATION properties won't check location path,

[jira] [Updated] (SPARK-28551) CTAS SQL with LOCATION properties won't check location path, cause data under this location lose

[ https://issues.apache.org/jira/browse/SPARK-28551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-28551: -- Labels: (was: SQL) > CTAS SQL with LOCATION properties won't check location path, cause

[jira] [Commented] (SPARK-25193) insert overwrite doesn't throw exception when drop old data fails

[ https://issues.apache.org/jira/browse/SPARK-25193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17058469#comment-17058469 ] Dongjoon Hyun commented on SPARK-25193: --- With `hive-2.3` profile, this will throw Exception to

[jira] [Resolved] (SPARK-31143) Spark 2.4.4 count distinct query much slower than Spark 1.6.2 and Hive 1.2.1

[ https://issues.apache.org/jira/browse/SPARK-31143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Ma resolved SPARK-31143. -- Resolution: Not A Problem > Spark 2.4.4 count distinct query much slower than Spark 1.6.2 and Hive

[jira] [Updated] (SPARK-30034) Use Apache Hive 2.3 dependency by default

[ https://issues.apache.org/jira/browse/SPARK-30034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30034: -- Labels: correctness release-notes (was: release-notes) > Use Apache Hive 2.3 dependency by

[jira] [Updated] (SPARK-30034) Use Apache Hive 2.3 dependency by default

[ https://issues.apache.org/jira/browse/SPARK-30034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-30034: -- Description: Some correctness issue like SPARK-25193 is fixed by `hive-2.3` only. > Use

[jira] [Created] (SPARK-31145) spark2.4.0 read hbase2.1.0 Error

HarSenZhao created SPARK-31145: -- Summary: spark2.4.0 read hbase2.1.0 Error Key: SPARK-31145 URL: https://issues.apache.org/jira/browse/SPARK-31145 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-25193) insert overwrite doesn't throw exception when drop old data fails

[ https://issues.apache.org/jira/browse/SPARK-25193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25193: -- Affects Version/s: (was: 2.2.0) 1.6.3 2.0.2

[jira] [Updated] (SPARK-31144) Wrap java.lang.Error with an exception for QueryExecutionListener.onFailure

[ https://issues.apache.org/jira/browse/SPARK-31144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-31144: - Issue Type: Bug (was: Test) > Wrap java.lang.Error with an exception for

[jira] [Created] (SPARK-31144) Wrap java.lang.Error with an exception for QueryExecutionListener.onFailure

Shixiong Zhu created SPARK-31144: Summary: Wrap java.lang.Error with an exception for QueryExecutionListener.onFailure Key: SPARK-31144 URL: https://issues.apache.org/jira/browse/SPARK-31144 Project:

[jira] [Created] (SPARK-31143) Spark 2.4.4 count distinct query much slower than Spark 1.6.2 and Hive 1.2.1

Kevin Ma created SPARK-31143: - Summary: Spark 2.4.4 count distinct query much slower than Spark 1.6.2 and Hive 1.2.1 Key: SPARK-31143 URL: https://issues.apache.org/jira/browse/SPARK-31143 Project: