[jira] [Created] (SPARK-32068) Spark 3 UI task launch time show in error time zone

2020-06-22 Thread Smith Cruise (Jira)
Smith Cruise created SPARK-32068: Summary: Spark 3 UI task launch time show in error time zone Key: SPARK-32068 URL: https://issues.apache.org/jira/browse/SPARK-32068 Project: Spark Issue

[jira] [Updated] (SPARK-26905) Revisit reserved/non-reserved keywords based on the ANSI SQL standard

2020-06-22 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-26905: Fix Version/s: 3.1.0 > Revisit reserved/non-reserved keywords based on the ANSI SQL standard >

[jira] [Updated] (SPARK-31950) Extract SQL keywords from the generated parser class in TableIdentifierParserSuite

2020-06-22 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-31950: Fix Version/s: 3.1.0 > Extract SQL keywords from the generated parser class in >

[jira] [Updated] (SPARK-31230) use statement plans in DataFrameWriter(V2)

2020-06-22 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-31230: Fix Version/s: 3.1.0 > use statement plans in DataFrameWriter(V2) >

[jira] [Updated] (SPARK-31584) NullPointerException when parsing event log with InMemoryStore

2020-06-22 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-31584: Fix Version/s: 3.1.0 > NullPointerException when parsing event log with InMemoryStore >

[jira] [Updated] (SPARK-31918) SparkR CRAN check gives a warning with R 4.0.0 on OSX

2020-06-22 Thread Xiao Li (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-31918: Target Version/s: 3.0.1 > SparkR CRAN check gives a warning with R 4.0.0 on OSX >

[jira] [Commented] (SPARK-29226) Upgrade jackson-databind to 2.9.10 and fix vulnerabilities.

2020-06-22 Thread ThimmeGowda (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17142631#comment-17142631 ] ThimmeGowda commented on SPARK-29226: -  Hi, am using spark 2.4.5 and can i upgrade jackson-databind

[jira] [Comment Edited] (SPARK-31918) SparkR CRAN check gives a warning with R 4.0.0 on OSX

2020-06-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17142620#comment-17142620 ] Hyukjin Kwon edited comment on SPARK-31918 at 6/23/20, 4:35 AM: I tested

[jira] [Comment Edited] (SPARK-31918) SparkR CRAN check gives a warning with R 4.0.0 on OSX

2020-06-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17142620#comment-17142620 ] Hyukjin Kwon edited comment on SPARK-31918 at 6/23/20, 4:35 AM: I tested

[jira] [Commented] (SPARK-31918) SparkR CRAN check gives a warning with R 4.0.0 on OSX

2020-06-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17142620#comment-17142620 ] Hyukjin Kwon commented on SPARK-31918: -- I tested it manually with the fix I mentioned

[jira] [Commented] (SPARK-31918) SparkR CRAN check gives a warning with R 4.0.0 on OSX

2020-06-22 Thread Shivaram Venkataraman (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17142618#comment-17142618 ] Shivaram Venkataraman commented on SPARK-31918: --- Thats great! [~hyukjin.kwon] -- so we can

[jira] [Commented] (SPARK-31801) Register shuffle map output metadata with a shuffle output tracker

2020-06-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17142616#comment-17142616 ] Apache Spark commented on SPARK-31801: -- User 'mccheah' has created a pull request for this issue:

[jira] [Comment Edited] (SPARK-31918) SparkR CRAN check gives a warning with R 4.0.0 on OSX

2020-06-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17142605#comment-17142605 ] Hyukjin Kwon edited comment on SPARK-31918 at 6/23/20, 4:08 AM: Okay,

[jira] [Issue Comment Deleted] (SPARK-31918) SparkR CRAN check gives a warning with R 4.0.0 on OSX

2020-06-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-31918: - Comment: was deleted (was: Nice, [~shivaram]. I just quickly tested, and the first option is

[jira] [Issue Comment Deleted] (SPARK-31918) SparkR CRAN check gives a warning with R 4.0.0 on OSX

2020-06-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-31918: - Comment: was deleted (was: Oh, wait, the worker should test SparkR built with R 4.0.1. In the

[jira] [Commented] (SPARK-31918) SparkR CRAN check gives a warning with R 4.0.0 on OSX

2020-06-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17142605#comment-17142605 ] Hyukjin Kwon commented on SPARK-31918: -- Okay, [~shivaram], the first option seems working although

[jira] [Assigned] (SPARK-32064) Supporting create temporary table

2020-06-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32064: Assignee: Apache Spark > Supporting create temporary table >

[jira] [Commented] (SPARK-32064) Supporting create temporary table

2020-06-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17142603#comment-17142603 ] Apache Spark commented on SPARK-32064: -- User 'LantaoJin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32064) Supporting create temporary table

2020-06-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32064: Assignee: (was: Apache Spark) > Supporting create temporary table >

[jira] [Commented] (SPARK-31918) SparkR CRAN check gives a warning with R 4.0.0 on OSX

2020-06-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17142591#comment-17142591 ] Hyukjin Kwon commented on SPARK-31918: -- Oh, wait, the worker should test SparkR built with R 4.0.1.

[jira] [Updated] (SPARK-32067) [K8s] Pod template from subsequently submission inadvertently applies to ongoing submission

2020-06-22 Thread James Yu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Yu updated SPARK-32067: - Summary: [K8s] Pod template from subsequently submission inadvertently applies to ongoing submission

[jira] [Updated] (SPARK-32067) [K8s] Pod template from subsequently submission inadvertently applies to the ongoing submission

2020-06-22 Thread James Yu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Yu updated SPARK-32067: - Description: THE BUG: The bug is reproducible by spark-submit two different apps (app1 and app2) with

[jira] [Comment Edited] (SPARK-31998) Change package references for ArrowBuf

2020-06-22 Thread YoungGyu Chun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17142417#comment-17142417 ] YoungGyu Chun edited comment on SPARK-31998 at 6/23/20, 3:14 AM: -

[jira] [Created] (SPARK-32067) [K8s] Pod template from subsequently submission inadvertently applies to the ongoing submission

2020-06-22 Thread James Yu (Jira)
James Yu created SPARK-32067: Summary: [K8s] Pod template from subsequently submission inadvertently applies to the ongoing submission Key: SPARK-32067 URL: https://issues.apache.org/jira/browse/SPARK-32067

[jira] [Comment Edited] (SPARK-31998) Change package references for ArrowBuf

2020-06-22 Thread YoungGyu Chun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17142417#comment-17142417 ] YoungGyu Chun edited comment on SPARK-31998 at 6/23/20, 3:12 AM: -

[jira] [Updated] (SPARK-32059) Nested Schema Pruning not Working in Window Functions

2020-06-22 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh updated SPARK-32059: Issue Type: Improvement (was: Bug) > Nested Schema Pruning not Working in Window Functions >

[jira] [Commented] (SPARK-32056) Repartition by key should support partition coalesce for AQE

2020-06-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17142571#comment-17142571 ] Apache Spark commented on SPARK-32056: -- User 'viirya' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32056) Repartition by key should support partition coalesce for AQE

2020-06-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32056: Assignee: Apache Spark > Repartition by key should support partition coalesce for AQE >

[jira] [Assigned] (SPARK-32056) Repartition by key should support partition coalesce for AQE

2020-06-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32056: Assignee: (was: Apache Spark) > Repartition by key should support partition coalesce

[jira] [Commented] (SPARK-32056) Repartition by key should support partition coalesce for AQE

2020-06-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17142570#comment-17142570 ] Apache Spark commented on SPARK-32056: -- User 'viirya' has created a pull request for this issue:

[jira] [Created] (SPARK-32065) Supporting analyze temporary table

2020-06-22 Thread Lantao Jin (Jira)
Lantao Jin created SPARK-32065: -- Summary: Supporting analyze temporary table Key: SPARK-32065 URL: https://issues.apache.org/jira/browse/SPARK-32065 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-31918) SparkR CRAN check gives a warning with R 4.0.0 on OSX

2020-06-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17142566#comment-17142566 ] Hyukjin Kwon commented on SPARK-31918: -- Nice, [~shivaram]. I just quickly tested, and the first

[jira] [Created] (SPARK-32066) Supporting create temporary table LIKE

2020-06-22 Thread Lantao Jin (Jira)
Lantao Jin created SPARK-32066: -- Summary: Supporting create temporary table LIKE Key: SPARK-32066 URL: https://issues.apache.org/jira/browse/SPARK-32066 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-32064) Supporting create temporary table

2020-06-22 Thread Lantao Jin (Jira)
Lantao Jin created SPARK-32064: -- Summary: Supporting create temporary table Key: SPARK-32064 URL: https://issues.apache.org/jira/browse/SPARK-32064 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-31918) SparkR CRAN check gives a warning with R 4.0.0 on OSX

2020-06-22 Thread Shivaram Venkataraman (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17142558#comment-17142558 ] Shivaram Venkataraman commented on SPARK-31918: --- I can confirm that with build from source

[jira] [Assigned] (SPARK-32062) Reset listenerRegistered in SparkSession

2020-06-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32062: Assignee: Apache Spark > Reset listenerRegistered in SparkSession >

[jira] [Assigned] (SPARK-32062) Reset listenerRegistered in SparkSession

2020-06-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32062: Assignee: (was: Apache Spark) > Reset listenerRegistered in SparkSession >

[jira] [Commented] (SPARK-32062) Reset listenerRegistered in SparkSession

2020-06-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17142552#comment-17142552 ] Apache Spark commented on SPARK-32062: -- User 'ulysses-you' has created a pull request for this

[jira] [Commented] (SPARK-32062) Reset listenerRegistered in SparkSession

2020-06-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17142553#comment-17142553 ] Apache Spark commented on SPARK-32062: -- User 'ulysses-you' has created a pull request for this

[jira] [Created] (SPARK-32063) Spark native temporary table

2020-06-22 Thread Lantao Jin (Jira)
Lantao Jin created SPARK-32063: -- Summary: Spark native temporary table Key: SPARK-32063 URL: https://issues.apache.org/jira/browse/SPARK-32063 Project: Spark Issue Type: New Feature

[jira] [Created] (SPARK-32062) Reset listenerRegistered in SparkSession

2020-06-22 Thread ulysses you (Jira)
ulysses you created SPARK-32062: --- Summary: Reset listenerRegistered in SparkSession Key: SPARK-32062 URL: https://issues.apache.org/jira/browse/SPARK-32062 Project: Spark Issue Type: Bug

[jira] [Comment Edited] (SPARK-25244) [Python] Setting `spark.sql.session.timeZone` only partially respected

2020-06-22 Thread Toby Harradine (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17142533#comment-17142533 ] Toby Harradine edited comment on SPARK-25244 at 6/23/20, 2:09 AM: -- Hi,

[jira] [Commented] (SPARK-25244) [Python] Setting `spark.sql.session.timeZone` only partially respected

2020-06-22 Thread Toby Harradine (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17142533#comment-17142533 ] Toby Harradine commented on SPARK-25244: Hi, I've just come across this issue in PySpark 2.4.6

[jira] [Commented] (SPARK-31918) SparkR CRAN check gives a warning with R 4.0.0 on OSX

2020-06-22 Thread Shivaram Venkataraman (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17142532#comment-17142532 ] Shivaram Venkataraman commented on SPARK-31918: --- [~hyukjin.kwon] I have R 4.0.2 and will

[jira] [Created] (SPARK-32061) potential regression if use memoryUsage instead of numRows

2020-06-22 Thread zhengruifeng (Jira)
zhengruifeng created SPARK-32061: Summary: potential regression if use memoryUsage instead of numRows Key: SPARK-32061 URL: https://issues.apache.org/jira/browse/SPARK-32061 Project: Spark

[jira] [Commented] (SPARK-31918) SparkR CRAN check gives a warning with R 4.0.0 on OSX

2020-06-22 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17142520#comment-17142520 ] Dongjoon Hyun commented on SPARK-31918: --- Unfortunately, no~ I downgraded to R 3.5.2 on both my

[jira] [Updated] (SPARK-32060) Huber loss Convergence

2020-06-22 Thread zhengruifeng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng updated SPARK-32060: - Description: |performace test in https://issues.apache.org/jira/browse/SPARK-31783, Huber loss

[jira] [Updated] (SPARK-32060) Huber loss Convergence

2020-06-22 Thread zhengruifeng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng updated SPARK-32060: - Parent: SPARK-30641 Issue Type: Sub-task (was: Bug) > Huber loss Convergence >

[jira] [Created] (SPARK-32060) Huber loss Convergence

2020-06-22 Thread zhengruifeng (Jira)
zhengruifeng created SPARK-32060: Summary: Huber loss Convergence Key: SPARK-32060 URL: https://issues.apache.org/jira/browse/SPARK-32060 Project: Spark Issue Type: Bug Components:

[jira] [Assigned] (SPARK-32059) Nested Schema Pruning not Working in Window Functions

2020-06-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32059: Assignee: (was: Apache Spark) > Nested Schema Pruning not Working in Window

[jira] [Commented] (SPARK-32059) Nested Schema Pruning not Working in Window Functions

2020-06-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17142515#comment-17142515 ] Apache Spark commented on SPARK-32059: -- User 'frankyin-factual' has created a pull request for this

[jira] [Assigned] (SPARK-32059) Nested Schema Pruning not Working in Window Functions

2020-06-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32059: Assignee: Apache Spark > Nested Schema Pruning not Working in Window Functions >

[jira] [Commented] (SPARK-32059) Nested Schema Pruning not Working in Window Functions

2020-06-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17142514#comment-17142514 ] Apache Spark commented on SPARK-32059: -- User 'frankyin-factual' has created a pull request for this

[jira] [Resolved] (SPARK-27148) Support CURRENT_TIME and LOCALTIME when ANSI mode enabled

2020-06-22 Thread YoungGyu Chun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YoungGyu Chun resolved SPARK-27148. --- Resolution: Later > Support CURRENT_TIME and LOCALTIME when ANSI mode enabled >

[jira] [Commented] (SPARK-27148) Support CURRENT_TIME and LOCALTIME when ANSI mode enabled

2020-06-22 Thread YoungGyu Chun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17142511#comment-17142511 ] YoungGyu Chun commented on SPARK-27148: --- A similar ticket was concluded by the following comment

[jira] [Resolved] (SPARK-7101) Spark SQL should support java.sql.Time

2020-06-22 Thread YoungGyu Chun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-7101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] YoungGyu Chun resolved SPARK-7101. -- Resolution: Later > Spark SQL should support java.sql.Time >

[jira] [Commented] (SPARK-7101) Spark SQL should support java.sql.Time

2020-06-22 Thread YoungGyu Chun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-7101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17142510#comment-17142510 ] YoungGyu Chun commented on SPARK-7101: -- A similar ticket was concluded by the following comment by

[jira] [Updated] (SPARK-32013) Support query execution before/after reading/writing over JDBC

2020-06-22 Thread Noritaka Sekiyama (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Noritaka Sekiyama updated SPARK-32013: -- Description: For ETL workload, there is a common requirement to perform SQL statement

[jira] [Updated] (SPARK-32013) Support query execution before/after reading/writing over JDBC

2020-06-22 Thread Noritaka Sekiyama (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Noritaka Sekiyama updated SPARK-32013: -- Description: For ETL workload, there is a common requirement to perform SQL statement

[jira] [Comment Edited] (SPARK-31918) SparkR CRAN check gives a warning with R 4.0.0 on OSX

2020-06-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17142495#comment-17142495 ] Hyukjin Kwon edited comment on SPARK-31918 at 6/23/20, 12:06 AM: - Ah,

[jira] [Commented] (SPARK-31918) SparkR CRAN check gives a warning with R 4.0.0 on OSX

2020-06-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17142495#comment-17142495 ] Hyukjin Kwon commented on SPARK-31918: -- Ah, yeah. That one I read [it in the release

[jira] [Updated] (SPARK-32059) Nested Schema Pruning not Working in Window Functions

2020-06-22 Thread Frank Yin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Frank Yin updated SPARK-32059: -- Description: Using tables and data structures in `SchemaPruningSuite.scala`   {code:java} // code

[jira] [Created] (SPARK-32059) Nested Schema Pruning not Working in Window Functions

2020-06-22 Thread Frank Yin (Jira)
Frank Yin created SPARK-32059: - Summary: Nested Schema Pruning not Working in Window Functions Key: SPARK-32059 URL: https://issues.apache.org/jira/browse/SPARK-32059 Project: Spark Issue Type:

[jira] [Commented] (SPARK-31918) SparkR CRAN check gives a warning with R 4.0.0 on OSX

2020-06-22 Thread Shivaram Venkataraman (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17142466#comment-17142466 ] Shivaram Venkataraman commented on SPARK-31918: --- Thanks [~hyukjin.kwon]. It looks like

[jira] [Resolved] (SPARK-32044) [SS] 2.4 Kafka continuous processing print mislead initial offsets log

2020-06-22 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-32044. --- Fix Version/s: 2.4.7 Resolution: Fixed Issue resolved by pull request 28887

[jira] [Assigned] (SPARK-32044) [SS] 2.4 Kafka continuous processing print mislead initial offsets log

2020-06-22 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-32044: - Assignee: Zhongwei Zhu > [SS] 2.4 Kafka continuous processing print mislead initial

[jira] [Issue Comment Deleted] (SPARK-32058) Use Apache Hadoop 3.2.0 dependency by default

2020-06-22 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32058: -- Comment: was deleted (was: User 'dongjoon-hyun' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32058) Use Apache Hadoop 3.2.0 dependency by default

2020-06-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32058: Assignee: Apache Spark > Use Apache Hadoop 3.2.0 dependency by default >

[jira] [Commented] (SPARK-32058) Use Apache Hadoop 3.2.0 dependency by default

2020-06-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17142449#comment-17142449 ] Apache Spark commented on SPARK-32058: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-32058) Use Apache Hadoop 3.2.0 dependency by default

2020-06-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32058: Assignee: (was: Apache Spark) > Use Apache Hadoop 3.2.0 dependency by default >

[jira] [Commented] (SPARK-32058) Use Apache Hadoop 3.2.0 dependency by default

2020-06-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17142447#comment-17142447 ] Apache Spark commented on SPARK-32058: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Created] (SPARK-32058) Use Apache Hadoop 3.2.0 dependency by default

2020-06-22 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-32058: - Summary: Use Apache Hadoop 3.2.0 dependency by default Key: SPARK-32058 URL: https://issues.apache.org/jira/browse/SPARK-32058 Project: Spark Issue Type:

[jira] [Commented] (SPARK-32037) Rename blacklisting feature to avoid language with racist connotation

2020-06-22 Thread Erik Krogen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17142426#comment-17142426 ] Erik Krogen commented on SPARK-32037: - Thanks for the suggestions [~H4ml3t]! * *quarantined* to me

[jira] [Comment Edited] (SPARK-31998) Change package references for ArrowBuf

2020-06-22 Thread YoungGyu Chun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17142417#comment-17142417 ] YoungGyu Chun edited comment on SPARK-31998 at 6/22/20, 9:16 PM: - I will

[jira] [Commented] (SPARK-31998) Change package references for ArrowBuf

2020-06-22 Thread YoungGyu Chun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17142417#comment-17142417 ] YoungGyu Chun commented on SPARK-31998: --- I will be working on this > Change package references

[jira] [Created] (SPARK-32057) SparkExecuteStatementOperation does not set CANCELED/CLOSED state correctly

2020-06-22 Thread Ali Smesseim (Jira)
Ali Smesseim created SPARK-32057: Summary: SparkExecuteStatementOperation does not set CANCELED/CLOSED state correctly Key: SPARK-32057 URL: https://issues.apache.org/jira/browse/SPARK-32057

[jira] [Commented] (SPARK-32025) CSV schema inference with boolean & integer

2020-06-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17142403#comment-17142403 ] Apache Spark commented on SPARK-32025: -- User 'planga82' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32025) CSV schema inference with boolean & integer

2020-06-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32025: Assignee: Apache Spark > CSV schema inference with boolean & integer >

[jira] [Assigned] (SPARK-32025) CSV schema inference with boolean & integer

2020-06-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32025: Assignee: (was: Apache Spark) > CSV schema inference with boolean & integer >

[jira] [Commented] (SPARK-32025) CSV schema inference with boolean & integer

2020-06-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17142402#comment-17142402 ] Apache Spark commented on SPARK-32025: -- User 'planga82' has created a pull request for this issue:

[jira] [Commented] (SPARK-32050) GBTClassifier not working with OnevsRest

2020-06-22 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17142387#comment-17142387 ] L. C. Hsieh commented on SPARK-32050: - I think this was fixed at SPARK-27007. > GBTClassifier not

[jira] [Updated] (SPARK-32056) Repartition by key should support partition coalesce for AQE

2020-06-22 Thread koert kuipers (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] koert kuipers updated SPARK-32056: -- Priority: Minor (was: Major) > Repartition by key should support partition coalesce for AQE

[jira] [Updated] (SPARK-32056) Repartition by key should support partition coalesce for AQE

2020-06-22 Thread koert kuipers (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] koert kuipers updated SPARK-32056: -- Description: when adaptive query execution is enabled the following expression should

[jira] [Created] (SPARK-32056) Repartition by key should support partition coalesce for AQE

2020-06-22 Thread koert kuipers (Jira)
koert kuipers created SPARK-32056: - Summary: Repartition by key should support partition coalesce for AQE Key: SPARK-32056 URL: https://issues.apache.org/jira/browse/SPARK-32056 Project: Spark

[jira] [Updated] (SPARK-32044) [SS] 2.4 Kafka continuous processing print mislead initial offsets log

2020-06-22 Thread Zhongwei Zhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhongwei Zhu updated SPARK-32044: - Summary: [SS] 2.4 Kafka continuous processing print mislead initial offsets log (was: [SS]

[jira] [Updated] (SPARK-32044) [SS] 2.4 Kakfa continuous processing print mislead initial offsets log

2020-06-22 Thread Zhongwei Zhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhongwei Zhu updated SPARK-32044: - Summary: [SS] 2.4 Kakfa continuous processing print mislead initial offsets log (was: [SS]

[jira] [Commented] (SPARK-32055) Unify getReader and getReaderForRange in ShuffleManager

2020-06-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17142276#comment-17142276 ] Apache Spark commented on SPARK-32055: -- User 'Ngone51' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32055) Unify getReader and getReaderForRange in ShuffleManager

2020-06-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32055: Assignee: (was: Apache Spark) > Unify getReader and getReaderForRange in

[jira] [Assigned] (SPARK-32055) Unify getReader and getReaderForRange in ShuffleManager

2020-06-22 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32055: Assignee: Apache Spark > Unify getReader and getReaderForRange in ShuffleManager >

[jira] [Commented] (SPARK-32038) Regression in handling NaN values in COUNT(DISTINCT)

2020-06-22 Thread Mithun Radhakrishnan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17142265#comment-17142265 ] Mithun Radhakrishnan commented on SPARK-32038: -- [~viirya], [~dongjoon], thank you. I'm

[jira] [Created] (SPARK-32055) Unify getReader and getReaderForRange in ShuffleManager

2020-06-22 Thread wuyi (Jira)
wuyi created SPARK-32055: Summary: Unify getReader and getReaderForRange in ShuffleManager Key: SPARK-32055 URL: https://issues.apache.org/jira/browse/SPARK-32055 Project: Spark Issue Type:

[jira] [Commented] (SPARK-31995) Spark Structure Streaming checkpiontFileManager ERROR when HDFS.DFSOutputStream.completeFile with IOException unable to close file because the last block does not have

2020-06-22 Thread Gabor Somogyi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17142234#comment-17142234 ] Gabor Somogyi commented on SPARK-31995: --- At the first glance I don't think so it's a Spark but

[jira] [Created] (SPARK-32054) Flaky test: org.apache.spark.sql.connector.FileDataSourceV2FallBackSuite.Fallback Parquet V2 to V1

2020-06-22 Thread Gabor Somogyi (Jira)
Gabor Somogyi created SPARK-32054: - Summary: Flaky test: org.apache.spark.sql.connector.FileDataSourceV2FallBackSuite.Fallback Parquet V2 to V1 Key: SPARK-32054 URL:

[jira] [Updated] (SPARK-31918) SparkR CRAN check gives a warning with R 4.0.0 on OSX

2020-06-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-31918: - Priority: Blocker (was: Major) > SparkR CRAN check gives a warning with R 4.0.0 on OSX >

[jira] [Resolved] (SPARK-29773) Unable to process empty ORC files in Hive Table using Spark SQL

2020-06-22 Thread Alexander Ermakov (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Ermakov resolved SPARK-29773. --- Fix Version/s: 2.4.4 Resolution: Fixed > Unable to process empty ORC files

[jira] [Commented] (SPARK-29773) Unable to process empty ORC files in Hive Table using Spark SQL

2020-06-22 Thread Alexander Ermakov (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17142177#comment-17142177 ] Alexander Ermakov commented on SPARK-29773: --- This issue has been resolved for Spark 2.4.4 >

[jira] [Commented] (SPARK-31918) SparkR CRAN check gives a warning with R 4.0.0 on OSX

2020-06-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17142127#comment-17142127 ] Hyukjin Kwon commented on SPARK-31918: -- Just to share what I investigated: Seems the problem

[jira] [Comment Edited] (SPARK-31918) SparkR CRAN check gives a warning with R 4.0.0 on OSX

2020-06-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17142073#comment-17142073 ] Hyukjin Kwon edited comment on SPARK-31918 at 6/22/20, 2:10 PM: It

[jira] [Commented] (SPARK-31918) SparkR CRAN check gives a warning with R 4.0.0 on OSX

2020-06-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17142073#comment-17142073 ] Hyukjin Kwon commented on SPARK-31918: -- It affects Spark 3.0 too, and seems failing with a

[jira] [Updated] (SPARK-31918) SparkR CRAN check gives a warning with R 4.0.0 on OSX

2020-06-22 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-31918: - Affects Version/s: 3.0.0 > SparkR CRAN check gives a warning with R 4.0.0 on OSX >

  1   2   >