[jira] [Created] (SPARK-28816) Document ADD JAR statement in SQL Reference.

2019-08-20 Thread Dilip Biswal (Jira)
Dilip Biswal created SPARK-28816: Summary: Document ADD JAR statement in SQL Reference. Key: SPARK-28816 URL: https://issues.apache.org/jira/browse/SPARK-28816 Project: Spark Issue Type:

[jira] [Created] (SPARK-28815) Document ADD FILE statement in SQL Reference.

2019-08-20 Thread Dilip Biswal (Jira)
Dilip Biswal created SPARK-28815: Summary: Document ADD FILE statement in SQL Reference. Key: SPARK-28815 URL: https://issues.apache.org/jira/browse/SPARK-28815 Project: Spark Issue Type:

[jira] [Created] (SPARK-28814) Document SET/RESET in SQL Reference.

2019-08-20 Thread Dilip Biswal (Jira)
Dilip Biswal created SPARK-28814: Summary: Document SET/RESET in SQL Reference. Key: SPARK-28814 URL: https://issues.apache.org/jira/browse/SPARK-28814 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-28812) Document SHOW PARTITIONS in SQL Reference.

2019-08-20 Thread Dilip Biswal (Jira)
Dilip Biswal created SPARK-28812: Summary: Document SHOW PARTITIONS in SQL Reference. Key: SPARK-28812 URL: https://issues.apache.org/jira/browse/SPARK-28812 Project: Spark Issue Type:

[jira] [Created] (SPARK-28813) Document SHOW CREATE TABLE in SQL Reference.

2019-08-20 Thread Dilip Biswal (Jira)
Dilip Biswal created SPARK-28813: Summary: Document SHOW CREATE TABLE in SQL Reference. Key: SPARK-28813 URL: https://issues.apache.org/jira/browse/SPARK-28813 Project: Spark Issue Type:

[jira] [Created] (SPARK-28811) Document SHOW TBLPROPERTIES in SQL Reference.

2019-08-20 Thread Dilip Biswal (Jira)
Dilip Biswal created SPARK-28811: Summary: Document SHOW TBLPROPERTIES in SQL Reference. Key: SPARK-28811 URL: https://issues.apache.org/jira/browse/SPARK-28811 Project: Spark Issue Type:

[jira] [Created] (SPARK-28809) Document SHOW TABLE in SQL Reference

2019-08-20 Thread Dilip Biswal (Jira)
Dilip Biswal created SPARK-28809: Summary: Document SHOW TABLE in SQL Reference Key: SPARK-28809 URL: https://issues.apache.org/jira/browse/SPARK-28809 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-28810) Document SHOW TABLES in SQL Reference.

2019-08-20 Thread Dilip Biswal (Jira)
Dilip Biswal created SPARK-28810: Summary: Document SHOW TABLES in SQL Reference. Key: SPARK-28810 URL: https://issues.apache.org/jira/browse/SPARK-28810 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-28808) Document SHOW FUNCTIONS in SQL Reference.

2019-08-20 Thread Dilip Biswal (Jira)
Dilip Biswal created SPARK-28808: Summary: Document SHOW FUNCTIONS in SQL Reference. Key: SPARK-28808 URL: https://issues.apache.org/jira/browse/SPARK-28808 Project: Spark Issue Type:

[jira] [Created] (SPARK-28807) Document SHOW DATABASES in SQL Reference.

2019-08-20 Thread Dilip Biswal (Jira)
Dilip Biswal created SPARK-28807: Summary: Document SHOW DATABASES in SQL Reference. Key: SPARK-28807 URL: https://issues.apache.org/jira/browse/SPARK-28807 Project: Spark Issue Type:

[jira] [Created] (SPARK-28806) Document SHOW COLUMNS in SQL Reference.

2019-08-20 Thread Dilip Biswal (Jira)
Dilip Biswal created SPARK-28806: Summary: Document SHOW COLUMNS in SQL Reference. Key: SPARK-28806 URL: https://issues.apache.org/jira/browse/SPARK-28806 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-28805) Document DESCRIBE FUNCTION in SQL Reference.

2019-08-20 Thread Dilip Biswal (Jira)
Dilip Biswal created SPARK-28805: Summary: Document DESCRIBE FUNCTION in SQL Reference. Key: SPARK-28805 URL: https://issues.apache.org/jira/browse/SPARK-28805 Project: Spark Issue Type:

[jira] [Created] (SPARK-28803) Document DESCRIBE TABLE in SQL Reference.

2019-08-20 Thread Dilip Biswal (Jira)
Dilip Biswal created SPARK-28803: Summary: Document DESCRIBE TABLE in SQL Reference. Key: SPARK-28803 URL: https://issues.apache.org/jira/browse/SPARK-28803 Project: Spark Issue Type:

[jira] [Created] (SPARK-28804) Document DESCRIBE QUERY in SQL Reference.

2019-08-20 Thread Dilip Biswal (Jira)
Dilip Biswal created SPARK-28804: Summary: Document DESCRIBE QUERY in SQL Reference. Key: SPARK-28804 URL: https://issues.apache.org/jira/browse/SPARK-28804 Project: Spark Issue Type:

[jira] [Created] (SPARK-28802) Document DESCRIBE DATABASE in SQL Reference.

2019-08-20 Thread Dilip Biswal (Jira)
Dilip Biswal created SPARK-28802: Summary: Document DESCRIBE DATABASE in SQL Reference. Key: SPARK-28802 URL: https://issues.apache.org/jira/browse/SPARK-28802 Project: Spark Issue Type:

[jira] [Created] (SPARK-28801) Document SELECT statement in SQL Reference.

2019-08-20 Thread Dilip Biswal (Jira)
Dilip Biswal created SPARK-28801: Summary: Document SELECT statement in SQL Reference. Key: SPARK-28801 URL: https://issues.apache.org/jira/browse/SPARK-28801 Project: Spark Issue Type:

[jira] [Updated] (SPARK-28790) Document CACHE TABLE statement in SQL Reference.

2019-08-20 Thread Huaxin Gao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxin Gao updated SPARK-28790: --- Summary: Document CACHE TABLE statement in SQL Reference. (was: Document CACHE TABLE statement. )

[jira] [Updated] (SPARK-28788) Document ANALYZE TABLE statement in SQL Reference.

2019-08-20 Thread Huaxin Gao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxin Gao updated SPARK-28788: --- Summary: Document ANALYZE TABLE statement in SQL Reference. (was: Document ANALYZE TABLE

[jira] [Created] (SPARK-28800) Document REPAIR TABLE statement in SQL Reference.

2019-08-20 Thread Dilip Biswal (Jira)
Dilip Biswal created SPARK-28800: Summary: Document REPAIR TABLE statement in SQL Reference. Key: SPARK-28800 URL: https://issues.apache.org/jira/browse/SPARK-28800 Project: Spark Issue

[jira] [Updated] (SPARK-28787) Document LOAD DATA statement in SQL Reference.

2019-08-20 Thread Huaxin Gao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxin Gao updated SPARK-28787: --- Summary: Document LOAD DATA statement in SQL Reference. (was: Document LOAD DATA statement.) >

[jira] [Created] (SPARK-28799) Document TRUNCATE TABLE in SQL Reference.

2019-08-20 Thread Dilip Biswal (Jira)
Dilip Biswal created SPARK-28799: Summary: Document TRUNCATE TABLE in SQL Reference. Key: SPARK-28799 URL: https://issues.apache.org/jira/browse/SPARK-28799 Project: Spark Issue Type:

[jira] [Created] (SPARK-28798) Document DROP TABLE/VIEW statement in SQL Reference.

2019-08-20 Thread Dilip Biswal (Jira)
Dilip Biswal created SPARK-28798: Summary: Document DROP TABLE/VIEW statement in SQL Reference. Key: SPARK-28798 URL: https://issues.apache.org/jira/browse/SPARK-28798 Project: Spark Issue

[jira] [Updated] (SPARK-28786) Document INSERT statement in SQL Reference.

2019-08-20 Thread Huaxin Gao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxin Gao updated SPARK-28786: --- Summary: Document INSERT statement in SQL Reference. (was: Document INSERT statement.) > Document

[jira] [Updated] (SPARK-28790) Document CACHE TABLE statement.

2019-08-20 Thread Huaxin Gao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxin Gao updated SPARK-28790: --- Summary: Document CACHE TABLE statement. (was: Auxilary Statements: CACHE TABLE statement) >

[jira] [Updated] (SPARK-28788) Document ANALYZE TABLE statement.

2019-08-20 Thread Huaxin Gao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxin Gao updated SPARK-28788: --- Summary: Document ANALYZE TABLE statement. (was: Auxilary Statements: Analyze Table) > Document

[jira] [Created] (SPARK-28797) Document DROP FUNCTION statement in SQL Reference.

2019-08-20 Thread Dilip Biswal (Jira)
Dilip Biswal created SPARK-28797: Summary: Document DROP FUNCTION statement in SQL Reference. Key: SPARK-28797 URL: https://issues.apache.org/jira/browse/SPARK-28797 Project: Spark Issue

[jira] [Updated] (SPARK-28787) Document LOAD DATA statement.

2019-08-20 Thread Huaxin Gao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxin Gao updated SPARK-28787: --- Summary: Document LOAD DATA statement. (was: Data Manipulation Statements: Load) > Document LOAD

[jira] [Created] (SPARK-28796) Document DROP DATABASE statement in SQL Reference.

2019-08-20 Thread Dilip Biswal (Jira)
Dilip Biswal created SPARK-28796: Summary: Document DROP DATABASE statement in SQL Reference. Key: SPARK-28796 URL: https://issues.apache.org/jira/browse/SPARK-28796 Project: Spark Issue

[jira] [Updated] (SPARK-28786) Document INSERT statement.

2019-08-20 Thread Huaxin Gao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxin Gao updated SPARK-28786: --- Summary: Document INSERT statement. (was: Data Manipulation Statements: Insert) > Document INSERT

[jira] [Created] (SPARK-28795) Document CREATE VIEW statement in SQL Reference.

2019-08-20 Thread Dilip Biswal (Jira)
Dilip Biswal created SPARK-28795: Summary: Document CREATE VIEW statement in SQL Reference. Key: SPARK-28795 URL: https://issues.apache.org/jira/browse/SPARK-28795 Project: Spark Issue Type:

[jira] [Created] (SPARK-28794) Document CREATE TABLE in SQL Reference.

2019-08-20 Thread Dilip Biswal (Jira)
Dilip Biswal created SPARK-28794: Summary: Document CREATE TABLE in SQL Reference. Key: SPARK-28794 URL: https://issues.apache.org/jira/browse/SPARK-28794 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-28793) Document CREATE FUNCTION in SQL Reference.

2019-08-20 Thread Dilip Biswal (Jira)
Dilip Biswal created SPARK-28793: Summary: Document CREATE FUNCTION in SQL Reference. Key: SPARK-28793 URL: https://issues.apache.org/jira/browse/SPARK-28793 Project: Spark Issue Type:

[jira] [Created] (SPARK-28792) Document CREATE DATABASE statement in SQL Reference.

2019-08-20 Thread Dilip Biswal (Jira)
Dilip Biswal created SPARK-28792: Summary: Document CREATE DATABASE statement in SQL Reference. Key: SPARK-28792 URL: https://issues.apache.org/jira/browse/SPARK-28792 Project: Spark Issue

[jira] [Created] (SPARK-28791) Document ALTER TABLE statement in SQL Reference.

2019-08-20 Thread Dilip Biswal (Jira)
Dilip Biswal created SPARK-28791: Summary: Document ALTER TABLE statement in SQL Reference. Key: SPARK-28791 URL: https://issues.apache.org/jira/browse/SPARK-28791 Project: Spark Issue Type:

[jira] [Created] (SPARK-28790) Auxilary Statements: CACHE TABLE statement

2019-08-20 Thread Huaxin Gao (Jira)
Huaxin Gao created SPARK-28790: -- Summary: Auxilary Statements: CACHE TABLE statement Key: SPARK-28790 URL: https://issues.apache.org/jira/browse/SPARK-28790 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-28789) Document ALTER DATABASE statement.

2019-08-20 Thread Dilip Biswal (Jira)
Dilip Biswal created SPARK-28789: Summary: Document ALTER DATABASE statement. Key: SPARK-28789 URL: https://issues.apache.org/jira/browse/SPARK-28789 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-28788) Auxilary Statements: Analyze Table

2019-08-20 Thread Huaxin Gao (Jira)
Huaxin Gao created SPARK-28788: -- Summary: Auxilary Statements: Analyze Table Key: SPARK-28788 URL: https://issues.apache.org/jira/browse/SPARK-28788 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-28787) Data Manipulation Statements: Load

2019-08-20 Thread Huaxin Gao (Jira)
Huaxin Gao created SPARK-28787: -- Summary: Data Manipulation Statements: Load Key: SPARK-28787 URL: https://issues.apache.org/jira/browse/SPARK-28787 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-28786) Data Manipulation Statements: Insert

2019-08-20 Thread Huaxin Gao (Jira)
Huaxin Gao created SPARK-28786: -- Summary: Data Manipulation Statements: Insert Key: SPARK-28786 URL: https://issues.apache.org/jira/browse/SPARK-28786 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-28672) [UDF] Duplicate function creation should not allow

2019-08-20 Thread ABHISHEK KUMAR GUPTA (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16911937#comment-16911937 ] ABHISHEK KUMAR GUPTA commented on SPARK-28672: -- Thanks for update. This JIRA is for not

[jira] [Comment Edited] (SPARK-20780) Spark Kafka10 Consumer Hangs

2019-08-20 Thread junzhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-20780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16911905#comment-16911905 ] junzhang edited comment on SPARK-20780 at 8/21/19 3:13 AM: --- [~Kevin_HW] we

[jira] [Commented] (SPARK-20780) Spark Kafka10 Consumer Hangs

2019-08-20 Thread junzhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-20780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16911905#comment-16911905 ] junzhang commented on SPARK-20780: -- [~Kevin_HW] we face the same problem。 Could you please point out

[jira] [Created] (SPARK-28785) Schema for type scala.Array[Float] is not supported

2019-08-20 Thread Jigao Fu (Jira)
Jigao Fu created SPARK-28785: Summary: Schema for type scala.Array[Float] is not supported Key: SPARK-28785 URL: https://issues.apache.org/jira/browse/SPARK-28785 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-28754) [UDF] Supports for alter, rename, owner change should be supported in Spark

2019-08-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16911861#comment-16911861 ] Hyukjin Kwon commented on SPARK-28754: -- Ah, it was under PostgresSQL feature parity. Thanks for

[jira] [Updated] (SPARK-28770) Flaky Tests: Test ReplayListenerSuite.End-to-end replay with compression failed

2019-08-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-28770: - Summary: Flaky Tests: Test ReplayListenerSuite.End-to-end replay with compression failed (was:

[jira] [Resolved] (SPARK-28779) CSV writer doesn't handle older Mac line endings

2019-08-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-28779. -- Resolution: Not A Problem > CSV writer doesn't handle older Mac line endings >

[jira] [Commented] (SPARK-28779) CSV writer doesn't handle older Mac line endings

2019-08-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16911857#comment-16911857 ] Hyukjin Kwon commented on SPARK-28779: -- You can set {{lineSep}} option. > CSV writer doesn't

[jira] [Commented] (SPARK-28783) Support plus + operator as concatenation for StringType columns in pyspark

2019-08-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16911856#comment-16911856 ] Hyukjin Kwon commented on SPARK-28783: -- I think it's a breaking change if we do. The current

[jira] [Resolved] (SPARK-28783) Support plus + operator as concatenation for StringType columns in pyspark

2019-08-20 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-28783. -- Resolution: Won't Fix > Support plus + operator as concatenation for StringType columns in

[jira] [Assigned] (SPARK-28541) Document Storage page

2019-08-20 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-28541: - Assignee: zhengruifeng > Document Storage page > - > >

[jira] [Resolved] (SPARK-28541) Document Storage page

2019-08-20 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-28541. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 25445

[jira] [Updated] (SPARK-28541) Document Storage page

2019-08-20 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-28541: -- Priority: Minor (was: Major) > Document Storage page > - > >

[jira] [Commented] (SPARK-28762) Read JAR main class if JAR is not located in local file system

2019-08-20 Thread Ivan Gozali (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16911830#comment-16911830 ] Ivan Gozali commented on SPARK-28762: - That makes sense, thank you for the explanation. > Read JAR

[jira] [Commented] (SPARK-28672) [UDF] Duplicate function creation should not allow

2019-08-20 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16911827#comment-16911827 ] Takeshi Yamamuro commented on SPARK-28672: -- I personally think the current behaviour looks ok

[jira] [Resolved] (SPARK-28322) DIV support decimal type

2019-08-20 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-28322. -- Fix Version/s: 3.0.0 Assignee: Marco Gaido Resolution: Fixed Resolved

[jira] [Commented] (SPARK-28190) Data Source - State

2019-08-20 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16911774#comment-16911774 ] Jungtaek Lim commented on SPARK-28190: -- Thanks for the guide! I'll try to write down SPIP and

[jira] [Updated] (SPARK-27937) Revert changes introduced as a part of Automatic namespace discovery [SPARK-24149]

2019-08-20 Thread Marcelo Vanzin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-27937: --- Docs Text: In Spark 3.0, the behavior for automatic delegation token retrieval for file

[jira] [Resolved] (SPARK-27937) Revert changes introduced as a part of Automatic namespace discovery [SPARK-24149]

2019-08-20 Thread Marcelo Vanzin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-27937. Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 24785

[jira] [Assigned] (SPARK-27937) Revert changes introduced as a part of Automatic namespace discovery [SPARK-24149]

2019-08-20 Thread Marcelo Vanzin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-27937: -- Assignee: Dhruve Ashar > Revert changes introduced as a part of Automatic namespace

[jira] [Commented] (SPARK-28190) Data Source - State

2019-08-20 Thread Jose Torres (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16911678#comment-16911678 ] Jose Torres commented on SPARK-28190: - Yeah, I think an SPIP is needed here. It sounds like we're

[jira] [Assigned] (SPARK-28784) StreamExecution and StreamingQueryManager should utilize CheckpointFileManager to interact with checkpoint directories

2019-08-20 Thread holdenk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] holdenk reassigned SPARK-28784: --- Assignee: Shruti Gumma > StreamExecution and StreamingQueryManager should utilize >

[jira] [Created] (SPARK-28784) StreamExecution and StreamingQueryManager should utilize CheckpointFileManager to interact with checkpoint directories

2019-08-20 Thread Shruti Gumma (Jira)
Shruti Gumma created SPARK-28784: Summary: StreamExecution and StreamingQueryManager should utilize CheckpointFileManager to interact with checkpoint directories Key: SPARK-28784 URL:

[jira] [Commented] (SPARK-28605) Performance regression in SS's foreach

2019-08-20 Thread Shixiong Zhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16911635#comment-16911635 ] Shixiong Zhu commented on SPARK-28605: -- [~kabhwan] Thanks for pointing it out. Yes, we can close

[jira] [Resolved] (SPARK-28605) Performance regression in SS's foreach

2019-08-20 Thread Shixiong Zhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-28605. -- Resolution: Invalid > Performance regression in SS's foreach >

[jira] [Created] (SPARK-28783) Support plus + operator as concatenation for StringType columns in pyspark

2019-08-20 Thread Louis Yang (Jira)
Louis Yang created SPARK-28783: -- Summary: Support plus + operator as concatenation for StringType columns in pyspark Key: SPARK-28783 URL: https://issues.apache.org/jira/browse/SPARK-28783 Project:

[jira] [Commented] (SPARK-28762) Read JAR main class if JAR is not located in local file system

2019-08-20 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16911608#comment-16911608 ] Dongjoon Hyun commented on SPARK-28762: --- Hi, [~igozali]. New feature and improvement cannot affect

[jira] [Commented] (SPARK-28762) Read JAR main class if JAR is not located in local file system

2019-08-20 Thread Ivan Gozali (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16911563#comment-16911563 ] Ivan Gozali commented on SPARK-28762: - Hi [~dongjoon], I'm not sure if you meant to change the Fix

[jira] [Resolved] (SPARK-28498) always create a fresh copy of the SparkSession before each test

2019-08-20 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-28498. - Assignee: (was: Wenchen Fan) Resolution: Won't Fix > always create a fresh copy of

[jira] [Commented] (SPARK-28444) Bump Kubernetes Client Version to 4.3.0

2019-08-20 Thread Ang Gao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16911391#comment-16911391 ] Ang Gao commented on SPARK-28444: - Also the newer version of k9s client starts to support EC Private

[jira] [Updated] (SPARK-28781) Unneccesary persist in PeriodicCheckpointer.update()

2019-08-20 Thread CacheCheck (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] CacheCheck updated SPARK-28781: --- Description: Once the update is called, newData is persisted at line 82. However, only when the

[jira] [Updated] (SPARK-28621) CheckCartesianProducts may throw some error which mismatch generated physical plan

2019-08-20 Thread Weichen Xu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weichen Xu updated SPARK-28621: --- Description: CheckCartesianProducts check logical plan which mismatch the physical plan. So when

[jira] [Updated] (SPARK-28781) Unneccesary persist in PeriodicCheckpointer.update()

2019-08-20 Thread CacheCheck (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] CacheCheck updated SPARK-28781: --- Description: Once the update is called, newData is persisted at line 82. However, only when the

[jira] [Updated] (SPARK-28621) may throw some mismatching error

2019-08-20 Thread Weichen Xu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weichen Xu updated SPARK-28621: --- Summary: may throw some mismatching error (was: CheckCartesianProducts do not work correctly) >

[jira] [Updated] (SPARK-28621) CheckCartesianProducts may throw some error which mismatch generated physical plan

2019-08-20 Thread Weichen Xu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weichen Xu updated SPARK-28621: --- Summary: CheckCartesianProducts may throw some error which mismatch generated physical plan (was:

[jira] [Updated] (SPARK-28621) CheckCartesianProducts do not work correctly

2019-08-20 Thread Weichen Xu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weichen Xu updated SPARK-28621: --- Description: CheckCartesianProducts do not work correctly. 1) CheckCartesianProducts check logical

[jira] [Updated] (SPARK-28621) CheckCartesianProducts do not work correctly

2019-08-20 Thread Weichen Xu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weichen Xu updated SPARK-28621: --- Description: CheckCartesianProducts check logical plan which mismatch the physical plan. So when

[jira] [Created] (SPARK-28782) explode() fails on aggregate expressions

2019-08-20 Thread Weichen Xu (Jira)
Weichen Xu created SPARK-28782: -- Summary: explode() fails on aggregate expressions Key: SPARK-28782 URL: https://issues.apache.org/jira/browse/SPARK-28782 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-28781) Unneccesary persist in PeriodicCheckpointer.update()

2019-08-20 Thread Dong Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dong Wang updated SPARK-28781: -- Environment: (was: Once update(newData) is called, newData is persisted. However, only when the

[jira] [Created] (SPARK-28781) Unneccesary persist in PeriodicCheckpointer.update()

2019-08-20 Thread Dong Wang (Jira)
Dong Wang created SPARK-28781: - Summary: Unneccesary persist in PeriodicCheckpointer.update() Key: SPARK-28781 URL: https://issues.apache.org/jira/browse/SPARK-28781 Project: Spark Issue Type:

[jira] [Commented] (SPARK-28707) Spark SQL select query result size issue

2019-08-20 Thread jobit mathew (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16911269#comment-16911269 ] jobit mathew commented on SPARK-28707: -- Thanks Sujith,Angerszhu and Dongjoon Hyun > Spark SQL

[jira] [Updated] (SPARK-28780) Delete the incorrect setWeightCol method in LinearSVCModel

2019-08-20 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-28780: -- Docs Text: Spark 2.2 accidentally introduced the method LinearSVCModel.setWeightCol. This method

[jira] [Resolved] (SPARK-28753) Dynamically reuse subqueries in AQE

2019-08-20 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-28753. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 25471

[jira] [Assigned] (SPARK-28753) Dynamically reuse subqueries in AQE

2019-08-20 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-28753: --- Assignee: Maryann Xue > Dynamically reuse subqueries in AQE >

[jira] [Resolved] (SPARK-28635) create CatalogManager to track registered v2 catalogs

2019-08-20 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-28635. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 25368

[jira] [Resolved] (SPARK-22531) Migrate the implementation of IDF from MLLib to ML

2019-08-20 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-22531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-22531. --- Resolution: Duplicate > Migrate the implementation of IDF from MLLib to ML >

[jira] [Reopened] (SPARK-22531) Migrate the implementation of IDF from MLLib to ML

2019-08-20 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-22531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reopened SPARK-22531: --- > Migrate the implementation of IDF from MLLib to ML >

[jira] [Reopened] (SPARK-21178) Add support for label specific metrics in MulticlassClassificationEvaluator

2019-08-20 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-21178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reopened SPARK-21178: --- > Add support for label specific metrics in MulticlassClassificationEvaluator >

[jira] [Resolved] (SPARK-21178) Add support for label specific metrics in MulticlassClassificationEvaluator

2019-08-20 Thread Sean Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-21178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21178. --- Resolution: Duplicate > Add support for label specific metrics in MulticlassClassificationEvaluator

[jira] [Commented] (SPARK-22955) Error generating jobs when Stopping JobGenerator gracefully

2019-08-20 Thread Nikita Gorbachevski (Jira)
[ https://issues.apache.org/jira/browse/SPARK-22955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16911208#comment-16911208 ] Nikita Gorbachevski commented on SPARK-22955: - The issue is still exists, submitted PR in

[jira] [Created] (SPARK-28780) Delete the incorrect setWeightCol method in LinearSVCModel

2019-08-20 Thread zhengruifeng (Jira)
zhengruifeng created SPARK-28780: Summary: Delete the incorrect setWeightCol method in LinearSVCModel Key: SPARK-28780 URL: https://issues.apache.org/jira/browse/SPARK-28780 Project: Spark

[jira] [Resolved] (SPARK-28650) Fix the guarantee of ForeachWriter

2019-08-20 Thread Shixiong Zhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-28650. -- Fix Version/s: 2.4.5 Assignee: Jungtaek Lim Resolution: Fixed > Fix the

[jira] [Resolved] (SPARK-28466) FileSystem closed error when to call Hive.moveFile

2019-08-20 Thread angerszhu (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] angerszhu resolved SPARK-28466. --- Resolution: Won't Fix > FileSystem closed error when to call Hive.moveFile >

[jira] [Commented] (SPARK-28770) Test ReplayListenerSuite.End-to-end replay with compression failed

2019-08-20 Thread zhao bo (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16911081#comment-16911081 ] zhao bo commented on SPARK-28770: - Hi Team, could you please to look this problem if you have times? We

[jira] [Assigned] (SPARK-28662) Create Hive Partitioned Table without specifying data type for partition columns will success unexpectedly

2019-08-20 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-28662: --- Assignee: Li Hao > Create Hive Partitioned Table without specifying data type for

[jira] [Resolved] (SPARK-28662) Create Hive Partitioned Table without specifying data type for partition columns will success unexpectedly

2019-08-20 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-28662. - Resolution: Fixed Issue resolved by pull request 25390

[jira] [Resolved] (SPARK-28483) Canceling a spark job using barrier mode but barrier tasks do not exit

2019-08-20 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-28483. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 25235

[jira] [Assigned] (SPARK-28483) Canceling a spark job using barrier mode but barrier tasks do not exit

2019-08-20 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-28483: --- Assignee: Weichen Xu > Canceling a spark job using barrier mode but barrier tasks do not