[jira] [Created] (SPARK-32814) Metaclasses are broken for a few classes in Python 3

2020-09-07 Thread Maciej Szymkiewicz (Jira)
Maciej Szymkiewicz created SPARK-32814: -- Summary: Metaclasses are broken for a few classes in Python 3 Key: SPARK-32814 URL: https://issues.apache.org/jira/browse/SPARK-32814 Project: Spark

[jira] [Commented] (SPARK-31511) Make BytesToBytesMap iterator() thread-safe

2020-09-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17191954#comment-17191954 ] Apache Spark commented on SPARK-31511: -- User 'cxzl25' has created a pull request for this issue:

[jira] [Commented] (SPARK-31511) Make BytesToBytesMap iterator() thread-safe

2020-09-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17191953#comment-17191953 ] Apache Spark commented on SPARK-31511: -- User 'cxzl25' has created a pull request for this issue:

[jira] [Resolved] (SPARK-32736) Avoid caching the removed decommissioned executors in TaskSchedulerImpl

2020-09-07 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-32736. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29579

[jira] [Updated] (SPARK-32764) compare of -0.0 < 0.0 return true

2020-09-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32764: -- Fix Version/s: (was: 3.0.1) 3.0.2 > compare of -0.0 < 0.0 return true

[jira] [Resolved] (SPARK-32764) compare of -0.0 < 0.0 return true

2020-09-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-32764. --- Fix Version/s: 3.1.0 3.0.1 Resolution: Fixed Issue resolved by

[jira] [Assigned] (SPARK-32764) compare of -0.0 < 0.0 return true

2020-09-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-32764: - Assignee: Wenchen Fan > compare of -0.0 < 0.0 return true >

[jira] [Resolved] (SPARK-32796) Make withField API support nested struct in array

2020-09-07 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-32796. - Resolution: Won't Fix > Make withField API support nested struct in array >

[jira] [Comment Edited] (SPARK-32354) Fix & re-enable failing R K8s tests

2020-09-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17191929#comment-17191929 ] Dongjoon Hyun edited comment on SPARK-32354 at 9/8/20, 3:29 AM: As I

[jira] [Commented] (SPARK-32354) Fix & re-enable failing R K8s tests

2020-09-07 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17191929#comment-17191929 ] Dongjoon Hyun commented on SPARK-32354: --- As I reported during 3.0.1 RC, `dev/make-distribution.sh`

[jira] [Updated] (SPARK-31511) Make BytesToBytesMap iterator() thread-safe

2020-09-07 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-31511: Fix Version/s: 2.4.7 > Make BytesToBytesMap iterator() thread-safe >

[jira] [Assigned] (SPARK-32812) Run tests script for Python fails in certain environments

2020-09-07 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-32812: Assignee: Haejoon Lee > Run tests script for Python fails in certain environments >

[jira] [Resolved] (SPARK-32812) Run tests script for Python fails in certain environments

2020-09-07 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-32812. -- Fix Version/s: 2.4.8 3.0.2 3.1.0 Resolution:

[jira] [Updated] (SPARK-32813) Reading parquet rdd in non columnar mode fails in multithreaded environment

2020-09-07 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh updated SPARK-32813: Affects Version/s: 3.1.0 > Reading parquet rdd in non columnar mode fails in multithreaded

[jira] [Assigned] (SPARK-32813) Reading parquet rdd in non columnar mode fails in multithreaded environment

2020-09-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32813: Assignee: Apache Spark > Reading parquet rdd in non columnar mode fails in multithreaded

[jira] [Commented] (SPARK-32813) Reading parquet rdd in non columnar mode fails in multithreaded environment

2020-09-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17191904#comment-17191904 ] Apache Spark commented on SPARK-32813: -- User 'viirya' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32813) Reading parquet rdd in non columnar mode fails in multithreaded environment

2020-09-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32813: Assignee: (was: Apache Spark) > Reading parquet rdd in non columnar mode fails in

[jira] [Resolved] (SPARK-32186) Development - Debugging

2020-09-07 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-32186. -- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29639

[jira] [Updated] (SPARK-32812) Run tests script for Python fails in certain environments

2020-09-07 Thread Haejoon Lee (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haejoon Lee updated SPARK-32812: Summary: Run tests script for Python fails in certain environments (was: Run tests script for

[jira] [Commented] (SPARK-32812) Run tests script for Python fails on local environment

2020-09-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17191899#comment-17191899 ] Apache Spark commented on SPARK-32812: -- User 'itholic' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32812) Run tests script for Python fails on local environment

2020-09-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32812: Assignee: (was: Apache Spark) > Run tests script for Python fails on local

[jira] [Commented] (SPARK-32812) Run tests script for Python fails on local environment

2020-09-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17191898#comment-17191898 ] Apache Spark commented on SPARK-32812: -- User 'itholic' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32812) Run tests script for Python fails on local environment

2020-09-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32812: Assignee: Apache Spark > Run tests script for Python fails on local environment >

[jira] [Resolved] (SPARK-32810) CSV/JSON data sources should avoid globbing paths when inferring schema

2020-09-07 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-32810. -- Fix Version/s: 2.4.8 3.0.2 3.1.0 Resolution:

[jira] [Resolved] (SPARK-32798) Make unionByName optionally fill missing columns with nulls in PySpark

2020-09-07 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-32798. -- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29657

[jira] [Assigned] (SPARK-32798) Make unionByName optionally fill missing columns with nulls in PySpark

2020-09-07 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-32798: Assignee: Haejoon Lee > Make unionByName optionally fill missing columns with nulls in

[jira] [Updated] (SPARK-32813) Reading parquet rdd in non columnar mode fails in multithreaded environment

2020-09-07 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh updated SPARK-32813: Priority: Major (was: Blocker) > Reading parquet rdd in non columnar mode fails in multithreaded

[jira] [Commented] (SPARK-32753) Deduplicating and repartitioning the same column create duplicate rows with AQE

2020-09-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17191882#comment-17191882 ] Apache Spark commented on SPARK-32753: -- User 'manuzhang' has created a pull request for this issue:

[jira] [Commented] (SPARK-32138) Drop Python 2, 3.4 and 3.5 in codes and documentation

2020-09-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17191876#comment-17191876 ] Apache Spark commented on SPARK-32138: -- User 'zero323' has created a pull request for this issue:

[jira] [Updated] (SPARK-32813) Reading parquet rdd in non columnar mode fails in multithreaded environment

2020-09-07 Thread Vladimir Klyushnikov (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Klyushnikov updated SPARK-32813: - Description: Reading parquet rdd in non columnar mode (i.e. with list fields)

[jira] [Commented] (SPARK-32810) CSV/JSON data sources should avoid globbing paths when inferring schema

2020-09-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17191822#comment-17191822 ] Apache Spark commented on SPARK-32810: -- User 'MaxGekk' has created a pull request for this issue:

subscribe

2020-09-07 Thread Bowen Li

[jira] [Commented] (SPARK-32603) CREATE/REPLACE TABLE AS SELECT not support multi-part identifiers

2020-09-07 Thread Huaxin Gao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17191813#comment-17191813 ] Huaxin Gao commented on SPARK-32603: I thought CREATE TABLE syntax will be unified for hive and

[jira] [Updated] (SPARK-32813) Reading parquet rdd in non columnar mode fails in multithreaded environment

2020-09-07 Thread Vladimir Klyushnikov (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Klyushnikov updated SPARK-32813: - Environment: Spark 3.0.0, Scala 2.12.12 > Reading parquet rdd in non columnar

[jira] [Updated] (SPARK-32813) Reading parquet rdd in non columnar mode fails in multithreaded environment

2020-09-07 Thread Vladimir Klyushnikov (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Klyushnikov updated SPARK-32813: - Description: Reading parquet rdd in non columnar mode (i.e. with list fields)

[jira] [Updated] (SPARK-32813) Reading parquet rdd in non columnar mode fails in multithreaded environment

2020-09-07 Thread Vladimir Klyushnikov (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Klyushnikov updated SPARK-32813: - Description: Reading parquet rdd in non columnar mode (i.e. with list fields)

[jira] [Created] (SPARK-32813) Reading parquet rdd in non columnar mode fails in multithreaded environment

2020-09-07 Thread Vladimir Klyushnikov (Jira)
Vladimir Klyushnikov created SPARK-32813: Summary: Reading parquet rdd in non columnar mode fails in multithreaded environment Key: SPARK-32813 URL: https://issues.apache.org/jira/browse/SPARK-32813

[jira] [Assigned] (SPARK-32753) Deduplicating and repartitioning the same column create duplicate rows with AQE

2020-09-07 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-32753: --- Assignee: Manu Zhang > Deduplicating and repartitioning the same column create duplicate

[jira] [Resolved] (SPARK-32753) Deduplicating and repartitioning the same column create duplicate rows with AQE

2020-09-07 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-32753. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29593

[jira] [Commented] (SPARK-32810) CSV/JSON data sources should avoid globbing paths when inferring schema

2020-09-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17191742#comment-17191742 ] Apache Spark commented on SPARK-32810: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Commented] (SPARK-32810) CSV/JSON data sources should avoid globbing paths when inferring schema

2020-09-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17191741#comment-17191741 ] Apache Spark commented on SPARK-32810: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Updated] (SPARK-32812) Run tests script for Python fails on local environment

2020-09-07 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-32812: - Component/s: PySpark > Run tests script for Python fails on local environment >

[jira] [Updated] (SPARK-32812) Run tests script for Python fails on local environment

2020-09-07 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-32812: - Component/s: (was: PySpark) Tests > Run tests script for Python fails on

[jira] [Created] (SPARK-32812) Run tests script for Python fails on local environment

2020-09-07 Thread Haejoon Lee (Jira)
Haejoon Lee created SPARK-32812: --- Summary: Run tests script for Python fails on local environment Key: SPARK-32812 URL: https://issues.apache.org/jira/browse/SPARK-32812 Project: Spark Issue

[jira] [Commented] (SPARK-32811) Replace IN predicate of continuous range with boundary checks

2020-09-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17191645#comment-17191645 ] Apache Spark commented on SPARK-32811: -- User 'hotienvu' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32811) Replace IN predicate of continuous range with boundary checks

2020-09-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32811: Assignee: (was: Apache Spark) > Replace IN predicate of continuous range with

[jira] [Assigned] (SPARK-32811) Replace IN predicate of continuous range with boundary checks

2020-09-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32811: Assignee: Apache Spark > Replace IN predicate of continuous range with boundary checks >

[jira] [Commented] (SPARK-32811) Replace IN predicate of continuous range with boundary checks

2020-09-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17191642#comment-17191642 ] Apache Spark commented on SPARK-32811: -- User 'hotienvu' has created a pull request for this issue:

[jira] [Commented] (SPARK-32808) Pass all `sql/core` module UTs in Scala 2.13

2020-09-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17191611#comment-17191611 ] Apache Spark commented on SPARK-32808: -- User 'LuciferYang' has created a pull request for this

[jira] [Assigned] (SPARK-32808) Pass all `sql/core` module UTs in Scala 2.13

2020-09-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32808: Assignee: Apache Spark > Pass all `sql/core` module UTs in Scala 2.13 >

[jira] [Assigned] (SPARK-32808) Pass all `sql/core` module UTs in Scala 2.13

2020-09-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32808: Assignee: (was: Apache Spark) > Pass all `sql/core` module UTs in Scala 2.13 >

[jira] [Created] (SPARK-32811) Replace IN predicate of continuous range with boundary checks

2020-09-07 Thread Vu Ho (Jira)
Vu Ho created SPARK-32811: - Summary: Replace IN predicate of continuous range with boundary checks Key: SPARK-32811 URL: https://issues.apache.org/jira/browse/SPARK-32811 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-32810) CSV/JSON data sources should avoid globbing paths when inferring schema

2020-09-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32810: Assignee: (was: Apache Spark) > CSV/JSON data sources should avoid globbing paths

[jira] [Commented] (SPARK-32810) CSV/JSON data sources should avoid globbing paths when inferring schema

2020-09-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17191578#comment-17191578 ] Apache Spark commented on SPARK-32810: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32810) CSV/JSON data sources should avoid globbing paths when inferring schema

2020-09-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32810: Assignee: Apache Spark > CSV/JSON data sources should avoid globbing paths when

[jira] [Assigned] (SPARK-32810) CSV/JSON data sources should avoid globbing paths when inferring schema

2020-09-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32810: Assignee: Apache Spark > CSV/JSON data sources should avoid globbing paths when

[jira] [Commented] (SPARK-32785) interval with dangling part should not results null

2020-09-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17191513#comment-17191513 ] Apache Spark commented on SPARK-32785: -- User 'yaooqinn' has created a pull request for this issue:

[jira] [Commented] (SPARK-32785) interval with dangling part should not results null

2020-09-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17191511#comment-17191511 ] Apache Spark commented on SPARK-32785: -- User 'yaooqinn' has created a pull request for this issue:

[jira] [Commented] (SPARK-32810) CSV/JSON data sources should avoid globbing paths when inferring schema

2020-09-07 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17191510#comment-17191510 ] Maxim Gekk commented on SPARK-32810: I am working on this. > CSV/JSON data sources should avoid

[jira] [Created] (SPARK-32810) CSV/JSON data sources should avoid globbing paths when inferring schema

2020-09-07 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-32810: -- Summary: CSV/JSON data sources should avoid globbing paths when inferring schema Key: SPARK-32810 URL: https://issues.apache.org/jira/browse/SPARK-32810 Project: Spark

[jira] [Assigned] (SPARK-32798) Make unionByName optionally fill missing columns with nulls in PySpark

2020-09-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32798: Assignee: Apache Spark > Make unionByName optionally fill missing columns with nulls in

[jira] [Commented] (SPARK-32798) Make unionByName optionally fill missing columns with nulls in PySpark

2020-09-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17191507#comment-17191507 ] Apache Spark commented on SPARK-32798: -- User 'itholic' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32798) Make unionByName optionally fill missing columns with nulls in PySpark

2020-09-07 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32798: Assignee: (was: Apache Spark) > Make unionByName optionally fill missing columns

[jira] [Reopened] (SPARK-32809) RDD different partitions cause didderent results

2020-09-07 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reopened SPARK-32809: -- > RDD different partitions cause didderent results >

[jira] [Resolved] (SPARK-32809) RDD different partitions cause didderent results

2020-09-07 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-32809. -- Resolution: Invalid > RDD different partitions cause didderent results >

[jira] [Commented] (SPARK-32809) RDD different partitions cause didderent results

2020-09-07 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17191485#comment-17191485 ] Hyukjin Kwon commented on SPARK-32809: -- The results is correct. For {{local[1]}}, there's a single

[jira] [Updated] (SPARK-32809) RDD different partitions cause didderent results

2020-09-07 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-32809: - Description: {code} class Exec3 { private val exec: SparkConf = new

[jira] [Updated] (SPARK-32808) Pass all `sql/core` module UTs in Scala 2.13

2020-09-07 Thread Yang Jie (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yang Jie updated SPARK-32808: - Description: Now there are  319 TESTS FAILED based on commit

[jira] [Updated] (SPARK-32809) RDD different partitions cause didderent results

2020-09-07 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-32809: - Priority: Major (was: Blocker) > RDD different partitions cause didderent results >

[jira] [Updated] (SPARK-32809) RDD different partitions cause didderent results

2020-09-07 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-32809: - Issue Type: Bug (was: Wish) > RDD different partitions cause didderent results >

[jira] [Updated] (SPARK-32809) RDD different partitions cause didderent results

2020-09-07 Thread zhangchenglong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhangchenglong updated SPARK-32809: --- Flags: Important Environment: spark2.2.0 ,scala 2.11.8 ,

[jira] [Commented] (SPARK-32809) RDD分区数对于计算结果的影响

2020-09-07 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17191475#comment-17191475 ] Hyukjin Kwon commented on SPARK-32809: -- Please also fix the title. What's the expected results and

[jira] [Updated] (SPARK-32809) RDD分区数对于计算结果的影响

2020-09-07 Thread zhangchenglong (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhangchenglong updated SPARK-32809: --- Description: class Exec3 { private val exec: SparkConf = new

[jira] [Assigned] (SPARK-32748) Support local property propagation in SubqueryBroadcastExec

2020-09-07 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-32748: --- Assignee: Zhenhua Wang > Support local property propagation in SubqueryBroadcastExec >

[jira] [Resolved] (SPARK-32748) Support local property propagation in SubqueryBroadcastExec

2020-09-07 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-32748. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29589

[jira] [Commented] (SPARK-32778) Accidental Data Deletion on calling saveAsTable

2020-09-07 Thread Aman Rastogi (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17191472#comment-17191472 ] Aman Rastogi commented on SPARK-32778: -- [~hyukjin.kwon] Sure, will reproduce with 2.4 > Accidental

[jira] [Resolved] (SPARK-32809) RDD分区数对于计算结果的影响

2020-09-07 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-32809. -- Resolution: Incomplete Please write in English which the community use to communicate. >

[jira] [Commented] (SPARK-32778) Accidental Data Deletion on calling saveAsTable

2020-09-07 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17191468#comment-17191468 ] Hyukjin Kwon commented on SPARK-32778: -- Spark 2.3 is EOL too .. can you reproduce in Spark 2.4 or

[jira] [Updated] (SPARK-32807) Spark ThriftServer multisession mode set DB use direct API

2020-09-07 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-32807: - Issue Type: Improvement (was: Bug) > Spark ThriftServer multisession mode set DB use direct

[jira] [Updated] (SPARK-32807) Spark ThriftServer multisession mode set DB use direct API

2020-09-07 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-32807: - Priority: Trivial (was: Major) > Spark ThriftServer multisession mode set DB use direct API >

[jira] [Resolved] (SPARK-32779) Spark/Hive3 interaction potentially causes deadlock

2020-09-07 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-32779. -- Fix Version/s: 3.1.0 3.0.1 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-32779) Spark/Hive3 interaction potentially causes deadlock

2020-09-07 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-32779: Assignee: Sandeep Katta > Spark/Hive3 interaction potentially causes deadlock >

[jira] [Updated] (SPARK-32779) Spark/Hive3 interaction potentially causes deadlock

2020-09-07 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32779?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-32779: - Fix Version/s: (was: 3.0.1) 3.0.2 > Spark/Hive3 interaction potentially

[jira] [Created] (SPARK-32809) RDD分区数对于计算结果的影响

2020-09-07 Thread zhangchenglong (Jira)
zhangchenglong created SPARK-32809: -- Summary: RDD分区数对于计算结果的影响 Key: SPARK-32809 URL: https://issues.apache.org/jira/browse/SPARK-32809 Project: Spark Issue Type: Bug Components:

[jira] [Commented] (SPARK-32793) Expose assert_true in Python/Scala APIs and add error message parameter

2020-09-07 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17191457#comment-17191457 ] Takeshi Yamamuro commented on SPARK-32793: -- You don't add it into SQL? I think BigQuery has a

[jira] [Assigned] (SPARK-32677) Load function resource before create

2020-09-07 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-32677: --- Assignee: ulysses you > Load function resource before create >

[jira] [Resolved] (SPARK-32677) Load function resource before create

2020-09-07 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-32677. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29502