[jira] [Updated] (SPARK-31935) Hadoop file system config should be effective in data source options

[ https://issues.apache.org/jira/browse/SPARK-31935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-31935: Fix Version/s: 2.4.7 > Hadoop file system config should be effective in data source options >

[jira] [Created] (SPARK-32144) Retain EXTERNAL property in hive table properties

ulysses you created SPARK-32144: --- Summary: Retain EXTERNAL property in hive table properties Key: SPARK-32144 URL: https://issues.apache.org/jira/browse/SPARK-32144 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-32142) Keep the original tests and codes to avoid potential conflicts in dev in ParquetFilterSuite and ParquetIOSuite

[ https://issues.apache.org/jira/browse/SPARK-32142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-32142. -- Fix Version/s: 3.1.0 3.0.1 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-32142) Keep the original tests and codes to avoid potential conflicts in dev in ParquetFilterSuite and ParquetIOSuite

[ https://issues.apache.org/jira/browse/SPARK-32142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-32142: Assignee: Hyukjin Kwon > Keep the original tests and codes to avoid potential conflicts

[jira] [Updated] (SPARK-32131) Fix AnalysisException messages at UNION/EXCEPT/MINUS operations

[ https://issues.apache.org/jira/browse/SPARK-32131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32131: -- Summary: Fix AnalysisException messages at UNION/EXCEPT/MINUS operations (was: union and set

[jira] [Commented] (SPARK-32131) union and set operations have wrong exception infomation

[ https://issues.apache.org/jira/browse/SPARK-32131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17149108#comment-17149108 ] Dongjoon Hyun commented on SPARK-32131: --- I also verified that this bug exists at 2.1.3 ~ 2.3.7 and

[jira] [Updated] (SPARK-32131) union and set operations have wrong exception infomation

[ https://issues.apache.org/jira/browse/SPARK-32131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32131: -- Affects Version/s: 2.1.3 > union and set operations have wrong exception infomation >

[jira] [Updated] (SPARK-32131) union and set operations have wrong exception infomation

[ https://issues.apache.org/jira/browse/SPARK-32131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32131: -- Affects Version/s: 2.2.3 > union and set operations have wrong exception infomation >

[jira] [Updated] (SPARK-32131) union and set operations have wrong exception infomation

[ https://issues.apache.org/jira/browse/SPARK-32131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32131: -- Affects Version/s: 2.3.4 > union and set operations have wrong exception infomation >

[jira] [Updated] (SPARK-32131) union and set operations have wrong exception infomation

[ https://issues.apache.org/jira/browse/SPARK-32131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32131: -- Affects Version/s: 2.4.6 > union and set operations have wrong exception infomation >

[jira] [Commented] (SPARK-32136) Spark producing incorrect groupBy results when key is a struct with nullable properties

[ https://issues.apache.org/jira/browse/SPARK-32136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17149104#comment-17149104 ] L. C. Hsieh commented on SPARK-32136: - Thanks for the ping. Will look at this. > Spark producing

[jira] [Commented] (SPARK-32136) Spark producing incorrect groupBy results when key is a struct with nullable properties

[ https://issues.apache.org/jira/browse/SPARK-32136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17149103#comment-17149103 ] Wenchen Fan commented on SPARK-32136: - cc [~viirya] > Spark producing incorrect groupBy results

[jira] [Commented] (SPARK-32143) Fast fail when the AQE skew join produce too many splits

[ https://issues.apache.org/jira/browse/SPARK-32143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17149098#comment-17149098 ] Apache Spark commented on SPARK-32143: -- User 'LantaoJin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32143) Fast fail when the AQE skew join produce too many splits

[ https://issues.apache.org/jira/browse/SPARK-32143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32143: Assignee: (was: Apache Spark) > Fast fail when the AQE skew join produce too many

[jira] [Commented] (SPARK-32143) Fast fail when the AQE skew join produce too many splits

[ https://issues.apache.org/jira/browse/SPARK-32143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17149095#comment-17149095 ] Apache Spark commented on SPARK-32143: -- User 'LantaoJin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32143) Fast fail when the AQE skew join produce too many splits

[ https://issues.apache.org/jira/browse/SPARK-32143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32143: Assignee: Apache Spark > Fast fail when the AQE skew join produce too many splits >

[jira] [Updated] (SPARK-32143) Fast fail when the AQE skew join produce too many splits

[ https://issues.apache.org/jira/browse/SPARK-32143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lantao Jin updated SPARK-32143: --- Description: In handling skewed SortMergeJoin, when matching partitions from the left side and the

[jira] [Updated] (SPARK-32143) Fast fail when the AQE skew join produce too many splits

[ https://issues.apache.org/jira/browse/SPARK-32143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lantao Jin updated SPARK-32143: --- Description: In handling skewed SortMergeJoin, when matching partitions from the left side and the

[jira] [Commented] (SPARK-32143) Fast fail when the AQE skew join produce too many splits

[ https://issues.apache.org/jira/browse/SPARK-32143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17149084#comment-17149084 ] Lantao Jin commented on SPARK-32143: A PR will be submitted soon. > Fast fail when the AQE skew

[jira] [Updated] (SPARK-32143) Fast fail when the AQE skew join produce too many splits

[ https://issues.apache.org/jira/browse/SPARK-32143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lantao Jin updated SPARK-32143: --- Description: In handling skewed SortMergeJoin, when matching partitions from the left side and the

[jira] [Created] (SPARK-32143) Fast fail when the AQE skew join produce too many splits

Lantao Jin created SPARK-32143: -- Summary: Fast fail when the AQE skew join produce too many splits Key: SPARK-32143 URL: https://issues.apache.org/jira/browse/SPARK-32143 Project: Spark Issue

[jira] [Commented] (SPARK-27780) Shuffle server & client should be versioned to enable smoother upgrade

[ https://issues.apache.org/jira/browse/SPARK-27780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17149083#comment-17149083 ] Jason Moore commented on SPARK-27780: - I encounter this on 3.0.0 running with a much older shuffle

[jira] [Commented] (SPARK-32136) Spark producing incorrect groupBy results when key is a struct with nullable properties

[ https://issues.apache.org/jira/browse/SPARK-32136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17149073#comment-17149073 ] Jason Moore commented on SPARK-32136: - Here is a similar test, and why it's a problem for what I'm

[jira] [Commented] (SPARK-32088) test of pyspark.sql.functions.timestamp_seconds failed if non-american timezone setting

[ https://issues.apache.org/jira/browse/SPARK-32088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17149044#comment-17149044 ] huangtianhua commented on SPARK-32088: -- the test success by the modification

[jira] [Assigned] (SPARK-32142) Keep the original tests and codes to avoid potential conflicts in dev in ParquetFilterSuite and ParquetIOSuite

[ https://issues.apache.org/jira/browse/SPARK-32142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32142: Assignee: (was: Apache Spark) > Keep the original tests and codes to avoid potential

[jira] [Assigned] (SPARK-32142) Keep the original tests and codes to avoid potential conflicts in dev in ParquetFilterSuite and ParquetIOSuite

[ https://issues.apache.org/jira/browse/SPARK-32142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32142: Assignee: Apache Spark > Keep the original tests and codes to avoid potential conflicts

[jira] [Commented] (SPARK-32142) Keep the original tests and codes to avoid potential conflicts in dev in ParquetFilterSuite and ParquetIOSuite

[ https://issues.apache.org/jira/browse/SPARK-32142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17149035#comment-17149035 ] Apache Spark commented on SPARK-32142: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Created] (SPARK-32142) Keep the original tests and codes to avoid potential conflicts in dev in ParquetFilterSuite and ParquetIOSuite

Hyukjin Kwon created SPARK-32142: Summary: Keep the original tests and codes to avoid potential conflicts in dev in ParquetFilterSuite and ParquetIOSuite Key: SPARK-32142 URL:

[jira] [Created] (SPARK-32141) Repartition leads to out of memory

Lekshmi Nair created SPARK-32141: Summary: Repartition leads to out of memory Key: SPARK-32141 URL: https://issues.apache.org/jira/browse/SPARK-32141 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-32130) Spark 3.0 json load performance is unacceptable in comparison of Spark 2.4

[ https://issues.apache.org/jira/browse/SPARK-32130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17149019#comment-17149019 ] Jungtaek Lim commented on SPARK-32130: -- There might be some tricks to make type inference for

[jira] [Commented] (SPARK-32140) Add summary to FMClassificationModel

[ https://issues.apache.org/jira/browse/SPARK-32140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17148985#comment-17148985 ] Apache Spark commented on SPARK-32140: -- User 'huaxingao' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32140) Add summary to FMClassificationModel

[ https://issues.apache.org/jira/browse/SPARK-32140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32140: Assignee: (was: Apache Spark) > Add summary to FMClassificationModel >

[jira] [Commented] (SPARK-32140) Add summary to FMClassificationModel

[ https://issues.apache.org/jira/browse/SPARK-32140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17148984#comment-17148984 ] Apache Spark commented on SPARK-32140: -- User 'huaxingao' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32140) Add summary to FMClassificationModel

[ https://issues.apache.org/jira/browse/SPARK-32140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32140: Assignee: Apache Spark > Add summary to FMClassificationModel >

[jira] [Commented] (SPARK-28664) ORDER BY in aggregate function

[ https://issues.apache.org/jira/browse/SPARK-28664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17148952#comment-17148952 ] Will Zimmerman commented on SPARK-28664: [~yumwang] - Would this allow for the changing of Null

[jira] [Updated] (SPARK-28664) ORDER BY in aggregate function

[ https://issues.apache.org/jira/browse/SPARK-28664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Will Zimmerman updated SPARK-28664: --- Attachment: image-2020-06-30-15-49-46-796.png > ORDER BY in aggregate function >

[jira] [Updated] (SPARK-20249) Add summary for LinearSVCModel

[ https://issues.apache.org/jira/browse/SPARK-20249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxin Gao updated SPARK-20249: --- Parent: SPARK-32139 Issue Type: Sub-task (was: Improvement) > Add summary for

[jira] [Updated] (SPARK-23631) Add summary to RandomForestClassificationModel

[ https://issues.apache.org/jira/browse/SPARK-23631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxin Gao updated SPARK-23631: --- Parent: SPARK-32139 Issue Type: Sub-task (was: New Feature) > Add summary to

[jira] [Created] (SPARK-32140) Add summary to FMClassificationModel

Huaxin Gao created SPARK-32140: -- Summary: Add summary to FMClassificationModel Key: SPARK-32140 URL: https://issues.apache.org/jira/browse/SPARK-32140 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-31893) Add a generic ClassificationSummary trait

[ https://issues.apache.org/jira/browse/SPARK-31893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Huaxin Gao updated SPARK-31893: --- Parent: SPARK-32139 Issue Type: Sub-task (was: Improvement) > Add a generic

[jira] [Created] (SPARK-32139) Unify Classification Training Summary

Huaxin Gao created SPARK-32139: -- Summary: Unify Classification Training Summary Key: SPARK-32139 URL: https://issues.apache.org/jira/browse/SPARK-32139 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-32130) Spark 3.0 json load performance is unacceptable in comparison of Spark 2.4

[ https://issues.apache.org/jira/browse/SPARK-32130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17148936#comment-17148936 ] Sean R. Owen commented on SPARK-32130: -- So, is the issue that it's trying and failing to parse

[jira] [Commented] (SPARK-32088) test of pyspark.sql.functions.timestamp_seconds failed if non-american timezone setting

[ https://issues.apache.org/jira/browse/SPARK-32088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17148913#comment-17148913 ] Apache Spark commented on SPARK-32088: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Updated] (SPARK-32121) ExternalShuffleBlockResolverSuite failed on Windows

[ https://issues.apache.org/jira/browse/SPARK-32121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Pan updated SPARK-32121: -- Issue Type: Bug (was: Test) > ExternalShuffleBlockResolverSuite failed on Windows >

[jira] [Resolved] (SPARK-31336) Support Oracle Kerberos login in JDBC connector

[ https://issues.apache.org/jira/browse/SPARK-31336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-31336. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 28863

[jira] [Updated] (SPARK-31935) Hadoop file system config should be effective in data source options

[ https://issues.apache.org/jira/browse/SPARK-31935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-31935: --- Affects Version/s: (was: 3.0.1) (was: 3.1.0)

[jira] [Assigned] (SPARK-31336) Support Oracle Kerberos login in JDBC connector

[ https://issues.apache.org/jira/browse/SPARK-31336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-31336: - Assignee: Gabor Somogyi > Support Oracle Kerberos login in JDBC connector >

[jira] [Commented] (SPARK-32132) Thriftserver interval returns "4 weeks 2 days" in 2.4 and "30 days" in 3.0

[ https://issues.apache.org/jira/browse/SPARK-32132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17148848#comment-17148848 ] Juliusz Sompolski commented on SPARK-32132: --- Also 2.4 adds "interval" at the start, while 3.0

[jira] [Updated] (SPARK-31935) Hadoop file system config should be effective in data source options

[ https://issues.apache.org/jira/browse/SPARK-31935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-31935: Issue Type: Bug (was: Improvement) > Hadoop file system config should be effective in data

[jira] [Commented] (SPARK-32132) Thriftserver interval returns "4 weeks 2 days" in 2.4 and "30 days" in 3.0

[ https://issues.apache.org/jira/browse/SPARK-32132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17148836#comment-17148836 ] Wenchen Fan commented on SPARK-32132: - We did it intentionally in

[jira] [Updated] (SPARK-32026) Add PrometheusServletSuite

[ https://issues.apache.org/jira/browse/SPARK-32026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eren Avsarogullari updated SPARK-32026: --- Description: This Jira aims to be added _PrometheusServletSuite_. *Note:* This

[jira] [Commented] (SPARK-32119) ExecutorPlugin doesn't work with Standalone Cluster

[ https://issues.apache.org/jira/browse/SPARK-32119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17148790#comment-17148790 ] Kousuke Saruta commented on SPARK-32119: Yeah I know it works with extraClassPath but as you

[jira] [Assigned] (SPARK-32138) Drop Python 2, 3.4 and 3.5 in codes and documentation

[ https://issues.apache.org/jira/browse/SPARK-32138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32138: Assignee: Apache Spark > Drop Python 2, 3.4 and 3.5 in codes and documentation >

[jira] [Assigned] (SPARK-32138) Drop Python 2, 3.4 and 3.5 in codes and documentation

[ https://issues.apache.org/jira/browse/SPARK-32138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32138: Assignee: (was: Apache Spark) > Drop Python 2, 3.4 and 3.5 in codes and

[jira] [Commented] (SPARK-32138) Drop Python 2, 3.4 and 3.5 in codes and documentation

[ https://issues.apache.org/jira/browse/SPARK-32138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17148766#comment-17148766 ] Apache Spark commented on SPARK-32138: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Updated] (SPARK-32138) Drop Python 2, 3.4 and 3.5 in codes and documentation

[ https://issues.apache.org/jira/browse/SPARK-32138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-32138: - Summary: Drop Python 2, 3.4 and 3.5 in codes and documentation (was: Drop Python 2, 3.4 and

[jira] [Updated] (SPARK-29802) Update remaining python scripts in repo to python3 shebang

[ https://issues.apache.org/jira/browse/SPARK-29802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-29802: - Summary: Update remaining python scripts in repo to python3 shebang (was: update remaining

[jira] [Updated] (SPARK-29919) Remove python2 test execution in Jenkins environment

[ https://issues.apache.org/jira/browse/SPARK-29919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-29919: - Summary: Remove python2 test execution in Jenkins environment (was: remove python2 test

[jira] [Comment Edited] (SPARK-29803) remove all instances of 'from __future__ import print_function'

[ https://issues.apache.org/jira/browse/SPARK-29803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17148755#comment-17148755 ] Hyukjin Kwon edited comment on SPARK-29803 at 6/30/20, 3:07 PM: I will

[jira] [Created] (SPARK-32138) Drop Python 2, 3.4 and 3.5 in the main and dev codes

Hyukjin Kwon created SPARK-32138: Summary: Drop Python 2, 3.4 and 3.5 in the main and dev codes Key: SPARK-32138 URL: https://issues.apache.org/jira/browse/SPARK-32138 Project: Spark Issue

[jira] [Updated] (SPARK-29919) remove python2 test execution

[ https://issues.apache.org/jira/browse/SPARK-29919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-29919: - Parent: SPARK-29909 Issue Type: Sub-task (was: Improvement) > remove python2 test

[jira] [Updated] (SPARK-29919) remove python2 test execution

[ https://issues.apache.org/jira/browse/SPARK-29919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-29919: - Parent: (was: SPARK-29909) Issue Type: Improvement (was: Sub-task) > remove

[jira] [Commented] (SPARK-29803) remove all instances of 'from __future__ import print_function'

[ https://issues.apache.org/jira/browse/SPARK-29803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17148755#comment-17148755 ] Hyukjin Kwon commented on SPARK-29803: -- I will do it at SPARK-29909 > remove all instances of

[jira] [Updated] (SPARK-29803) remove all instances of 'from __future__ import print_function'

[ https://issues.apache.org/jira/browse/SPARK-29803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-29803: - Parent: (was: SPARK-29909) Issue Type: Improvement (was: Sub-task) > remove all

[jira] [Resolved] (SPARK-29803) remove all instances of 'from __future__ import print_function'

[ https://issues.apache.org/jira/browse/SPARK-29803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-29803. -- Resolution: Duplicate > remove all instances of 'from __future__ import print_function' >

[jira] [Commented] (SPARK-32119) ExecutorPlugin doesn't work with Standalone Cluster

[ https://issues.apache.org/jira/browse/SPARK-32119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17148746#comment-17148746 ] Thomas Graves commented on SPARK-32119: --- You can specify the jars in extraClassPath but it

[jira] [Commented] (SPARK-32135) Show Spark Driver name on Spark history web page

[ https://issues.apache.org/jira/browse/SPARK-32135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17148742#comment-17148742 ] Thomas Graves commented on SPARK-32135: --- [~gaurangi]can you please clarify what you mean by

[jira] [Commented] (SPARK-31816) Create high level description about JDBC connection providers for users/developers

[ https://issues.apache.org/jira/browse/SPARK-31816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17148712#comment-17148712 ] Gabor Somogyi commented on SPARK-31816: --- Originally I've thought it's enough to add developer

[jira] [Updated] (SPARK-31816) Create high level description about JDBC connection providers for users/developers

[ https://issues.apache.org/jira/browse/SPARK-31816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Somogyi updated SPARK-31816: -- Summary: Create high level description about JDBC connection providers for users/developers

[jira] [Assigned] (SPARK-32068) Spark 3 UI task launch time show in error time zone

[ https://issues.apache.org/jira/browse/SPARK-32068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves reassigned SPARK-32068: - Assignee: JinxinTang > Spark 3 UI task launch time show in error time zone >

[jira] [Resolved] (SPARK-32068) Spark 3 UI task launch time show in error time zone

[ https://issues.apache.org/jira/browse/SPARK-32068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-32068. --- Fix Version/s: 3.1.0 3.0.1 Resolution: Fixed > Spark 3 UI task

[jira] [Commented] (SPARK-31797) Adds TIMESTAMP_SECONDS, TIMESTAMP_MILLIS and TIMESTAMP_MICROS functions

[ https://issues.apache.org/jira/browse/SPARK-31797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17148694#comment-17148694 ] Apache Spark commented on SPARK-31797: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Commented] (SPARK-31797) Adds TIMESTAMP_SECONDS, TIMESTAMP_MILLIS and TIMESTAMP_MICROS functions

[ https://issues.apache.org/jira/browse/SPARK-31797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17148696#comment-17148696 ] Apache Spark commented on SPARK-31797: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Comment Edited] (SPARK-32130) Spark 3.0 json load performance is unacceptable in comparison of Spark 2.4

[ https://issues.apache.org/jira/browse/SPARK-32130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17148663#comment-17148663 ] Sanjeev Mishra edited comment on SPARK-32130 at 6/30/20, 1:35 PM: -- I

[jira] [Commented] (SPARK-32130) Spark 3.0 json load performance is unacceptable in comparison of Spark 2.4

[ https://issues.apache.org/jira/browse/SPARK-32130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17148663#comment-17148663 ] Sanjeev Mishra commented on SPARK-32130: I tried to load entire dataset using above suggestions

[jira] [Updated] (SPARK-32130) Spark 3.0 json load performance is unacceptable in comparison of Spark 2.4

[ https://issues.apache.org/jira/browse/SPARK-32130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sanjeev Mishra updated SPARK-32130: --- Description: We are planning to move to Spark 3 but the read performance of our json files

[jira] [Updated] (SPARK-29919) remove python2 test execution

[ https://issues.apache.org/jira/browse/SPARK-29919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-29919: - Parent: (was: SPARK-27884) Issue Type: Bug (was: Sub-task) > remove python2 test

[jira] [Updated] (SPARK-29919) remove python2 test execution

[ https://issues.apache.org/jira/browse/SPARK-29919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-29919: - Parent: SPARK-29909 Issue Type: Sub-task (was: Bug) > remove python2 test execution >

[jira] [Commented] (SPARK-31060) Handle column names containing `dots` in data source `Filter`

[ https://issues.apache.org/jira/browse/SPARK-31060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17148594#comment-17148594 ] Apache Spark commented on SPARK-31060: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Commented] (SPARK-31026) Parquet predicate pushdown on columns with dots

[ https://issues.apache.org/jira/browse/SPARK-31026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17148593#comment-17148593 ] Apache Spark commented on SPARK-31026: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Commented] (SPARK-31060) Handle column names containing `dots` in data source `Filter`

[ https://issues.apache.org/jira/browse/SPARK-31060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17148592#comment-17148592 ] Apache Spark commented on SPARK-31060: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Commented] (SPARK-31026) Parquet predicate pushdown on columns with dots

[ https://issues.apache.org/jira/browse/SPARK-31026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17148590#comment-17148590 ] Apache Spark commented on SPARK-31026: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Commented] (SPARK-17636) Parquet predicate pushdown for nested fields

[ https://issues.apache.org/jira/browse/SPARK-17636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17148589#comment-17148589 ] Apache Spark commented on SPARK-17636: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Commented] (SPARK-17636) Parquet predicate pushdown for nested fields

[ https://issues.apache.org/jira/browse/SPARK-17636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17148588#comment-17148588 ] Apache Spark commented on SPARK-17636: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Commented] (SPARK-25556) Predicate Pushdown for Nested fields

[ https://issues.apache.org/jira/browse/SPARK-25556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17148587#comment-17148587 ] Apache Spark commented on SPARK-25556: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Commented] (SPARK-32083) Unnecessary tasks are launched when input is empty with AQE

[ https://issues.apache.org/jira/browse/SPARK-32083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17148585#comment-17148585 ] Apache Spark commented on SPARK-32083: -- User 'manuzhang' has created a pull request for this issue:

[jira] [Commented] (SPARK-32083) Unnecessary tasks are launched when input is empty with AQE

[ https://issues.apache.org/jira/browse/SPARK-32083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17148584#comment-17148584 ] Apache Spark commented on SPARK-32083: -- User 'manuzhang' has created a pull request for this issue:

[jira] [Updated] (SPARK-32137) AttributeError: Can only use .dt accessor with datetimelike values

[ https://issues.apache.org/jira/browse/SPARK-32137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Lacalle Castillo updated SPARK-32137: --- Priority: Critical (was: Major) > AttributeError: Can only use .dt

[jira] [Created] (SPARK-32137) AttributeError: Can only use .dt accessor with datetimelike values

David Lacalle Castillo created SPARK-32137: -- Summary: AttributeError: Can only use .dt accessor with datetimelike values Key: SPARK-32137 URL: https://issues.apache.org/jira/browse/SPARK-32137

[jira] [Commented] (SPARK-32130) Spark 3.0 json load performance is unacceptable in comparison of Spark 2.4

[ https://issues.apache.org/jira/browse/SPARK-32130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17148570#comment-17148570 ] Jungtaek Lim commented on SPARK-32130: -- Looks like we already saw the difference but we missed to

[jira] [Commented] (SPARK-32130) Spark 3.0 json load performance is unacceptable in comparison of Spark 2.4

[ https://issues.apache.org/jira/browse/SPARK-32130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17148559#comment-17148559 ] Jungtaek Lim commented on SPARK-32130: -- So anyone can just reproduce via running spark-shell on

[jira] [Commented] (SPARK-32130) Spark 3.0 json load performance is unacceptable in comparison of Spark 2.4

[ https://issues.apache.org/jira/browse/SPARK-32130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17148546#comment-17148546 ] Jungtaek Lim commented on SPARK-32130: -- For me it's reproduced consistently. Please make sure you

[jira] [Commented] (SPARK-32130) Spark 3.0 json load performance is unacceptable in comparison of Spark 2.4

[ https://issues.apache.org/jira/browse/SPARK-32130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17148501#comment-17148501 ] JinxinTang commented on SPARK-32130: [~gourav.sengupta] Nice notebook, is seems the row count() is 

[jira] [Comment Edited] (SPARK-32130) Spark 3.0 json load performance is unacceptable in comparison of Spark 2.4

[ https://issues.apache.org/jira/browse/SPARK-32130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17148459#comment-17148459 ] Gourav edited comment on SPARK-32130 at 6/30/20, 9:02 AM: -- [~JinxinTang] and

[jira] [Commented] (SPARK-32130) Spark 3.0 json load performance is unacceptable in comparison of Spark 2.4

[ https://issues.apache.org/jira/browse/SPARK-32130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17148459#comment-17148459 ] Gourav commented on SPARK-32130: [~JinxinTang] and [~lotus2you] I think that it is only the first time

[jira] [Updated] (SPARK-32130) Spark 3.0 json load performance is unacceptable in comparison of Spark 2.4

[ https://issues.apache.org/jira/browse/SPARK-32130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gourav updated SPARK-32130: --- Attachment: SPARK 32130 - replication and findings.ipynb > Spark 3.0 json load performance is unacceptable

[jira] [Updated] (SPARK-32130) Spark 3.0 json load performance is unacceptable in comparison of Spark 2.4

[ https://issues.apache.org/jira/browse/SPARK-32130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gourav updated SPARK-32130: --- Attachment: (was: SPARK 32130 - replication and findings.ipynb) > Spark 3.0 json load performance is

[jira] [Updated] (SPARK-32130) Spark 3.0 json load performance is unacceptable in comparison of Spark 2.4

[ https://issues.apache.org/jira/browse/SPARK-32130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gourav updated SPARK-32130: --- Attachment: SPARK 32130 - replication and findings.ipynb > Spark 3.0 json load performance is unacceptable

[jira] [Updated] (SPARK-32136) Spark producing incorrect groupBy results when key is a struct with nullable properties

[ https://issues.apache.org/jira/browse/SPARK-32136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Moore updated SPARK-32136: Description: I'm in the process of migrating from Spark 2.4.x to Spark 3.0.0 and I'm noticing a

[jira] [Created] (SPARK-32136) Spark producing incorrect groupBy results when key is a struct with nullable properties

Jason Moore created SPARK-32136: --- Summary: Spark producing incorrect groupBy results when key is a struct with nullable properties Key: SPARK-32136 URL: https://issues.apache.org/jira/browse/SPARK-32136

  1   2   >