[jira] [Issue Comment Deleted] (SPARK-32282) Improve EnsureRquirement.reorderJoinKeys to handle more scenarios such as PartitioningCollection

2020-07-11 Thread Terry Kim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Terry Kim updated SPARK-32282: -- Comment: was deleted (was: [https://github.com/apache/spark/pull/29074]) > Improve

[jira] [Assigned] (SPARK-32282) Improve EnsureRquirement.reorderJoinKeys to handle more scenarios such as PartitioningCollection

2020-07-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32282: Assignee: Apache Spark > Improve EnsureRquirement.reorderJoinKeys to handle more

[jira] [Commented] (SPARK-32282) Improve EnsureRquirement.reorderJoinKeys to handle more scenarios such as PartitioningCollection

2020-07-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17156187#comment-17156187 ] Apache Spark commented on SPARK-32282: -- User 'imback82' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32282) Improve EnsureRquirement.reorderJoinKeys to handle more scenarios such as PartitioningCollection

2020-07-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32282: Assignee: Apache Spark > Improve EnsureRquirement.reorderJoinKeys to handle more

[jira] [Commented] (SPARK-32282) Improve EnsureRquirement.reorderJoinKeys to handle more scenarios such as PartitioningCollection

2020-07-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17156188#comment-17156188 ] Apache Spark commented on SPARK-32282: -- User 'imback82' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32282) Improve EnsureRquirement.reorderJoinKeys to handle more scenarios such as PartitioningCollection

2020-07-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32282: Assignee: (was: Apache Spark) > Improve EnsureRquirement.reorderJoinKeys to handle

[jira] [Commented] (SPARK-32282) Improve EnsureRquirement.reorderJoinKeys to handle more scenarios such as PartitioningCollection

2020-07-11 Thread Terry Kim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17156186#comment-17156186 ] Terry Kim commented on SPARK-32282: --- [https://github.com/apache/spark/pull/29074] > Improve

[jira] [Created] (SPARK-32282) Improve EnsureRquirement.reorderJoinKeys to handle more scenarios such as PartitioningCollection

2020-07-11 Thread Terry Kim (Jira)
Terry Kim created SPARK-32282: - Summary: Improve EnsureRquirement.reorderJoinKeys to handle more scenarios such as PartitioningCollection Key: SPARK-32282 URL: https://issues.apache.org/jira/browse/SPARK-32282

[jira] [Updated] (SPARK-32271) Update CrossValidator to parallelize fit method across folds

2020-07-11 Thread Austin Jordan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Austin Jordan updated SPARK-32271: -- Description: Currently, fitting a CrossValidator is only parallelized across models. This

[jira] [Updated] (SPARK-32228) Partition column of hive table was capitalized as directory name when stored on HDFS

2020-07-11 Thread Kernel Force (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kernel Force updated SPARK-32228: - Summary: Partition column of hive table was capitalized as directory name when stored on HDFS

[jira] [Resolved] (SPARK-32154) Use ExpressionEncoder for the return type of ScalaUDF to convert to catalyst type

2020-07-11 Thread Takeshi Yamamuro (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-32154. -- Fix Version/s: 3.1.0 Assignee: wuyi Resolution: Fixed Resolved by

[jira] [Resolved] (SPARK-29358) Make unionByName optionally fill missing columns with nulls

2020-07-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-29358. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 28996

[jira] [Assigned] (SPARK-29358) Make unionByName optionally fill missing columns with nulls

2020-07-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-29358: - Assignee: L. C. Hsieh > Make unionByName optionally fill missing columns with nulls >

[jira] [Assigned] (SPARK-29292) Fix internal usages of mutable collection for Seq in 2.13

2020-07-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-29292: Assignee: Sean R. Owen (was: Apache Spark) > Fix internal usages of mutable collection

[jira] [Assigned] (SPARK-29292) Fix internal usages of mutable collection for Seq in 2.13

2020-07-11 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-29292: Assignee: Apache Spark (was: Sean R. Owen) > Fix internal usages of mutable collection

[jira] [Reopened] (SPARK-29292) Fix internal usages of mutable collection for Seq in 2.13

2020-07-11 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reopened SPARK-29292: -- Oh this isn't resolved yet; will take a few more PRs. > Fix internal usages of mutable

[jira] [Resolved] (SPARK-31226) Fix SizeBasedCoalesce in tests

2020-07-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-31226. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 27988

[jira] [Assigned] (SPARK-31226) Fix SizeBasedCoalesce in tests

2020-07-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-31226: - Assignee: angerszhu > Fix SizeBasedCoalesce in tests > --

[jira] [Resolved] (SPARK-29292) Fix internal usages of mutable collection for Seq in 2.13

2020-07-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-29292. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 28971

[jira] [Assigned] (SPARK-30010) Remove deprecated SparkConf.setAll(Traversable)

2020-07-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-30010: - Assignee: Sean R. Owen > Remove deprecated SparkConf.setAll(Traversable) >

[jira] [Resolved] (SPARK-30010) Remove deprecated SparkConf.setAll(Traversable)

2020-07-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-30010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-30010. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 28971

[jira] [Resolved] (SPARK-32200) Redirect to the history page when accessed to /history without appliation id

2020-07-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-32200. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29016

[jira] [Resolved] (SPARK-32245) Implement the base to run Spark tests in GitHun Actions

2020-07-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-32245. --- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 29057

[jira] [Commented] (SPARK-31635) Spark SQL Sort fails when sorting big data points

2020-07-11 Thread Chen Zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17156111#comment-17156111 ] Chen Zhang commented on SPARK-31635: Hello [~george21], I have submitted a PR. Please take a look.

[jira] [Updated] (SPARK-32281) Spark wipes out SORTED spec in metastore when DESC is used

2020-07-11 Thread Bruce Robbins (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-32281: -- Summary: Spark wipes out SORTED spec in metastore when DESC is used (was: Spark wipes out

[jira] [Created] (SPARK-32281) Spark wipes out SORTED spec in metastore when when DESC is used

2020-07-11 Thread Bruce Robbins (Jira)
Bruce Robbins created SPARK-32281: - Summary: Spark wipes out SORTED spec in metastore when when DESC is used Key: SPARK-32281 URL: https://issues.apache.org/jira/browse/SPARK-32281 Project: Spark

[jira] [Created] (SPARK-32280) AnalysisException thrown when query contains several JOINs

2020-07-11 Thread Jira
David Lindelöf created SPARK-32280: -- Summary: AnalysisException thrown when query contains several JOINs Key: SPARK-32280 URL: https://issues.apache.org/jira/browse/SPARK-32280 Project: Spark

[jira] [Resolved] (SPARK-32150) Upgrade to Zstd 1.4.5-4

2020-07-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-32150. -- Resolution: Fixed Issue resolved by pull request 28969

[jira] [Resolved] (SPARK-32232) IllegalArgumentException: MultilayerPerceptronClassifier_... parameter solver given invalid value auto

2020-07-11 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-32232. -- Fix Version/s: 3.1.0 3.0.1 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-32232) IllegalArgumentException: MultilayerPerceptronClassifier_... parameter solver given invalid value auto

2020-07-11 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-32232: Assignee: Huaxin Gao > IllegalArgumentException: MultilayerPerceptronClassifier_...

[jira] [Resolved] (SPARK-32173) Deduplicate code in FromUTCTimestamp and ToUTCTimestamp

2020-07-11 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-32173. -- Fix Version/s: 3.1.0 Assignee: Maxim Gekk Resolution: Fixed Fixed in

[jira] [Assigned] (SPARK-32149) Improve file path name normalisation at block resolution within the external shuffle service

2020-07-11 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim reassigned SPARK-32149: Assignee: Attila Zsolt Piros > Improve file path name normalisation at block resolution

[jira] [Resolved] (SPARK-32149) Improve file path name normalisation at block resolution within the external shuffle service

2020-07-11 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-32149. -- Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 28967

[jira] [Assigned] (SPARK-32238) Use Utils.getSimpleName to avoid hitting Malformed class name in ScalaUDF

2020-07-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-32238: - Assignee: wuyi > Use Utils.getSimpleName to avoid hitting Malformed class name in

[jira] [Resolved] (SPARK-32238) Use Utils.getSimpleName to avoid hitting Malformed class name in ScalaUDF

2020-07-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-32238. --- Fix Version/s: 3.1.0 2.4.7 3.0.1 Resolution:

[jira] [Commented] (SPARK-32220) Cartesian Product Hint cause data error

2020-07-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17155984#comment-17155984 ] Dongjoon Hyun commented on SPARK-32220: --- This lands at branch-3.0 via

[jira] [Updated] (SPARK-32220) Cartesian Product Hint cause data error

2020-07-11 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-32220: -- Fix Version/s: 3.0.1 > Cartesian Product Hint cause data error >