[jira] [Updated] (SPARK-32046) current_timestamp called in a cache dataframe freezes the time for all future calls

2020-06-29 Thread Dustin Smith (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dustin Smith updated SPARK-32046: - Description: If I call current_timestamp 3 times while caching the dataframe variable in order

[jira] [Updated] (SPARK-32046) current_timestamp called in a cache dataframe freezes the time for all future calls

2020-06-29 Thread Dustin Smith (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dustin Smith updated SPARK-32046: - Labels: caching sql time (was: ) > current_timestamp called in a cache dataframe freezes the

[jira] [Created] (SPARK-32135) Show Spark Driver name on Spark history web page

2020-06-29 Thread Gaurangi Saxena (Jira)
Gaurangi Saxena created SPARK-32135: --- Summary: Show Spark Driver name on Spark history web page Key: SPARK-32135 URL: https://issues.apache.org/jira/browse/SPARK-32135 Project: Spark Issue

[jira] [Commented] (SPARK-32119) ExecutorPlugin doesn't work with Standalone Cluster

2020-06-29 Thread Kousuke Saruta (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17148257#comment-17148257 ] Kousuke Saruta commented on SPARK-32119: [~lucacanali] Thanks. I got this is a known limitation.

[jira] [Commented] (SPARK-32013) Support query execution before/after reading/writing over JDBC

2020-06-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17148239#comment-17148239 ] Apache Spark commented on SPARK-32013: -- User 'moomindani' has created a pull request for this

[jira] [Assigned] (SPARK-32013) Support query execution before/after reading/writing over JDBC

2020-06-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32013: Assignee: Apache Spark > Support query execution before/after reading/writing over JDBC

[jira] [Assigned] (SPARK-32013) Support query execution before/after reading/writing over JDBC

2020-06-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32013: Assignee: (was: Apache Spark) > Support query execution before/after reading/writing

[jira] [Updated] (SPARK-32026) Add PrometheusServletSuite

2020-06-29 Thread Eren Avsarogullari (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eren Avsarogullari updated SPARK-32026: --- Summary: Add PrometheusServletSuite (was: Add PrometheusServlet Unit Test

[jira] [Commented] (SPARK-32056) Repartition by key should support partition coalesce for AQE

2020-06-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17148219#comment-17148219 ] Apache Spark commented on SPARK-32056: -- User 'viirya' has created a pull request for this issue:

[jira] [Commented] (SPARK-32056) Repartition by key should support partition coalesce for AQE

2020-06-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17148218#comment-17148218 ] Apache Spark commented on SPARK-32056: -- User 'viirya' has created a pull request for this issue:

[jira] [Updated] (SPARK-32133) Forbid time field steps for date start/end in Sequence

2020-06-29 Thread JinxinTang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JinxinTang updated SPARK-32133: --- Description: *Sequence time field steps for date start/end looks strange in spark as follows:*

[jira] [Commented] (SPARK-32026) Add PrometheusServlet Unit Test coverage

2020-06-29 Thread Eren Avsarogullari (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17148211#comment-17148211 ] Eren Avsarogullari commented on SPARK-32026: Currently, _PrometheusServlet_ follows Spark

[jira] [Commented] (SPARK-32133) Forbid time field steps for date start/end in Sequence

2020-06-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17148202#comment-17148202 ] Apache Spark commented on SPARK-32133: -- User 'TJX2014' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32133) Forbid time field steps for date start/end in Sequence

2020-06-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32133: Assignee: Apache Spark > Forbid time field steps for date start/end in Sequence >

[jira] [Commented] (SPARK-32133) Forbid time field steps for date start/end in Sequence

2020-06-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17148201#comment-17148201 ] Apache Spark commented on SPARK-32133: -- User 'TJX2014' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32133) Forbid time field steps for date start/end in Sequence

2020-06-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32133: Assignee: (was: Apache Spark) > Forbid time field steps for date start/end in

[jira] [Updated] (SPARK-32026) Add PrometheusServlet Unit Test coverage

2020-06-29 Thread Eren Avsarogullari (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eren Avsarogullari updated SPARK-32026: --- Summary: Add PrometheusServlet Unit Test coverage (was: Support consistency on

[jira] [Updated] (SPARK-32133) Forbid time field steps for date start/end in Sequence

2020-06-29 Thread JinxinTang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] JinxinTang updated SPARK-32133: --- Summary: Forbid time field steps for date start/end in Sequence (was: Forbid time field steps for

[jira] [Created] (SPARK-32134) YARN: archives rename with # doesn't work for https

2020-06-29 Thread Manos Tsagkias (Jira)
Manos Tsagkias created SPARK-32134: -- Summary: YARN: archives rename with # doesn't work for https Key: SPARK-32134 URL: https://issues.apache.org/jira/browse/SPARK-32134 Project: Spark

[jira] [Created] (SPARK-32133) Forbid time field steps for date start/end

2020-06-29 Thread JinxinTang (Jira)
JinxinTang created SPARK-32133: -- Summary: Forbid time field steps for date start/end Key: SPARK-32133 URL: https://issues.apache.org/jira/browse/SPARK-32133 Project: Spark Issue Type:

[jira] [Commented] (SPARK-32122) Exception while writing dataframe with enum fields

2020-06-29 Thread Sai kiran Krishna murthy (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17148128#comment-17148128 ] Sai kiran Krishna murthy commented on SPARK-32122: -- Looks like the issue is resolved in

[jira] [Comment Edited] (SPARK-32119) ExecutorPlugin doesn't work with Standalone Cluster

2020-06-29 Thread Luca Canali (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17147750#comment-17147750 ] Luca Canali edited comment on SPARK-32119 at 6/29/20, 8:09 PM: --- I am

[jira] [Commented] (SPARK-32132) Thriftserver interval returns "4 weeks 2 days" in 2.4 and "30 days" in 3.0

2020-06-29 Thread Juliusz Sompolski (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17148086#comment-17148086 ] Juliusz Sompolski commented on SPARK-32132: --- cc [~hyukjin.kwon] [~cloud_fan] [~Qin Yao] >

[jira] [Created] (SPARK-32132) Thriftserver interval returns "4 weeks 2 days" in 2.4 and "30 days" in 3.0

2020-06-29 Thread Juliusz Sompolski (Jira)
Juliusz Sompolski created SPARK-32132: - Summary: Thriftserver interval returns "4 weeks 2 days" in 2.4 and "30 days" in 3.0 Key: SPARK-32132 URL: https://issues.apache.org/jira/browse/SPARK-32132

[jira] [Updated] (SPARK-32131) union and set operations have wrong exception infomation

2020-06-29 Thread philipse (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] philipse updated SPARK-32131: - Description: Union and set operations can only be performed on tables with the compatible column

[jira] [Commented] (SPARK-32131) union and set operations have wrong exception infomation

2020-06-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17147960#comment-17147960 ] Apache Spark commented on SPARK-32131: -- User 'GuoPhilipse' has created a pull request for this

[jira] [Assigned] (SPARK-32131) union and set operations have wrong exception infomation

2020-06-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32131: Assignee: (was: Apache Spark) > union and set operations have wrong exception

[jira] [Assigned] (SPARK-32131) union and set operations have wrong exception infomation

2020-06-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32131: Assignee: Apache Spark > union and set operations have wrong exception infomation >

[jira] [Commented] (SPARK-32131) union and set operations have wrong exception infomation

2020-06-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17147959#comment-17147959 ] Apache Spark commented on SPARK-32131: -- User 'GuoPhilipse' has created a pull request for this

[jira] [Created] (SPARK-32131) union and set operations have wrong exception infomation

2020-06-29 Thread philipse (Jira)
philipse created SPARK-32131: Summary: union and set operations have wrong exception infomation Key: SPARK-32131 URL: https://issues.apache.org/jira/browse/SPARK-32131 Project: Spark Issue Type:

[jira] [Commented] (SPARK-32094) Patch cloudpickle.py with typing module side-effect fix

2020-06-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17147941#comment-17147941 ] Apache Spark commented on SPARK-32094: -- User 'codesue' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32094) Patch cloudpickle.py with typing module side-effect fix

2020-06-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32094: Assignee: Apache Spark > Patch cloudpickle.py with typing module side-effect fix >

[jira] [Commented] (SPARK-32094) Patch cloudpickle.py with typing module side-effect fix

2020-06-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17147940#comment-17147940 ] Apache Spark commented on SPARK-32094: -- User 'codesue' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32094) Patch cloudpickle.py with typing module side-effect fix

2020-06-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32094: Assignee: (was: Apache Spark) > Patch cloudpickle.py with typing module side-effect

[jira] [Updated] (SPARK-32130) Spark 3.0 json load performance is unacceptable in comparison of Spark 2.4

2020-06-29 Thread Sanjeev Mishra (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sanjeev Mishra updated SPARK-32130: --- Description: We are planning to move to Spark 3 but the read performance of our json files

[jira] [Commented] (SPARK-32028) App id link in history summary page point to wrong application attempt

2020-06-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17147920#comment-17147920 ] Apache Spark commented on SPARK-32028: -- User 'srowen' has created a pull request for this issue:

[jira] [Updated] (SPARK-32130) Spark 3.0 json load performance is unacceptable in comparison of Spark 2.4

2020-06-29 Thread Sanjeev Mishra (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sanjeev Mishra updated SPARK-32130: --- Attachment: small-anon.tar > Spark 3.0 json load performance is unacceptable in comparison

[jira] [Created] (SPARK-32130) Spark 3.0 json load performance is unacceptable in comparison of Spark 2.4

2020-06-29 Thread Sanjeev Mishra (Jira)
Sanjeev Mishra created SPARK-32130: -- Summary: Spark 3.0 json load performance is unacceptable in comparison of Spark 2.4 Key: SPARK-32130 URL: https://issues.apache.org/jira/browse/SPARK-32130

[jira] [Updated] (SPARK-31935) Hadoop file system config should be effective in data source options

2020-06-29 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-31935: -- Affects Version/s: (was: 2.4.7) > Hadoop file system config should be effective in data

[jira] [Commented] (SPARK-31935) Hadoop file system config should be effective in data source options

2020-06-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17147873#comment-17147873 ] Apache Spark commented on SPARK-31935: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Commented] (SPARK-31935) Hadoop file system config should be effective in data source options

2020-06-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17147871#comment-17147871 ] Apache Spark commented on SPARK-31935: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Commented] (SPARK-32129) Support AQE skew join with Union

2020-06-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17147828#comment-17147828 ] Apache Spark commented on SPARK-32129: -- User 'LantaoJin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32129) Support AQE skew join with Union

2020-06-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32129: Assignee: (was: Apache Spark) > Support AQE skew join with Union >

[jira] [Assigned] (SPARK-32129) Support AQE skew join with Union

2020-06-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32129: Assignee: Apache Spark > Support AQE skew join with Union >

[jira] [Resolved] (SPARK-24615) SPIP: Accelerator-aware task scheduling for Spark

2020-06-29 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-24615. --- Fix Version/s: 3.0.0 Resolution: Fixed > SPIP: Accelerator-aware task scheduling for

[jira] [Updated] (SPARK-32129) Support AQE skew join with Union

2020-06-29 Thread Lantao Jin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lantao Jin updated SPARK-32129: --- Description: Current, the AQE skew join only supports two tables join such as {code} SMJ :-Sort :

[jira] [Updated] (SPARK-32129) Support AQE skew join with Union

2020-06-29 Thread Lantao Jin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lantao Jin updated SPARK-32129: --- Description: Current, the AQE skew join only supports two tables join such as {code} SMJ :-Sort :

[jira] [Created] (SPARK-32129) Support AQE skew join with Union

2020-06-29 Thread Lantao Jin (Jira)
Lantao Jin created SPARK-32129: -- Summary: Support AQE skew join with Union Key: SPARK-32129 URL: https://issues.apache.org/jira/browse/SPARK-32129 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-32127) Check rules for MERGE INTO should use MergeAction.condition other than MergeAction.children

2020-06-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17147766#comment-17147766 ] Apache Spark commented on SPARK-32127: -- User 'xianyinxin' has created a pull request for this

[jira] [Commented] (SPARK-32127) Check rules for MERGE INTO should use MergeAction.condition other than MergeAction.children

2020-06-29 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17147765#comment-17147765 ] Wenchen Fan commented on SPARK-32127: - This is also fixed in 3.1 by

[jira] [Resolved] (SPARK-32127) Check rules for MERGE INTO should use MergeAction.condition other than MergeAction.children

2020-06-29 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-32127. - Fix Version/s: 3.1.0 3.0.1 Resolution: Fixed Issue resolved by pull

[jira] [Assigned] (SPARK-32127) Check rules for MERGE INTO should use MergeAction.condition other than MergeAction.children

2020-06-29 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-32127: --- Assignee: Xianyin Xin > Check rules for MERGE INTO should use MergeAction.condition other

[jira] [Resolved] (SPARK-32030) Support unlimited MATCHED and NOT MATCHED clauses in MERGE INTO

2020-06-29 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-32030. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 28875

[jira] [Assigned] (SPARK-32030) Support unlimited MATCHED and NOT MATCHED clauses in MERGE INTO

2020-06-29 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-32030: --- Assignee: Xianyin Xin > Support unlimited MATCHED and NOT MATCHED clauses in MERGE INTO >

[jira] [Commented] (SPARK-32119) ExecutorPlugin doesn't work with Standalone Cluster

2020-06-29 Thread Luca Canali (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17147750#comment-17147750 ] Luca Canali commented on SPARK-32119: - I am adding the comment that the limitation discussed in this

[jira] [Commented] (SPARK-32027) EventLoggingListener threw java.util.ConcurrentModificationException

2020-06-29 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17147746#comment-17147746 ] Jungtaek Lim commented on SPARK-32027: -- There're two different problems: # An event object is

[jira] [Assigned] (SPARK-32123) [Python] Setting `spark.sql.session.timeZone` only partially respected

2020-06-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32123: Assignee: Apache Spark > [Python] Setting `spark.sql.session.timeZone` only partially

[jira] [Commented] (SPARK-32123) [Python] Setting `spark.sql.session.timeZone` only partially respected

2020-06-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17147727#comment-17147727 ] Apache Spark commented on SPARK-32123: -- User 'TJX2014' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32123) [Python] Setting `spark.sql.session.timeZone` only partially respected

2020-06-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32123: Assignee: (was: Apache Spark) > [Python] Setting `spark.sql.session.timeZone` only

[jira] [Commented] (SPARK-32027) EventLoggingListener threw java.util.ConcurrentModificationException

2020-06-29 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17147705#comment-17147705 ] Jungtaek Lim commented on SPARK-32027: -- [~yumwang] Do you know the specific commit of the build

[jira] [Resolved] (SPARK-32056) Repartition by key should support partition coalesce for AQE

2020-06-29 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-32056. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 28900

[jira] [Resolved] (SPARK-32055) Unify getReader and getReaderForRange in ShuffleManager

2020-06-29 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-32055. - Fix Version/s: 3.1.0 Resolution: Fixed Issue resolved by pull request 28895

[jira] [Assigned] (SPARK-32055) Unify getReader and getReaderForRange in ShuffleManager

2020-06-29 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-32055: --- Assignee: wuyi > Unify getReader and getReaderForRange in ShuffleManager >

[jira] [Assigned] (SPARK-32056) Repartition by key should support partition coalesce for AQE

2020-06-29 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-32056: --- Assignee: L. C. Hsieh > Repartition by key should support partition coalesce for AQE >

[jira] [Comment Edited] (SPARK-31635) Spark SQL Sort fails when sorting big data points

2020-06-29 Thread Chen Zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17147653#comment-17147653 ] Chen Zhang edited comment on SPARK-31635 at 6/29/20, 9:51 AM: -- In fact, the

[jira] [Commented] (SPARK-31635) Spark SQL Sort fails when sorting big data points

2020-06-29 Thread Chen Zhang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17147653#comment-17147653 ] Chen Zhang commented on SPARK-31635: In fact, the RDD API corresponding to _DF.sort().take()_ is

[jira] [Commented] (SPARK-32128) Add a parameter to enable skip verify the input and output paths for InsertIntoHadoopFsRelationCommand

2020-06-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17147589#comment-17147589 ] Apache Spark commented on SPARK-32128: -- User 'wankunde' has created a pull request for this issue:

[jira] [Assigned] (SPARK-32128) Add a parameter to enable skip verify the input and output paths for InsertIntoHadoopFsRelationCommand

2020-06-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32128: Assignee: (was: Apache Spark) > Add a parameter to enable skip verify the input and

[jira] [Assigned] (SPARK-32128) Add a parameter to enable skip verify the input and output paths for InsertIntoHadoopFsRelationCommand

2020-06-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32128: Assignee: Apache Spark > Add a parameter to enable skip verify the input and output

[jira] [Commented] (SPARK-32128) Add a parameter to enable skip verify the input and output paths for InsertIntoHadoopFsRelationCommand

2020-06-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17147588#comment-17147588 ] Apache Spark commented on SPARK-32128: -- User 'wankunde' has created a pull request for this issue:

[jira] [Created] (SPARK-32128) Add a parameter to enable skip verify the input and output paths for InsertIntoHadoopFsRelationCommand

2020-06-29 Thread Wan Kun (Jira)
Wan Kun created SPARK-32128: --- Summary: Add a parameter to enable skip verify the input and output paths for InsertIntoHadoopFsRelationCommand Key: SPARK-32128 URL: https://issues.apache.org/jira/browse/SPARK-32128

[jira] [Commented] (SPARK-32127) Check rules for MERGE INTO should use MergeAction.condition other than MergeAction.children

2020-06-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17147575#comment-17147575 ] Apache Spark commented on SPARK-32127: -- User 'xianyinxin' has created a pull request for this

[jira] [Commented] (SPARK-32127) Check rules for MERGE INTO should use MergeAction.condition other than MergeAction.children

2020-06-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17147574#comment-17147574 ] Apache Spark commented on SPARK-32127: -- User 'xianyinxin' has created a pull request for this

[jira] [Assigned] (SPARK-32127) Check rules for MERGE INTO should use MergeAction.condition other than MergeAction.children

2020-06-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32127: Assignee: Apache Spark > Check rules for MERGE INTO should use MergeAction.condition

[jira] [Assigned] (SPARK-32127) Check rules for MERGE INTO should use MergeAction.condition other than MergeAction.children

2020-06-29 Thread Apache Spark (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-32127: Assignee: (was: Apache Spark) > Check rules for MERGE INTO should use

[jira] [Updated] (SPARK-32127) Check rules for MERGE INTO should use MergeAction.condition other than MergeAction.children

2020-06-29 Thread Xianyin Xin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianyin Xin updated SPARK-32127: Summary: Check rules for MERGE INTO should use MergeAction.condition other than

[jira] [Updated] (SPARK-32127) Check rules for MERGE INTO should use MergeAction.condition other than MeregAction.children

2020-06-29 Thread Xianyin Xin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-32127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xianyin Xin updated SPARK-32127: Description: [SPARK-30924|https://issues.apache.org/jira/browse/SPARK-30924] adds some check

[jira] [Created] (SPARK-32127) Check rules for MERGE INTO should use MergeAction.condition other than MeregAction.children

2020-06-29 Thread Xianyin Xin (Jira)
Xianyin Xin created SPARK-32127: --- Summary: Check rules for MERGE INTO should use MergeAction.condition other than MeregAction.children Key: SPARK-32127 URL: https://issues.apache.org/jira/browse/SPARK-32127