Mingjia Liu created SPARK-32609:
---
Summary: Incorrect exchange reuse with DataSourceV2
Key: SPARK-32609
URL: https://issues.apache.org/jira/browse/SPARK-32609
Project: Spark
Issue Type: Bug
[
https://issues.apache.org/jira/browse/SPARK-32609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17177199#comment-17177199
]
Mingjia Liu commented on SPARK-32609:
-
Mitigation:
Turn off spark.sql.exchange.reu
[
https://issues.apache.org/jira/browse/SPARK-32609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mingjia Liu updated SPARK-32609:
Target Version/s: 2.4.5, 2.4.7 (was: 2.4.5)
> Incorrect exchange reuse with DataSourceV2
> --
[
https://issues.apache.org/jira/browse/SPARK-32609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17177231#comment-17177231
]
Mingjia Liu commented on SPARK-32609:
-
I am currently working on a fix & unit test
Mingjia Liu created SPARK-32708:
---
Summary: Query optimization fail to reuse exchange with
DataSourceV2
Key: SPARK-32708
URL: https://issues.apache.org/jira/browse/SPARK-32708
Project: Spark
Is
[
https://issues.apache.org/jira/browse/SPARK-32708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mingjia Liu updated SPARK-32708:
Summary: Query optimization fails to reuse exchange with DataSourceV2
(was: Query optimization fa
[
https://issues.apache.org/jira/browse/SPARK-32708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mingjia Liu updated SPARK-32708:
Description:
Repro query:
{code:java}
spark.conf.set("spark.sql.exchange.reuse","true")
spark.read
[
https://issues.apache.org/jira/browse/SPARK-32708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mingjia Liu updated SPARK-32708:
Description:
Repro query:
{code:java}
spark.conf.set("spark.sql.exchange.reuse","true")
spark.read
[
https://issues.apache.org/jira/browse/SPARK-32708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mingjia Liu updated SPARK-32708:
Description:
Repro query:
{code:java}
spark.conf.set("spark.sql.exchange.reuse","true")
spark.read
[
https://issues.apache.org/jira/browse/SPARK-32708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17185523#comment-17185523
]
Mingjia Liu edited comment on SPARK-32708 at 8/27/20, 12:01 AM:
--
[
https://issues.apache.org/jira/browse/SPARK-32708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17185523#comment-17185523
]
Mingjia Liu commented on SPARK-32708:
-
This difference happened at ReuseExchange rul
[
https://issues.apache.org/jira/browse/SPARK-32708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17185524#comment-17185524
]
Mingjia Liu commented on SPARK-32708:
-
This bug is brought to the surface by [link
[
https://issues.apache.org/jira/browse/SPARK-32708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17185523#comment-17185523
]
Mingjia Liu edited comment on SPARK-32708 at 8/27/20, 12:06 AM:
--
[
https://issues.apache.org/jira/browse/SPARK-32708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17185526#comment-17185526
]
Mingjia Liu commented on SPARK-32708:
-
I can't repro the issue at Spark 3.0
Fix:
[
https://issues.apache.org/jira/browse/SPARK-32708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mingjia Liu updated SPARK-32708:
Comment: was deleted
(was: I can't repro the issue at Spark 3.0
*Fix:* 2.4 needs to partially fo
[
https://issues.apache.org/jira/browse/SPARK-32708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17185526#comment-17185526
]
Mingjia Liu edited comment on SPARK-32708 at 8/27/20, 12:10 AM:
--
[
https://issues.apache.org/jira/browse/SPARK-32708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17185523#comment-17185523
]
Mingjia Liu edited comment on SPARK-32708 at 8/27/20, 12:10 AM:
--
[
https://issues.apache.org/jira/browse/SPARK-32708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17185523#comment-17185523
]
Mingjia Liu edited comment on SPARK-32708 at 8/27/20, 12:11 AM:
--
[
https://issues.apache.org/jira/browse/SPARK-32708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17185523#comment-17185523
]
Mingjia Liu edited comment on SPARK-32708 at 8/27/20, 12:11 AM:
--
[
https://issues.apache.org/jira/browse/SPARK-32708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17185523#comment-17185523
]
Mingjia Liu edited comment on SPARK-32708 at 8/27/20, 12:14 AM:
--
20 matches
Mail list logo
