[ https://issues.apache.org/jira/browse/SPARK-3292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Patrick Wendell resolved SPARK-3292. ------------------------------------ Resolution: Won't Fix Seems like this is a necessary feature of the current design and can be partially worked around by filtering in user space. > Shuffle Tasks run incessantly even though there's no inputs > ----------------------------------------------------------- > > Key: SPARK-3292 > URL: https://issues.apache.org/jira/browse/SPARK-3292 > Project: Spark > Issue Type: Improvement > Components: Streaming > Affects Versions: 1.0.2 > Reporter: guowei > > such as repartition groupby join and cogroup > for example. > if i want the shuffle outputs save as hadoop file ,even though there is no > inputs , many emtpy file generate too. > it's too expensive , -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org