[jira] [Comment Edited] (SPARK-28210) Shuffle Storage API: Reads
[ https://issues.apache.org/jira/browse/SPARK-28210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17196013#comment-17196013 ] Attila Zsolt Piros edited comment on SPARK-28210 at 9/15/20, 9:38 AM: -- [~tianczha] [~devaraj] I would like to work on this issue if that's fine for you. I intend to progress along the ideas of the linked PR: to pass the metadata when the reducer task is constructed. was (Author: attilapiros): [~tianczha] [~devaraj] I would like to work on this issue if that's fine for you. I would like to progress along the ideas of the linked PR: to pass the metadata when the reducer task is constructed. > Shuffle Storage API: Reads > -- > > Key: SPARK-28210 > URL: https://issues.apache.org/jira/browse/SPARK-28210 > Project: Spark > Issue Type: Sub-task > Components: Shuffle, Spark Core >Affects Versions: 3.1.0 >Reporter: Matt Cheah >Priority: Major > > As part of the effort to store shuffle data in arbitrary places, this issue > tracks implementing an API for reading the shuffle data stored by the write > API. Also ensure that the existing shuffle implementation is refactored to > use the API. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Comment Edited] (SPARK-28210) Shuffle Storage API: Reads
[ https://issues.apache.org/jira/browse/SPARK-28210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17166027#comment-17166027 ] Tianchen Zhang edited comment on SPARK-28210 at 7/27/20, 11:40 PM: --- Hi [~devaraj], do you mind sharing some ideas about your change? Is it based on the linked PR in this JIRA? We are also finding a way to have our own storage implementation but are blocked by the lack of reader's API. Thanks. was (Author: tianczha): Hi [~devaraj], do you mind share some ideas about your change? Is it based on the linked PR in this JIRA? We are also finding a way to have our own storage implementation but are blocked by the lack of reader's API. Thanks. > Shuffle Storage API: Reads > -- > > Key: SPARK-28210 > URL: https://issues.apache.org/jira/browse/SPARK-28210 > Project: Spark > Issue Type: Sub-task > Components: Shuffle, Spark Core >Affects Versions: 3.1.0 >Reporter: Matt Cheah >Priority: Major > > As part of the effort to store shuffle data in arbitrary places, this issue > tracks implementing an API for reading the shuffle data stored by the write > API. Also ensure that the existing shuffle implementation is refactored to > use the API. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
