Josh Rosen created SPARK-10704: ---------------------------------- Summary: Consolidate HashShuffleReader and ShuffleReader and refactor ShuffleManager.getReader() Key: SPARK-10704 URL: https://issues.apache.org/jira/browse/SPARK-10704 Project: Spark Issue Type: Sub-task Components: Shuffle Reporter: Josh Rosen Assignee: Josh Rosen
The current shuffle code has an interface named ShuffleReader with only one implementation, HashShuffleReader. This naming is confusing, since the same read path code is used for both sort- and hash-based shuffle. We should consolidate these classes. In addition, there are aspects of ShuffleManager.getReader()'s API which don't make a lot of sense: it exposes the ability to request a contiguous range of shuffle partitions, but this feature isn't supported by any ShuffleReader implementations and isn't used anywhere in the existing code. We should clean this up, too. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org