[ http://issues.apache.org/jira/browse/HADOOP-451?page=comments#action_12437900 ] Doug Cutting commented on HADOOP-451: -------------------------------------
I like option 3 best -- add a RecordReader.getProgress() method, replacing its getPos() method with something abstract and file-independent. I also wonder whether we then really need Split.getCost(). We could instead promise to process splits in the order they're generated, so jobs could still control the order if they liked. > Add a Split interface > --------------------- > > Key: HADOOP-451 > URL: http://issues.apache.org/jira/browse/HADOOP-451 > Project: Hadoop > Issue Type: Improvement > Components: mapred > Reporter: Doug Cutting > > The InputFormat interface has a method: > FileSplit[] getSplits(); > This should change to: > Split[] getSplits(); > The Split interface would look like: > public interface Split extends Writable { > /** Returns a list of hosts that contain this split. > This is only used to optimize task placement, so this may be empty. */ > String[] getLocations(FileSystem fs); > /** The relative, estimated cost of operating on this. Typically the size > of the data in the split. > Used to prioritize tasks in a job (high-cost tasks are run first). */ > long getCost(); > } -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
