Piggybacking Mappers on the Reducer of the previous job step
------------------------------------------------------------
Key: ADDR-24
URL: http://issues.apache.org/jira/browse/ADDR-24
Project: Addressing
Type: New Feature
Reporter: paul sutter
Priority: Minor
It would be nice to be able to piggyback the mappers of step N+1 onto the
reducers of step N.
This would mean having the same number of mappers as reducers, only works for
multistep jobs, and obviously adds complexity to node failure recovery.
But, it would be a useful speedup since most of the overhead is in
serialization, buffer copies, disk io, and network io, rather than in the
mappers and reducers themselves.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
