[ https://issues.apache.org/jira/browse/MAPREDUCE-205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Harsh J resolved MAPREDUCE-205. ------------------------------- Resolution: Not A Problem This never seemed to generate any demand and the discussion has also grown stale. Should be doable in user-land in MR2 though, so closing out for now. > Add ability to send "signals" to jobs and tasks > ----------------------------------------------- > > Key: MAPREDUCE-205 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-205 > Project: Hadoop Map/Reduce > Issue Type: New Feature > Reporter: Andrzej Bialecki > Assignee: Owen O'Malley > > In some cases it would be useful to be able to "signal" a job and its tasks > about some external condition, or to broadcast a specific message to all > tasks in a job. Currently we can only send a single pseudo-signal, that is > to kill a job. > Example 1: some jobs may be gracefully terminated even if they didn't > complete all their work, e.g. Fetcher in Nutch may be running for a very long > time if it blocks on relatively few sites left over from the fetchlist. In > such case it would be very useful to send it a message requesting that it > discards the rest of its input and gracefully completes its map tasks. > Example 2: available bandwidth for fetching may be different at different > times of day, e.g. daytime vs. nighttime, or total external link usage by > other applications. Fetcher jobs often run for several hours. It would be > good to be able to send a "signal" to the Fetcher to throttle or un-throttle > its bandwidth usage depending on external conditions. > Job implementations could react to these messages either by implementing a > method, or by registering a listener, whichever seems more natural. > I'm not quite sure how to go about implementing it, I guess this would have > to be a part of TaskUmbilicalProtocol but my knowledge here is a bit fuzzy > ... ;) Comments are welcome. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira