Hi, According to my understanding, it looks like a refactoring issue to reduce the complexity and the memory footprint of master server by removing back-calls of TaskInProgress and map structures. Right?
I think, that's a nice idea. But we didn't implement the task killing, re-attempting functions yet. So, I would propose to schedule it for 0.2.1 release. -Edward On Thu, Dec 16, 2010 at 12:55 PM, Lin Chia-Hung <[email protected]> wrote: > When working on BSPPeer fault tolerance > (https://issues.apache.org/jira/browse/HAMA-199), I came across > another issue - the task distribution currently employed in HAMA is > done by GroomServer requesting tasks from BSPMaster, similar to one > used in Hadoop mapreduce. This strategy has discussion at > https://issues.apache.org/jira/browse/MAPREDUCE-278 regarding to e.g > memory footprint, race condition, etc. Although this issue do not > directly relate to the BSPPeer fault tolerance, the strategy > (GroomServer requests tasks from BSPMaster) employed may have > potential issue e.g. task can not be reschedule to the node expected. > > So I would like to know if there is any chance the HAMA task > distribution mechanism may work toward this direction (proactive task > assignment)? > > Thanks > -- Best Regards, Edward J. Yoon [email protected] http://blog.udanax.org
