[
https://issues.apache.org/jira/browse/GIRAPH-274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13427086#comment-13427086
]
Eli Reisman commented on GIRAPH-274:
------------------------------------
I think on our cluster there's a couple of options you set to ensure it takes.
I mentioned it as an emergency workaround, I can't use it here, but your
situation might be different.
The zombie thing is more an annoyance than a problem for us, but our ops people
don't like it when it happens, and users seem to forget often during testing
and debugging to kill their jobs. Then Giraph devs get blamed...
Making the thread smarter doesn't sound like it buys us anything, as it still
needs to read progress data on regular intervals, and do work to measure and
evaluate it...and then call progress just as often anyway. This seems like it
means replacing progress calls with more overhead and perhaps synchronization
just to make the code prettier? Giraph is already up against the wall
resource-wise, especially during INPUT_SUPERSTEP.
I think the beautiful solution you're looking for here involves a move from
Hadoop to YARN. For now, I just want my jobs to last through load-in again.
Anyway, there are calls in the old 246 patch that will make this happen. Sorry
if its ugly, I don't blame you for a few dry heaves. The excitement of seeing
your output data come out the other side will help the nausea pass. :)
> Jobs still failing due to tasks timeout during INPUT_SUPERSTEP
> --------------------------------------------------------------
>
> Key: GIRAPH-274
> URL: https://issues.apache.org/jira/browse/GIRAPH-274
> Project: Giraph
> Issue Type: Bug
> Affects Versions: 0.2.0
> Reporter: Jaeho Shin
> Assignee: Jaeho Shin
> Fix For: 0.2.0
>
> Attachments: GIRAPH-274.patch
>
>
> Even after GIRAPH-267, jobs were failing during INPUT_SUPERSTEP when some
> workers don't get to reserve an input split, while others were loading
> vertices for a long time. (related to GIRAPH-246 and GIRAPH-267)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
