jeongyooneo commented on issue #2: [NEMO-7] Intra-TaskGroup pipelining URL: https://github.com/apache/incubator-nemo/pull/2#issuecomment-373625012 Thanks for the review! I've addressed most of them, except the following two: 1) For refactoring metric collection code into a separate class requested by @johnyangk, I think we'd better do that in another issue. Since there are more types of metric to be added, it'd be great if we clean them up after the types of metric we need and their usages become concrete. 2) For checking TaskGroup complete event by collecting finished Task ids, there are subtle task completion timing issues that made me keep this as is. - TaskGroup is complete when all of its Tasks are complete, i.e. each Task consumed all of its input. - As input is processed element-wise, speculating the exact point when _final_ task is finished is hard. There can be multiple _final_ tasks that process input independently, whose finishing point are known only to each task. Thus, we still need to collect those _final_ task ids to check TaskGroup completion.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
