ChiaHung, Yes, I'm thinking similar things.
On Tue, Aug 12, 2014 at 4:11 PM, Chia-Hung Lin <cli...@googlemail.com> wrote: > I am currently working on this part based on the superstep api, > similar to the Superstep.java in the trunk. > > The checkpointer[1] saves bundle message instead of single message. > Not very sure if this is what you are looking for? > > [1]. > https://github.com/chlin501/hama/blob/peer-comm-mech-changed/core/src/main/scala/org/apache/hama/monitor/Checkpointer.scala > > > > > On 12 August 2014 15:04, Edward J. Yoon <edwardy...@apache.org> wrote: >> I think that transferring single messages at a time is not a wise way. >> Bundle is used to avoid network overheads and contentions. So, if we >> use Bundle, each processor always sends/receives an bundles. >> >> BSPMessageBundle is Writable (and Iterable). And it manages the >> serialized message as a byte array. If we write an bundles when >> checkpointing or using Disk-queue, it'll be more simple and faster. >> >> In Spilling Queue case, it always requires the process of unbundling >> and putting messages into queue. >> >> >> On Tue, Aug 12, 2014 at 2:41 PM, Tommaso Teofili >> <tommaso.teof...@gmail.com> wrote: >>> -1, can't we first discuss? Also it'd be helpful to be more specific on the >>> problems. >>> Tommaso >>> >>> >>> >>> 2014-08-12 4:25 GMT+02:00 Edward J. Yoon <edwardy...@apache.org>: >>> >>>> All, >>>> >>>> I'll delete Spilling queue, and rewrite checkpoint/recovery >>>> implementation (checkpointing bundles is better than checkpointing all >>>> messages). Current implementation is quite mess :/ there are huge >>>> deserialization/serialization overheads.. >>>> >>>> -- >>>> Best Regards, Edward J. Yoon >>>> CEO at DataSayer Co., Ltd. >>>> >> >> >> >> -- >> Best Regards, Edward J. Yoon >> CEO at DataSayer Co., Ltd. -- Best Regards, Edward J. Yoon CEO at DataSayer Co., Ltd.
