I am +1. That plan is going to make things much more straight to other developers too.
Anastasis On 13 Μαϊ 2014, at 7:27 π.μ., Edward J. Yoon <[email protected]> wrote: > Just opened https://issues.apache.org/jira/browse/HAMA-903 > > On Tue, May 13, 2014 at 10:51 AM, Edward J. Yoon <[email protected]> > wrote: >> The AbstractMessageManager.loopBackMessages() method unbundle messages >> into localQueueForNextIteration. Finally, in clearOutgoingMessages() >> method, localQueue is switched by localQueueForNextIteration. >> >> My goal is simplify this procedure. This is huge overhead. >> >> Basically, BSPMessageBundle provides iterator access to the messages >> contained in the bundle. So, we can skip whole conversion from bundle >> to queue. >> >>> Is it going to use rpc? >> >> I won't touch RPC mechanism. Only queue implementations will be changed. >> >> On Mon, May 12, 2014 at 11:59 PM, Chia-Hung Lin <[email protected]> >> wrote: >>> Is it going to use rpc? >>> >>> Will it still use the interface, for instance, MessageManager.java? >>> >>> Just to check if any point for integrating with the current ongoing >>> refactoring process. >>> >>> If possible, perhaps decoupling io part and rpc from interface would >>> somehow simplify the integration progress. >>> >>> >>> On 12 May 2014 09:01, Edward J. Yoon <[email protected]> wrote: >>>> The old design of outgoing/incoming message queues is readable but it >>>> has some problems, and the most performance and memory issues are >>>> dependent upon this part. >>>> >>>> 1) To send a messages to destination Peer, we serialize, compress, and >>>> bundle the messages. So, using disk or spilling queue for the outgoing >>>> messages is pointless and cause of degradation. This issue SOLVED by >>>> HAMA-853. We'll need to add disk-based bundle in the future. >>>> >>>> 2) Receive-side queue is also the same. Instead of unbundling (and >>>> deserializing, decompressing) bundles into {memory, disk, or spilling} >>>> queue, we should use bundles in efficient and asynchronous way. >>>> >>>> If you agree with this, I'll start to refactor the whole queue system. >>>> >>>> If you have any other ideas e.g., asynchronous message >>>> synchronization, Pls let me know. >>>> >>>> Thanks. >>>> >>>> -- >>>> Best Regards, Edward J. Yoon >>>> CEO at DataSayer Co., Ltd. >> >> >> >> -- >> Best Regards, Edward J. Yoon >> CEO at DataSayer Co., Ltd. > > > > -- > Best Regards, Edward J. Yoon > CEO at DataSayer Co., Ltd.
