On Tue, Aug 15, 2017 at 2:20 PM, Nan Zhu <zhunanmcg...@gmail.com> wrote: > The key design consideration here is that how you model the state of > applications, if in actor, then there will be no synchronization involved > and yielding a cleaner design; if in a shared data structure, you will have > to be careful about coordinating threads here (we actually have a design > based on shared data structure and we eventually discard to pursue a > cleaner one).
I'm not really sure what you're talking about here, since I did not suggest a "shared data structure", and I'm not really sure what that means in this context. > I think bulk API can make life easier comparing to the shared data > structure, but it raises up two questions > > 1. Are we going to update all applications in the uniform pace, even they > are submitted in different time? Yes. While there are applications that need monitoring, you poll YARN at a constant frequency. Basically what would be done by multiple threads, but there's a single one. > 2. Are we going to use a single thread for everything, including send/recv > req/res and parse, etc. Why not. The expensive part is not parsing results, I'll bet, but having a whole bunch of different tasks opening and closing YARN connections. -- Marcelo
