[ https://issues.apache.org/jira/browse/YARN-3367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Naganarasimha G R updated YARN-3367: ------------------------------------ Attachment: YARN-3367.YARN-2928.001.patch Uploading an initial patch with no test case for this jira : Some open points which needs more discussion # Timelineclient async calls are only to ensure the client need not wait till the server response & just return immediately after requesting to post entity or even in server side we need to ensure some thing ? As currently we are trying to send the async parameter to the server. # According earlier discussion we had to decide whether to have 2 cross 2 matric wrt sync/async & writer flush & not flush in server side, but after YARN-4061 (Fault tolerant writer for timeline v2), i presume client need not ensure much as consistency will be handled in server side and IMO it would be sufficent to just have non blocking call for async # Is it important to maintain the order of events which are sent from sync and async ? i.e. Is it req to ensure all the async events are also pushed along with the current sync event or is it ok to send only the sync ? (current patch just ensures async events are in order) . # Whether its req to merge entities of multiple async calls as they belong to same application ? Please kindly review and share your thoughts on the above points. cc /[~sjlee0] Informing you, as you had asked to include you in discussion for these points and also you were not watching for this jira > Replace starting a separate thread for post entity with event loop in > TimelineClient > ------------------------------------------------------------------------------------ > > Key: YARN-3367 > URL: https://issues.apache.org/jira/browse/YARN-3367 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver > Affects Versions: YARN-2928 > Reporter: Junping Du > Assignee: Naganarasimha G R > Attachments: YARN-3367.YARN-2928.001.patch > > > Since YARN-3039, we add loop in TimelineClient to wait for > collectorServiceAddress ready before posting any entity. In consumer of > TimelineClient (like AM), we are starting a new thread for each call to get > rid of potential deadlock in main thread. This way has at least 3 major > defects: > 1. The consumer need some additional code to wrap a thread before calling > putEntities() in TimelineClient. > 2. It cost many thread resources which is unnecessary. > 3. The sequence of events could be out of order because each posting > operation thread get out of waiting loop randomly. > We should have something like event loop in TimelineClient side, > putEntities() only put related entities into a queue of entities and a > separated thread handle to deliver entities in queue to collector via REST > call. -- This message was sent by Atlassian JIRA (v6.3.4#6332)