[ https://issues.apache.org/jira/browse/YARN-3981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15451391#comment-15451391 ]
Rohith Sharma K S commented on YARN-3981: ----------------------------------------- bq. We can launch collectors as separate process for this use case? There are 2 ways. Firstly run as default service in NM? Pros is RM can make use of underlaying NM-RM communication protocol. RM need to just track of collector address. Secondly run as separate process? Pro's is is that admin can decide off-client-writer-cluster. Con's is RM need to track this daemon separately at high level bq. For storing those entities posted from clients, can we put them in the entity table, but just leave some unknown fields empty? Will that be a concern for the storage API's semantics? Yes, it is concern for storage API's semantics. Current schema structure for entity table is *userName!clusterId!flowName!flowRunId!AppId!entityType!entityId*. So, key would become *userName!clusterId!flowName!null!null!entityType!entityId* which I am not sure doe HBase support this as a key for read/write. > support timeline clients not associated with an application > ----------------------------------------------------------- > > Key: YARN-3981 > URL: https://issues.apache.org/jira/browse/YARN-3981 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver > Affects Versions: YARN-2928 > Reporter: Sangjin Lee > Assignee: Rohith Sharma K S > Labels: YARN-5355 > > In the current v.2 design, all timeline writes must belong in a > flow/application context (cluster + user + flow + flow run + application). > But there are use cases that require writing data outside the context of an > application. One such example is a higher level client (e.g. tez client or > hive/oozie/cascading client) writing flow-level data that spans multiple > applications. We need to find a way to support them. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org