[jira] [Commented] (YARN-3240) [Data Mode] Implement client API to put generic entities
[ https://issues.apache.org/jira/browse/YARN-3240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15369750#comment-15369750 ] Hudson commented on YARN-3240: -- SUCCESS: Integrated in Hadoop-trunk-Commit #10074 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/10074/]) YARN-3240. Implement client API to put generic entities. Contributed by (sjlee: rev 4f0c7eaff3b66df6e87aa251e1316a85f38ffab5) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/client/api/TimelineClient.java * hadoop-project/pom.xml * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/timelineservice/TimelineEntities.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/main/java/org/apache/hadoop/yarn/server/timelineservice/aggregator/PerNodeAggregatorServer.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/main/java/org/apache/hadoop/yarn/server/timelineservice/aggregator/BaseAggregatorService.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/pom.xml * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/client/api/impl/TimelineClientImpl.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice/src/main/java/org/apache/hadoop/yarn/server/timelineservice/aggregator/PerNodeAggregatorWebService.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/pom.xml * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/api/records/timelineservice/TestTimelineServiceRecords.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/pom.xml * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests/src/test/java/org/apache/hadoop/yarn/server/timelineservice/TestTimelineServiceClientIntegration.java YARN-3240. Added ".keep" file which was missed before. (sjlee: rev 309592b9e5520acd7d44f834c0dfb8356b2d5897) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/timeline/.keep > [Data Mode] Implement client API to put generic entities > > > Key: YARN-3240 > URL: https://issues.apache.org/jira/browse/YARN-3240 > Project: Hadoop YARN > Issue Type: Sub-task > Components: timelineserver >Reporter: Zhijie Shen >Assignee: Zhijie Shen > Fix For: YARN-2928 > > Attachments: YARN-3240.1.patch, YARN-3240.2.patch, YARN-3240.3.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-3240) [Data Mode] Implement client API to put generic entities
[ https://issues.apache.org/jira/browse/YARN-3240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14356257#comment-14356257 ] Zhijie Shen commented on YARN-3240: --- bq. We should strive to preserve the state where v.2 can be disabled without affecting v.1 whatsoever, and vice versa Yeah, the newly added v2 apis doesn't affect the existing v1 apis. As mentioned before, we do it in the same class as most HTTP related code can be reused, which is actually the major body while the specific logic of putting the entity is a simple call. [Data Mode] Implement client API to put generic entities Key: YARN-3240 URL: https://issues.apache.org/jira/browse/YARN-3240 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Reporter: Zhijie Shen Assignee: Zhijie Shen Fix For: YARN-2928 Attachments: YARN-3240.1.patch, YARN-3240.2.patch, YARN-3240.3.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3240) [Data Mode] Implement client API to put generic entities
[ https://issues.apache.org/jira/browse/YARN-3240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14351164#comment-14351164 ] Sangjin Lee commented on YARN-3240: --- Sorry for chiming in late on this. Thanks [~zjshen] for working on this! I continue to worry about ATS v.1 and timeline service v.2 getting mixed up. It makes me nervous when I see the same class/interface contain code for both. We should strive to preserve the state where v.2 can be disabled without affecting v.1 whatsoever, and vice versa. We should at least review it very carefully in that regard, but it would be good to be able to see it more clearly. Thoughts? A small nit: PerNodeAggregatorServer and PerNodeAggregatorWebService contain wildcard imports. [Data Mode] Implement client API to put generic entities Key: YARN-3240 URL: https://issues.apache.org/jira/browse/YARN-3240 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Reporter: Zhijie Shen Assignee: Zhijie Shen Fix For: YARN-2928 Attachments: YARN-3240.1.patch, YARN-3240.2.patch, YARN-3240.3.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3240) [Data Mode] Implement client API to put generic entities
[ https://issues.apache.org/jira/browse/YARN-3240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14336811#comment-14336811 ] Zhijie Shen commented on YARN-3240: --- Thanks for committing the patch. PS: .keep seems to be missed, I added it in a followup commit. [Data Mode] Implement client API to put generic entities Key: YARN-3240 URL: https://issues.apache.org/jira/browse/YARN-3240 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Reporter: Zhijie Shen Assignee: Zhijie Shen Fix For: YARN-2928 Attachments: YARN-3240.1.patch, YARN-3240.2.patch, YARN-3240.3.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3240) [Data Mode] Implement client API to put generic entities
[
https://issues.apache.org/jira/browse/YARN-3240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14335106#comment-14335106
]
Junping Du commented on YARN-3240:
--
Thanks [~zjshen] for the patch! I am reviewing this patch now, and a couple of
comments so far:
{code}
+ //TODO: It needs to be updated by the discovery service
private URI resURI;
{code}
Looks like we are creating one TimelineClient for every application so we have
multiple TimelineClients within NM. Do we think about the other way - one
TimelineClient can talk to different app URLs (put url as a parameter in every
call, so client can be more stateless)? I don't have any preference here and I
think compatibility with old client could be a good reason here. But just
curious on our decisions here.
In addition, I think we need a new constructor to take resURI as a parameter
because this is not get from configuration now but get from caller of
TimelineClient who know the resource details (address of aggregator).
And a setter is also needed to resURI because when caller (AM or NMs) have any
failure in PUT/POST (as IOException so far), its retry logic will notify RM to
recovery (through heartbeat or allocate request, addressed in YARN-3039) and
set it back afterwards.
{code}
catch (RuntimeException re) {
+ // runtime exception is expected if the client cannot connect the server
+ String msg =
+ Failed to get the response from the timeline server.;
+ LOG.error(msg, re);
+ throw new IOException(re);
+}
+if (resp == null ||
+resp.getClientResponseStatus() != ClientResponse.Status.OK) {
+ String msg =
+ Failed to get the response from the timeline server.;
+ LOG.error(msg);
+ if (LOG.isDebugEnabled() resp != null) {
+String output = resp.getEntity(String.class);
+LOG.debug(HTTP error code: + resp.getStatus()
++ Server response : \n + output);
+ }
+ throw new YarnException(msg);
+}
{code}
Looks like we are differentiate 404 and 500 here with IOException and
YarnException which looks fine to me. Do we plan to have different handling
logic (in caller part) for two failure cases?
Other looks good to me.
[Data Mode] Implement client API to put generic entities
Key: YARN-3240
URL: https://issues.apache.org/jira/browse/YARN-3240
Project: Hadoop YARN
Issue Type: Sub-task
Components: timelineserver
Reporter: Zhijie Shen
Assignee: Zhijie Shen
Attachments: YARN-3240.1.patch, YARN-3240.2.patch
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3240) [Data Mode] Implement client API to put generic entities
[ https://issues.apache.org/jira/browse/YARN-3240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14335901#comment-14335901 ] Junping Du commented on YARN-3240: -- Will commit it tomorrow if no more comments from others. [Data Mode] Implement client API to put generic entities Key: YARN-3240 URL: https://issues.apache.org/jira/browse/YARN-3240 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Reporter: Zhijie Shen Assignee: Zhijie Shen Attachments: YARN-3240.1.patch, YARN-3240.2.patch, YARN-3240.3.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3240) [Data Mode] Implement client API to put generic entities
[ https://issues.apache.org/jira/browse/YARN-3240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14335892#comment-14335892 ] Junping Du commented on YARN-3240: -- +1. Patch looks good. [Data Mode] Implement client API to put generic entities Key: YARN-3240 URL: https://issues.apache.org/jira/browse/YARN-3240 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Reporter: Zhijie Shen Assignee: Zhijie Shen Attachments: YARN-3240.1.patch, YARN-3240.2.patch, YARN-3240.3.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3240) [Data Mode] Implement client API to put generic entities
[ https://issues.apache.org/jira/browse/YARN-3240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14335730#comment-14335730 ] Li Lu commented on YARN-3240: - Patch LGTM. Let's figure out a possible solution of YARN-3087 after this is in. [Data Mode] Implement client API to put generic entities Key: YARN-3240 URL: https://issues.apache.org/jira/browse/YARN-3240 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Reporter: Zhijie Shen Assignee: Zhijie Shen Attachments: YARN-3240.1.patch, YARN-3240.2.patch, YARN-3240.3.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
