[
https://issues.apache.org/jira/browse/YARN-9554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16840317#comment-16840317
]
Prabhu Joseph edited comment on YARN-9554 at 5/15/19 11:57 AM:
---------------------------------------------------------------
{{TimelineEntity}} DAO class has field with {{Set}} interface which JAXB can't
handle and so {{ContextFactory}} throws {{JAXBException}} shown in description
while creating {{JAXBContextImpl}}. This will be ignored by jersey.
{code:java}
INFO: Couldn't find grammar element for class
org.apache.hadoop.yarn.api.records.timeline.TimelineEntity
{code}
All timeline put entities request will invoke createContext with TimelineEntity
throws JAXBException and jaxbContext is always null. This again causes slowness
due to synchronization while calling createContext which YARN-7266 tried to fix.
{code:java}
Fix of YARN-7266
synchronized (ContextFactory.class) {
if (jaxbContext == null) {
jaxbContext = (JAXBContext) m.invoke((Object) null, classes,
properties);
}
}
return jaxbContext;
{code}
*The patch includes below fixes:*
1. If {{createContext}} is for {{TimelineEntity}} and {{TimelineEntities}},
throw {{JAXBException}} (suppressed stacktrace) immediately.
2. Reuse single {{JAXBContextImpl}} for other DAO classes from
{{AHSWebServices}} and {{TimelineWebServices}}.
3. If {{createContext}} is for any other classes like
{{com.sun.research.ws.wadl.Application}}, let create new context as above
context does not know about this class.
*Testing Covered:*
1. Junit test classes from hadoop-yarn-server-applicationhistoryservice runs
fine
2. Functional Testing
{code:java}
1. AHSWebServices and TimelineWebServices REST API both from browser and curl
command - XML and JSON format.
http://<AHS>:8188/ws/v1/applicationhistory/about
http://<AHS>:8188/ws/v1/applicationhistory/apps/application_1557825335381_0001/appattempts/appattempt_1557825335381_0001_000001/containers/container_1557825335381_0001_01_000001
http://<AHS>:8188/ws/v1/applicationhistory/apps/application_1557825335381_0001/appattempts/appattempt_1557825335381_0001_000001/containers/
http://<AHS>:8188/ws/v1/applicationhistory/apps/application_1557825335381_0001/appattempts/appattempt_1557825335381_0001_000001/
http://<AHS>:8188/ws/v1/timeline/about/ws/v1/applicationhistory/apps/application_1557825335381_0001/appattempts/
http://<AHS>:8188/ws/v1/applicationhistory/apps/application_1557825335381_0001/
http://<AHS>:8188/ws/v1/applicationhistory/apps/
http://<AHS>:8188/ws/v1/timeline/about:8188/ws/v1/applicationhistory
http://<AHS>:8188/ws/v1/timeline
http://<AHS>:8188/ws/v1/timeline/about
http://<AHS>:8188/ws/v1/timeline/YARN_APPLICATION
http://<AHS>:8188/ws/v1/timeline/YARN_APPLICATION/application_1557825335381_0001
http://<AHS>:8188/ws/v1/timeline/YARN_APPLICATION/events
http://<AHS>:8188/ws/v1/timeline/HIVE_QUERY_ID
http://<AHS>:8188/ws/v1/timeline/TEZ_DAG_ID
Insert Domain using PUT:
curl -H "Accept: application/json" -H "Content-Type: application/json" -X PUT
http://<AHS>:8188/ws/v1/timeline/domain -d
'{"id":"abd","description":"test1","owner":"ambari-qa","readers":"ambari-qa","writers":"ambari-qa","createdtime":"123456","modifiedtime":"123456"}'
{"errors":[]}
Get Domain:
http://<AHS>:8188/ws/v1/timeline/domain
http://<AHS>:8188/ws/v1/timeline/domain/abc
{"domains":[{"id":"abc","description":"test","owner":"dr.who","readers":"ambari-qa","writers":"ambari-qa","createdtime":1557835184393,"modifiedtime":1557835209581}]}
Wrong URL:
http://<AHS>:8188/ws/v1/applicationhistory/apps/application_1557825335381_0001/appattempts/containers/
Wrong Accept Type:
curl -H "Accept: application/xml"
http://<AHS>:8188/ws/v1/timeline/YARN_APPLICATION
2. MapReduce Service Check
3. Tez Service Check
4. Hive Queries
5. Tez View
6. ApplicationHistory Web App
http://<AHS>:8188/applicationhistory/
7. PUT Entities:
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.yarn.client.api.TimelineClient;
import org.apache.hadoop.yarn.api.records.timeline.TimelineEntity;
import org.apache.hadoop.yarn.api.records.timeline.TimelinePutResponse;
public class Putter {
public static void main(String[] arg) {
TimelineClient client = TimelineClient.createTimelineClient();
client.init(new Configuration());
client.start();
TimelineEntity entity = new TimelineEntity();
entity.setEntityId(arg[0]);
entity.setEntityType("dummy");
entity.setStartTime(System.currentTimeMillis());
try {
TimelinePutResponse response = client.putEntities(entity);
System.out.println("RESPONSE="+response.toString());
} catch (Exception e) {
e.printStackTrace();
}
client.stop();
}
}
8. GET Entities:
http://<AHS>:8188/ws/v1/timeline/dummy
{code}
was (Author: prabhu joseph):
{{TimelineEntity}} DAO class has field with {{Set}} interface which JAXB can't
handle and so {{ContextFactory}} throws {{JAXBException}} shown in description
while creating {{JAXBContextImpl}}. This will be ignored by jersey.
{code:java}
INFO: Couldn't find grammar element for class
org.apache.hadoop.yarn.api.records.timeline.TimelineEntity
{code}
All timeline put entities request will invoke createContext with TimelineEntity
throws JAXBException and jaxbContext is always null. This again causes slowness
due to synchronization while calling createContext which YARN-7266 tried to fix.
{code:java}
Fix of YARN-7266
synchronized (ContextFactory.class) {
if (jaxbContext == null) {
jaxbContext = (JAXBContext) m.invoke((Object) null, classes,
properties);
}
}
return jaxbContext;
{code}
The patch includes below fixes:
1. If {{createContext}} is for {{TimelineEntity}} and {{TimelineEntities}},
throw {{JAXBException}} (suppressed stacktrace) immediately.
2. Reuse single {{JAXBContextImpl}} for other DAO classes from
{{AHSWebServices}} and {{TimelineWebServices}}.
3. If {{createContext}} is for any other classes like
{{com.sun.research.ws.wadl.Application}}, let create new context as above
context does not know about this class.
Below are the testing done:
1. Junit test classes from hadoop-yarn-server-applicationhistoryservice runs
fine
2. Functional Testing
{code:java}
1. AHSWebServices and TimelineWebServices REST API both from browser and curl
command - XML and JSON format.
http://<AHS>:8188/ws/v1/applicationhistory/about
http://<AHS>:8188/ws/v1/applicationhistory/apps/application_1557825335381_0001/appattempts/appattempt_1557825335381_0001_000001/containers/container_1557825335381_0001_01_000001
http://<AHS>:8188/ws/v1/applicationhistory/apps/application_1557825335381_0001/appattempts/appattempt_1557825335381_0001_000001/containers/
http://<AHS>:8188/ws/v1/applicationhistory/apps/application_1557825335381_0001/appattempts/appattempt_1557825335381_0001_000001/
http://<AHS>:8188/ws/v1/timeline/about/ws/v1/applicationhistory/apps/application_1557825335381_0001/appattempts/
http://<AHS>:8188/ws/v1/applicationhistory/apps/application_1557825335381_0001/
http://<AHS>:8188/ws/v1/applicationhistory/apps/
http://<AHS>:8188/ws/v1/timeline/about:8188/ws/v1/applicationhistory
http://<AHS>:8188/ws/v1/timeline
http://<AHS>:8188/ws/v1/timeline/about
http://<AHS>:8188/ws/v1/timeline/YARN_APPLICATION
http://<AHS>:8188/ws/v1/timeline/YARN_APPLICATION/application_1557825335381_0001
http://<AHS>:8188/ws/v1/timeline/YARN_APPLICATION/events
http://<AHS>:8188/ws/v1/timeline/HIVE_QUERY_ID
http://<AHS>:8188/ws/v1/timeline/TEZ_DAG_ID
Insert Domain using PUT:
curl -H "Accept: application/json" -H "Content-Type: application/json" -X PUT
http://<AHS>:8188/ws/v1/timeline/domain -d
'{"id":"abd","description":"test1","owner":"ambari-qa","readers":"ambari-qa","writers":"ambari-qa","createdtime":"123456","modifiedtime":"123456"}'
{"errors":[]}
Get Domain:
http://<AHS>:8188/ws/v1/timeline/domain
http://<AHS>:8188/ws/v1/timeline/domain/abc
{"domains":[{"id":"abc","description":"test","owner":"dr.who","readers":"ambari-qa","writers":"ambari-qa","createdtime":1557835184393,"modifiedtime":1557835209581}]}
Wrong URL:
http://<AHS>:8188/ws/v1/applicationhistory/apps/application_1557825335381_0001/appattempts/containers/
Wrong Accept Type:
curl -H "Accept: application/xml"
http://<AHS>:8188/ws/v1/timeline/YARN_APPLICATION
2. MapReduce Service Check
3. Tez Service Check
4. Hive Queries
5. Tez View
6. ApplicationHistory Web App
http://<AHS>:8188/applicationhistory/
7. PUT Entities:
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.yarn.client.api.TimelineClient;
import org.apache.hadoop.yarn.api.records.timeline.TimelineEntity;
import org.apache.hadoop.yarn.api.records.timeline.TimelinePutResponse;
public class Putter {
public static void main(String[] arg) {
TimelineClient client = TimelineClient.createTimelineClient();
client.init(new Configuration());
client.start();
TimelineEntity entity = new TimelineEntity();
entity.setEntityId(arg[0]);
entity.setEntityType("dummy");
entity.setStartTime(System.currentTimeMillis());
try {
TimelinePutResponse response = client.putEntities(entity);
System.out.println("RESPONSE="+response.toString());
} catch (Exception e) {
e.printStackTrace();
}
client.stop();
}
}
8. GET Entities:
http://<AHS>:8188/ws/v1/timeline/dummy
{code}
> TimelineEntity DAO has java.util.Set interface which JAXB can't handle
> ----------------------------------------------------------------------
>
> Key: YARN-9554
> URL: https://issues.apache.org/jira/browse/YARN-9554
> Project: Hadoop YARN
> Issue Type: Bug
> Components: timelineservice
> Affects Versions: 3.3.0
> Reporter: Prabhu Joseph
> Assignee: Prabhu Joseph
> Priority: Major
> Attachments: YARN-9554-001.patch
>
>
> TimelineEntity DAO has java.util.Set interface which JAXB can't handle. This
> breaks the fix of YARN-7266.
> {code}
> Caused by: com.sun.xml.internal.bind.v2.runtime.IllegalAnnotationsException:
> 1 counts of IllegalAnnotationExceptions
> java.util.Set is an interface, and JAXB can't handle interfaces.
> this problem is related to the following location:
> at java.util.Set
> at public java.util.HashMap
> org.apache.hadoop.yarn.api.records.timeline.TimelineEntity.getPrimaryFiltersJAXB()
> at org.apache.hadoop.yarn.api.records.timeline.TimelineEntity
> at public java.util.List
> org.apache.hadoop.yarn.api.records.timeline.TimelineEntities.getEntities()
> at org.apache.hadoop.yarn.api.records.timeline.TimelineEntities
> at
> com.sun.xml.internal.bind.v2.runtime.IllegalAnnotationsException$Builder.check(IllegalAnnotationsException.java:91)
> at
> com.sun.xml.internal.bind.v2.runtime.JAXBContextImpl.getTypeInfoSet(JAXBContextImpl.java:445)
> at
> com.sun.xml.internal.bind.v2.runtime.JAXBContextImpl.<init>(JAXBContextImpl.java:277)
> at
> com.sun.xml.internal.bind.v2.runtime.JAXBContextImpl.<init>(JAXBContextImpl.java:124)
> at
> com.sun.xml.internal.bind.v2.runtime.JAXBContextImpl$JAXBContextBuilder.build(JAXBContextImpl.java:1123)
> at
> com.sun.xml.internal.bind.v2.ContextFactory.createContext(ContextFactory.java:147)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
