Hello Hitesh Shah, This is Timeline log sniipet snippet: INFO: Initiating Jersey application, version 'Jersey: 1.9 09/02/2011 11:17 AM' Jul 23, 2015 10:27:38 AM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.webapp.GenericExceptionHandler to GuiceManagedComponentProvider with the scope "Singleton" Jul 23, 2015 10:27:38 AM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.webapp.YarnJacksonJaxbJsonProvider to GuiceManagedComponentProvider with the scope "Singleton" Jul 23, 2015 10:27:39 AM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.AHSWebServices to GuiceManagedComponentProvider with the scope "Singleton" Jul 23, 2015 10:27:39 AM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider INFO: Binding org.apache.hadoop.yarn.server.timeline.webapp.TimelineWebServices to GuiceManagedComponentProvider with the scope "Singleton" 15/07/23 10:30:49 INFO timeline.LeveldbTimelineStore: Discarded 0 entities for timestamp 1437013849959 and earlier in 0.001 seconds 15/07/23 10:35:49 INFO timeline.LeveldbTimelineStore: Discarded 0 entities for timestamp 1437014149960 and earlier in 0.001 seconds
[email protected] From: Hitesh Shah Date: 2015-07-22 23:12 To: user Subject: Re: tez use timeline server get exception Hello jjtang, Can you check the timeline daemon logs on yhd-jqhadoop200.int.yihaodian.com to see if there are any errors? From a functional view point, timeline errors are not fatal but this does mean that the Tez UI might not be able to show all progress info due to the failures to write to history. Also, specifically, I would recommend setting "tez.runtime.convert.user-payload.to.history-text” to false due to the additional load that it creates on Timeline. I know that the tez docs mention it setting it to true but I have a pending jira to update them to not mention this setting. There have been quite a few known reports on problems with Timeline scaling to handling a lot of load. There is some current work in progress to alleviate that. Please follow YARN-3942 and TEZ-2628 jira for more info. Additionally, I am not sure if you can patch TimelineServer but the patch from this jira can help improve performance: YARN-3448 thanks — Hitesh On Jul 21, 2015, at 11:15 PM, jjtang <[email protected]> wrote: > > 2015-07-22 14:00:48,439 ERROR [HistoryEventHandlingThread] > impl.TimelineClientImpl: Failed to get the response from the timeline server. > 2015-07-22 14:00:48,439 WARN [HistoryEventHandlingThread] > ats.ATSHistoryLoggingService: Could not handle history events > org.apache.hadoop.yarn.exceptions.YarnException: Failed to get the response > from the timeline server. > at > org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl.doPosting(TimelineClientImpl.java:339) > > at > org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl.putEntities(TimelineClientImpl.java:301) > > at > org.apache.tez.dag.history.logging.ats.ATSHistoryLoggingService.handleEvents(ATSHistoryLoggingService.java:342) > > at > org.apache.tez.dag.history.logging.ats.ATSHistoryLoggingService.access$700(ATSHistoryLoggingService.java:52) > > at > org.apache.tez.dag.history.logging.ats.ATSHistoryLoggingService$1.run(ATSHistoryLoggingService.java:186) > > at java.lang.Thread.run(Thread.java:745) > > > My enviroment is hadoop 2.6.0-cdh5.4.3 hive 1.1.0-cdh5.4.3 tez0.7. I type > http://yhd-jqhadoop200.int.yihaodian.com:8188/ws/v1/timeline/ in IE .IE > retrun {"About":"Timeline API"}. > My hiveserver2 hive-site.xml as shown in the following XML snippet: > > <property> > <name>hive.server2.authentication</name> > <value>CUSTOM</value> > </property> > > <property> > <name>hive.server2.custom.authentication.class</name> > <value>com.yhd.hive.auth.YhdPasswdAuthenticationProvider</value> > </property> > > <property> > <name>hive.hs2.authfile</name> > <value>/usr/auth.properties</value> > </property> > <property> > <name>tez.lib.uris</name> > <value>${fs.defaultFS}/user/tez/tez-0.7.0.tar.gz</value> > </property> > <property> > <name>hive.user.install.directory</name> > <value>${fs.defaultFS}/user/</value> > </property> > <property> > <name>tez.history.logging.service.class</name> > <value>org.apache.tez.dag.history.logging.ats.ATSHistoryLoggingService</value> > > </property> > <property> > <name>tez.runtime.convert.user-payload.to.history-text</name> > <value>true</value> > </property> > <property> > <description>Indicate to clients whether Timeline service is enabled or not. > If enabled, the TimelineClient library used by end-users will post entities > and events to the Timeline server.</description> > <name>yarn.timeline-service.enabled</name> > <value>true</value> > </property> > > <property> > <description>The hostname of the Timeline service web > application.</description> > <name>yarn.timeline-service.hostname</name> > <value>yhd-jqhadoop200.int.yihaodian.com</value> > </property> > > <property> > <description>Enables cross-origin support (CORS) for web services where > cross-origin web response headers are needed. For example, javascript making > a web services request to the timeline server.</description> > <name>yarn.timeline-service.http-cross-origin.enabled</name> > <value>true</value> > </property> > <property> > <name>tez.tez-ui.history-url.base</name> > <value>http://yhd-jqhadoop200.int.yihaodian.com:8088/tezui</value> > </property> > > > > [email protected] > > > <log.txt>
