UPDATE: Tez UI (using 1.0 storage mechanism) has been successfully enabled
on our CDH 5.8.3 cluster. Here's the summary:
- Build Tez against CDH repo (5.8.3 version)
- Build against 1.9.13 versions of jackson jars, listed in the following:
- jackson-mapper-asl
- jackson-core-asl
- jackson-jaxrs
- jackson-xc
- Set a separate timeline server using Apache 2.7.3 version of cluster
- Setup Tez UI using apache webserver
- Follow instructions on how to setup each of these components (ATS, CDH
cluster to publish events to ATS, CORS etc)
Cheers,
Manoj
On Fri, Feb 24, 2017 at 11:15 AM, Manoj Murumkar <manoj.murum...@gmail.com>
wrote:
> Also, please note that I am using ATS with apache 2.7.3 version.
>
> Jan,
>
> I looked at the post you had made earlier.
>
> we are using Tez 0.8.4 successfully both with Hive and Pig on our
> > Cloudera CDH 5.7.1 cluster.
>
> CDH comes with Pig 0.12 which doesn't have tez as a supported engine. How did
> you manage to get that working?
>
>
> On Fri, Feb 24, 2017 at 10:50 AM, Manoj Murumkar <manoj.murum...@gmail.com
> > wrote:
>
>> So, I am able to get the metric posting part working (part where tez
>> posts counters to backend). I can see data coming in and without any java
>> exceptions in the run logs of tez sessions. I essentially built tez with
>> 1.9.13 version of jackson libraries.
>>
>> <dependency>
>> <groupId>org.codehaus.jackson</groupId>
>> <artifactId>jackson-mapper-asl</artifactId>
>> <version>1.9.13</version>
>> </dependency>
>> <dependency>
>> <groupId>org.codehaus.jackson</groupId>
>> <artifactId>jackson-core-asl</artifactId>
>> <version>1.9.13</version>
>> </dependency>
>> <dependency>
>> <groupId>org.codehaus.jackson</groupId>
>> <artifactId>jackson-jaxrs</artifactId>
>> <version>1.9.13</version>
>> </dependency>
>> <dependency>
>> <groupId>org.codehaus.jackson</groupId>
>> <artifactId>jackson-xc</artifactId>
>> <version>1.9.13</version>
>>
>> Now, when I check the timeline server webUI, I see nulls in pretty much
>> everything (see screenshot). Could this be another version incompatibility
>> issue?
>>
>> Thanks,
>>
>> Manoj
>>
>>
>>
>> On Fri, Feb 24, 2017 at 9:43 AM, Edward Capriolo <edlinuxg...@gmail.com>
>> wrote:
>>
>>> We should really just force tez into the build of hive. It sooo fin
>>> stupid this situation. Just drives people at impala / spark. Im brining it
>>> up in hive.
>>>
>>>
>>> On Friday, February 24, 2017, Jan Morlock <jan.morl...@googlemail.com>
>>> wrote:
>>>
>>>> Hi Manoj,
>>>>
>>>> sorry for the late reply. Your problem appears to be similar to
>>>>
>>>> https://issues.apache.org/jira/browse/HIVE-8271
>>>>
>>>> So please check whether you have both jackson-1.8 and jackson-1.9
>>>> somewhere in your classpath.
>>>>
>>>> That being said, you should know, that you can make a lot of friends if
>>>> you succeed in getting the Tez UI running on CDH.
>>>> I also once tried but didn't pursue further. You can find my post here:
>>>>
>>>> http://mail-archives.apache.org/mod_mbox/tez-user/201608.mbo
>>>> x/%3cCAO25eDBaHYcKaL+pBp2wR0TyXOcaMAsJU+YDvEmFVnYQer6ALg@mai
>>>> l.gmail.com%3e
>>>>
>>>> I hope that helps.
>>>>
>>>> Cheers
>>>> Jan
>>>>
>>>>
>>>>
>>>>
>>>> 2017-02-17 0:02 GMT+01:00 Manoj Murumkar <manoj.murum...@gmail.com>:
>>>>
>>>>> Also, hive version that's bundled with CDH5.8 is 1.1.0 which is also
>>>>> bundled with same class. Perhaps that's the one in effect? If that's the
>>>>> case, is there no way to use timeline server for history with this
>>>>> version?
>>>>>
>>>>> On Thu, Feb 16, 2017 at 2:53 PM, Manoj Murumkar <
>>>>> manoj.murum...@gmail.com> wrote:
>>>>>
>>>>>> Btw, here's the environment info:
>>>>>>
>>>>>> Tez: 0.8.4
>>>>>> Timeline server: bundled with apache 2.6.5 version
>>>>>>
>>>>>> I have added following libraries (that are used by timeline server)
>>>>>> in the path, so there's no mis-match, but no luck. I am still getting
>>>>>> same
>>>>>> error.
>>>>>>
>>>>>> HADOOP_CLASSPATH=/usr/local/tez/client/lib/old/jackson-core-
>>>>>> asl-1.9.13.jar:/usr/local/tez/client/lib/old/jackson-mapper-
>>>>>> asl-1.9.13.jar:....
>>>>>> HADOOP_USER_CLASSPATH_FIRST=true
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Thu, Feb 16, 2017 at 12:02 PM, Manoj Murumkar <
>>>>>> manoj.murum...@gmail.com> wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> Wondering if anyone has successfully enabled Tez-UI (by running ATS
>>>>>>> from a separate apache installation) on CDH 5.8 cluster. Would
>>>>>>> appreciate
>>>>>>> any information on this. We are have trouble with API incompatibility
>>>>>>> (error pasted below):
>>>>>>>
>>>>>>> 2017-02-16 19:40:13,874 [FATAL] [HistoryEventHandlingThread]
>>>>>>> |yarn.YarnUncaughtExceptionHandler|: Thread
>>>>>>> Thread[HistoryEventHandlingThread,5,main] threw an Error. Shutting
>>>>>>> down now...
>>>>>>> java.lang.AbstractMethodError:
>>>>>>> org.codehaus.jackson.map.AnnotationIntrospector.findSerializer(Lorg/codehaus/jackson/map/introspect/Annotated;)Ljava/lang/Object;
>>>>>>> at
>>>>>>> org.codehaus.jackson.map.ser.BasicSerializerFactory.findSerializerFromAnnotation(BasicSerializerFactory.java:362)
>>>>>>> at
>>>>>>> org.codehaus.jackson.map.ser.BeanSerializerFactory.createSerializer(BeanSerializerFactory.java:252)
>>>>>>> at
>>>>>>> org.codehaus.jackson.map.ser.StdSerializerProvider._createUntypedSerializer(StdSerializerProvider.java:782)
>>>>>>> at
>>>>>>> org.codehaus.jackson.map.ser.StdSerializerProvider._createAndCacheUntypedSerializer(StdSerializerProvider.java:735)
>>>>>>> at
>>>>>>> org.codehaus.jackson.map.ser.StdSerializerProvider.findValueSerializer(StdSerializerProvider.java:344)
>>>>>>> at
>>>>>>> org.codehaus.jackson.map.ser.StdSerializerProvider.findTypedValueSerializer(StdSerializerProvider.java:420)
>>>>>>> at
>>>>>>> org.codehaus.jackson.map.ser.StdSerializerProvider._serializeValue(StdSerializerProvider.java:601)
>>>>>>> at
>>>>>>> org.codehaus.jackson.map.ser.StdSerializerProvider.serializeValue(StdSerializerProvider.java:256)
>>>>>>> at
>>>>>>> org.codehaus.jackson.map.ObjectMapper.writeValue(ObjectMapper.java:1604)
>>>>>>> at
>>>>>>> org.codehaus.jackson.jaxrs.JacksonJsonProvider.writeTo(JacksonJsonProvider.java:527)
>>>>>>> at
>>>>>>> com.sun.jersey.api.client.RequestWriter.writeRequestEntity(RequestWriter.java:300)
>>>>>>> at
>>>>>>> com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:204)
>>>>>>> at
>>>>>>> com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:147)
>>>>>>> at
>>>>>>> org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl$TimelineJerseyRetryFilter$1.run(TimelineClientImpl.java:226)
>>>>>>> at
>>>>>>> org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl$TimelineClientConnectionRetry.retryOn(TimelineClientImpl.java:162)
>>>>>>> at
>>>>>>> org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl$TimelineJerseyRetryFilter.handle(TimelineClientImpl.java:237)
>>>>>>> at com.sun.jersey.api.client.Client.handle(Client.java:648)
>>>>>>> at
>>>>>>> com.sun.jersey.api.client.WebResource.handle(WebResource.java:670)
>>>>>>> at
>>>>>>> com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
>>>>>>> at
>>>>>>> com.sun.jersey.api.client.WebResource$Builder.post(WebResource.java:563)
>>>>>>> at
>>>>>>> org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl.doPostingObject(TimelineClientImpl.java:472)
>>>>>>> at
>>>>>>> org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl.doPosting(TimelineClientImpl.java:321)
>>>>>>> at
>>>>>>> org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl.putEntities(TimelineClientImpl.java:301)
>>>>>>> at
>>>>>>> org.apache.tez.dag.history.logging.ats.ATSHistoryLoggingService.handleEvents(ATSHistoryLoggingService.java:357)
>>>>>>> at
>>>>>>> org.apache.tez.dag.history.logging.ats.ATSHistoryLoggingService.access$700(ATSHistoryLoggingService.java:53)
>>>>>>> at
>>>>>>> org.apache.tez.dag.history.logging.ats.ATSHistoryLoggingService$1.run(ATSHistoryLoggingService.java:190)
>>>>>>> at java.lang.Thread.run(Thread.java:745)
>>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>>> Manoj
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>> --
>>> Sorry this was sent from mobile. Will do less grammar and spell check
>>> than usual.
>>>
>>
>>
>
