Re: Getting Tez working against cdh 5.3

2015-01-23 Thread Gopal V
On 1/20/15, 4:11 PM, Edward Capriolo wrote: I see. That helped a lot. java.lang.UnsatisfiedLinkError: org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy()Z at org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy(Native ... I am not sure what this fix is doing exactly. Does

Re: Getting Tez working against cdh 5.3

2015-01-20 Thread Edward Capriolo
I see. That helped a lot. java.lang.UnsatisfiedLinkError: org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy()Z at org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy(Native Method) at org.apache.hadoop.io.compress.SnappyCodec.checkNativeCodeLoaded(SnappyCodec.java:63)

Re: Getting Tez working against cdh 5.3

2015-01-20 Thread Gopal V
On 1/20/15, 12:34 PM, Edward Capriolo wrote: Actually more likely something like this: https://issues.apache.org/jira/browse/TEZ-1621 I have a working Hive-13 + Tez install on CDH-5.2.0-1.cdh5.2.0.p0.36. Most of the work needed to get that to work was to build all of Hive+Tez against the CDH

Re: Getting Tez working against cdh 5.3

2015-01-20 Thread Prasanth Jayachandran
Hi Ed, Can you post your concerns to u...@tez.apache.org? Someone should be able to help you out with the issue. - Prasanth On Tue, Jan 20, 2015 at 12:35 PM, Edward Capriolo wrote: > Actually more likely something like this: > https://issues.apache.org/jira/browse/TEZ-1621 > On Tue, Ja

Re: Getting Tez working against cdh 5.3

2015-01-20 Thread Edward Capriolo
Actually more likely something like this: https://issues.apache.org/jira/browse/TEZ-1621 On Tue, Jan 20, 2015 at 2:02 PM, Prasanth Jayachandran < pjayachand...@hortonworks.com> wrote: > My guess is.. > "java" binary is not in PATH of the shell script that launches the > container.. try creating

Re: Getting Tez working against cdh 5.3

2015-01-20 Thread Edward Capriolo
Actually you are correct. Java is not universally on the path. Is there a way to make tez/hadoop respect/use whatever the node manager is using. If I am running node-managing I must have java right? :) On Tue, Jan 20, 2015 at 2:02 PM, Prasanth Jayachandran < pjayachand...@hortonworks.com> wrote:

Re: Getting Tez working against cdh 5.3

2015-01-20 Thread Edward Capriolo
Silly question but... 2015-01-20 15:01:45,366 INFO [IPC Server handler 0 on 41329] org.apache.tez.dag.app.rm.container.AMContainerImpl: AMContainer container_1420748315294_70716_01_02 transitioned from IDLE to RUNNING via event C_PULL_TA 2015-01-20 15:01:45,366 INFO [IPC Server handler 0 on 41

Re: Getting Tez working against cdh 5.3

2015-01-20 Thread Edward Capriolo
Java is on the PATH of our datanode/nodemanager systems [ecapriolo@production-hadoop-cdh-67-142 ~]$ which java /usr/bin/java [ecapriolo@production-hadoop-cdh-67-142 ~]$ java -version java version "1.7.0_65" OpenJDK Runtime Environment (rhel-2.5.1.2.el6_5-x86_64 u65-b17) OpenJDK 64-Bit Server VM (

Re: Getting Tez working against cdh 5.3

2015-01-20 Thread Prasanth Jayachandran
My guess is.. "java" binary is not in PATH of the shell script that launches the container.. try creating a symbolic link in /bin/ to point to java.. On Tue, Jan 20, 2015 at 7:22 AM, Edward Capriolo wrote: > It seems that CDH does not ship with enough jars to run tez out of the box. > > I have

Getting Tez working against cdh 5.3

2015-01-20 Thread Edward Capriolo
It seems that CDH does not ship with enough jars to run tez out of the box. I have found the related cloudera forked pom. In this pom hive is built against tez 0.4.1-incubating-tez2.0-SHAPSHOT Thus I followed the instructions here: http://tez.apache.org/install_pre_0_5_0.html hive> dfs -lsr /a