Hi,
With HDP-2.1.2 we are encountering the problem of:
2014-09-11 10:14:42,881 ERROR [TezChild]
org.apache.tez.runtime.task.TezTaskRunner: Exception of type Error.
Exiting now
java.lang.UnsatisfiedLinkError:
org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy()Z
at org.apache.hado
Hi Subroto,
It could be an installer/distro issue. You may need to redirect your question
to the Hortonworks forums to get an answer on this.
But yes, you are right - the env needs to be setup correctly to load the
required native libs. Out of curiosity, are you using Ambari to setup your
cl
Hi Hitesh,
I'm facing with same problem in Apache version. Can you run jobs with
snappy compression with 0.5.2 release?
Thanks,
- Tsuyoshi
On Mon, Nov 17, 2014 at 9:26 PM, Hitesh Shah wrote:
> Hi Subroto,
>
> It could be an installer/distro issue. You may need to redirect your question
> to t
Dear Tez community,
a customer is running into following problem with a Tez-0.5 job.
On the client side it looks as follow:
- progress reporting shows that all tasks are complete:
(DagRunner.java:148) - Progress on 'Import job (26234): Raw History
ImportJob#import(Group by operation)#im
Yes. I was going to do that soon. No one has suggested any changes to the
proposed 0.5.3 to close out the 0.5 line of releases. So maybe I will get
that out soon and update the website once. Then we can move on to 0.6 with
the UI focus.
Bikas
-Original Message-
From: Tsuyoshi OZAWA [mailt
Are these direct Tez jobs ?
The task and am env do need to be setup to make use of Snappy. The property
that needs to be set depends on the job being run. Tez direct would require
tez.am.launch.env and tez.task.launch.env to be set. For Hive, Pig, MR, etc
this may be different, depending on how the
On 11/17/14, 4:54 AM, Tsuyoshi OZAWA wrote:
I'm facing with same problem in Apache version. Can you run jobs with
snappy compression with 0.5.2 release?
There is a thread going on in the Apache mailing list about stopping the
shipping of the .so files as part of the binary distribution.
Are
The signal to the AM could have come in from the NodeManager if the AM
exceeds it's memory limits. Do you have YARN memory enforcement enabled on
this cluster ? You could look up the NM logs to see if this were the case.
Otherwise the RM logs to figure out when and if a KILL request for the app
was
By default, native libs are expected at $HADOOP_COMMON_HOME/lib/native/ for
linux and added to the AM/task launch env via
"LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HADOOP_COMMON_HOME/lib/native/". For
Windows the PATH is updated with the location instead of LD_LIBRARY_PATH.
If your native libs are in a
