If I have some jars that I would like including into ALL of my jobs is
there some shared lib directory on HDFS that will be available to all of
my nodes? Something similar to how Oozie uses a shared lib directory
when submitting workflows.
As of right now I've been cheating and copying
my jobs is
> there some shared lib directory on HDFS that will be available to all of my
> nodes? Something similar to how Oozie uses a shared lib directory when
> submitting workflows.
>
> As of right now I've been cheating and copying these jars into the hadoop
> lib directory on
he
archive gets untarred presuming one uses a relative path inside the archive.
Kevin
-Original Message-
From: 顾荣 [mailto:gurongwal...@gmail.com]
Sent: Monday, April 11, 2011 11:12 AM
To: common-user@hadoop.apache.org
Subject: Re: Shared lib?
Hi Mark,
I also met your problem,I found
I've been steadily adding more and more shared libraries to the -files option
of my pipes command and have had moderate success in that each time I add a new
library the app no longer fails on that library, but rather on the next one.
However, I've hit a snag. I'm getting the following error ev
I love working on a problem for an hour, sending an email for help, then
solving it. Problem was the space after the comma in the -files option:
$ hadoop pipes -D hadoop.pipes.java.recordreader=true -D
hadoop.pipes.java.recordwriter=true -files \
...[SEVERAL .so FILES TO DISTRIBUTED CACHE]
hdfs
