The UDF interface does not currently include the ability for a UDF to
indicate additional jars it would like to have packaged and sent along.
Alan.
On Mar 10, 2010, at 2:21 AM, Tamir Kamara wrote:
Hi,
Register is working fine but it means that the user needs to know
when it's
needed to
Okay what do you mean by package and send along? What is the pig way to
include additional jars? e.g. we want to use a 3rd party library to encode
json and how can our UDF reference that jar?
On Mar 15, 2010, at 12:49 PM, Alan Gates wrote:
The UDF interface does not currently include the
Your UDF will reference the classes the regular way - just use imports. The
trick is to make sure the jars are on the machine classpath. Two ways to
do this -- pre-load them on the cluster and have them configured to be on
the default classpath, or use Pig's REGISTER keyword to register both your
Hey Corbin,
Alternatively, you could use whatever build tool you're using (Maven, Ant)
and include the JSON library as a dependency and configure so that you can
have it build a jar with dependencies.
On Mon, Mar 15, 2010 at 5:08 PM, Dmitriy Ryaboy dvrya...@gmail.com wrote:
Your UDF will
Hey,
How's the progress on teh JSON UDF? If you post it on the Pig JIRA I could
get a chance to take a look and help out. Also it would get the ball rolling
on getting the UDF added to piggybank
On Mon, Mar 15, 2010 at 4:52 PM, Corbin Hoenes cor...@tynt.com wrote:
Okay what do you mean by
Hi,
I have a function (eval) that needs to use an external jar.
In M/R world this can be accomplished by uploading the jar to the dfs and
using DistributedCache.addFileToClassPath.
How do I do the same (have the jar available for the udf) in pig?
Thanks,
Tamir
Using *REGISTER myfunc.jar;*
refer here:
http://hadoop.apache.org/pig/docs/r0.5.0/piglatin_reference.html#REGISTER
On Wed, Mar 10, 2010 at 4:52 PM, Tamir Kamara tamirkam...@gmail.com wrote:
Hi,
I have a function (eval) that needs to use an external jar.
In M/R world this can be
Sorry maybe I misunderstand you. It seems you'd like to use third-party
library in your udf, then you need to package your udf and third-party
library in one jar.
On Wed, Mar 10, 2010 at 5:21 PM, Jeff Zhang zjf...@gmail.com wrote:
Using *REGISTER myfunc.jar;*
refer here:
Hi Jeff,
You are right - I want to use another jar in my own udf.
Packaging both into a single jar is certainly an option but I was hoping pig
would be able to do something similar to regular map-reduce where I push the
jar before hand to the DFS and then add it to the class path via the
Hi,
In M/R when you need an extra jar to use do you add the jar into the class
path by calling:
DistributedCache.addFileToClassPath(dfs-path-to-jar);
I imagine that the register command does something similar under the covers
but I was just looking for a way to have the UDF load its own
Hi Tamir,
I cannot send the reply from your thread (always receive failed delivered
message from google), so I reply to you in this thread.
The register command will cause the udf jar been packaged with pig.jar, and
the resulted jar is the mapredue job jar. If you do not want to been
bothered to
11 matches
Mail list logo
