Hey,
I am trying to add a native lib file from HDFS to distributed cache so it can
be loaded inside a UDF. This seems pretty standard and has been around for
years, so I’m guessing I am overlooking something. Here’s my basic approach,
please let me know if you spot the problem.
1. Put library in hdfs, located at ‘hdfs://localhost:8020/tmp/libcpp.dylib
2. Loaded into distributed cache in pig script:
set mapred.create.symlink yes;
set mapred.cache.files
'hdfs://localhost:8020/tmp/libcpp_jni_operators.dylib#libcpp_jni_operators.dylib';
Inside UDF:
System.loadLibrary("libcpp_jni_operators.dylib”);
The UDF fails due to library not found on java.library.path. (And inspecting
that path at runtime, the library is indeed not present).
Thanks,
Adam
