Thanks for answering.
I run my Hadoop in single node, not cluster mode.
On Thu, Apr 30, 2009 at 11:21 AM, jason hadoop <jason.had...@gmail.com> wrote:
> You need to make sure that the shared library is available on the
> tasktracker nodes, either by installing it, or by pushing it around via the
> distributed cache
>
>
>
> On Wed, Apr 29, 2009 at 8:19 PM, Ian jonhson <jonhson....@gmail.com> wrote:
>
>> Dear all,
>>
>> I wrote a plugin codes for Hadoop, which calls the interfaces
>> in Cpp-built .so library. The plugin codes are written in java,
>> so I prepared a JNI class to encapsulate the C interfaces.
>>
>> The java codes can be executed successfully when I compiled
>> it and run it standalone. However, it does not work when I embedded
>> in Hadoop. The exception shown out is (found in Hadoop logs):
>>
>>
>> ------------ screen dump ---------------------
>>
>> # grep myClass logs/* -r
>> logs/hadoop-hadoop-tasktracker-testbed0.container.org.out:Exception in
>> thread "JVM Runner jvm_200904261632_0001_m_-1217897050 spawned."
>> java.lang.UnsatisfiedLinkError:
>> org.apache.hadoop.mapred.myClass.myClassfsMount(Ljava/lang/String;)I
>> logs/hadoop-hadoop-tasktracker-testbed0.container.org.out: at
>> org.apache.hadoop.mapred.myClass.myClassfsMount(Native Method)
>> logs/hadoop-hadoop-tasktracker-testbed0.container.org.out:Exception in
>> thread "JVM Runner jvm_200904261632_0001_m_-1887898624 spawned."
>> java.lang.UnsatisfiedLinkError:
>> org.apache.hadoop.mapred.myClass.myClassfsMount(Ljava/lang/String;)I
>> logs/hadoop-hadoop-tasktracker-testbed0.container.org.out: at
>> org.apache.hadoop.mapred.myClass.myClassfsMount(Native Method)
>> ...
>>
>> --------------------------------------------------------
>>
>> It seems the library can not be loaded in Hadoop. My codes
>> (myClass.java) is like:
>>
>>
>> --------------- myClass.java ------------------
>>
>> public class myClass
>> {
>>
>> public static final Log LOG =
>> LogFactory.getLog("org.apache.hadoop.mapred.myClass");
>>
>>
>> public myClass() {
>>
>> try {
>> //System.setProperty("java.library.path",
>> "/usr/local/lib");
>>
>> /* The above line does not work, so I have to
>> do something
>> * like following line.
>> */
>> addDir(new String("/usr/local/lib"));
>> System.loadLibrary("myclass");
>> }
>> catch(UnsatisfiedLinkError e) {
>> LOG.info( "Cannot load library:\n " +
>> e.toString() );
>> }
>> catch(IOException ioe) {
>> LOG.info( "IO error:\n " +
>> ioe.toString() );
>> }
>>
>> }
>>
>> /* Since the System.setProperty() does not work, I have to add the
>> following
>> * function to force the path is added in java.library.path
>> */
>> public static void addDir(String s) throws IOException {
>>
>> try {
>> Field field =
>> ClassLoader.class.getDeclaredField("usr_paths");
>> field.setAccessible(true);
>> String[] paths = (String[])field.get(null);
>> for (int i = 0; i < paths.length; i++) {
>> if (s.equals(paths[i])) {
>> return;
>> }
>> }
>> String[] tmp = new String[paths.length+1];
>> System.arraycopy(paths,0,tmp,0,paths.length);
>> tmp[paths.length] = s;
>>
>> field.set(null,tmp);
>> } catch (IllegalAccessException e) {
>> throw new IOException("Failed to get
>> permissions to set library path");
>> } catch (NoSuchFieldException e) {
>> throw new IOException("Failed to get field
>> handle to set library path");
>> }
>> }
>>
>> public native int myClassfsMount(String subsys);
>> public native int myClassfsUmount(String subsys);
>>
>>
>> }
>>
>> --------------------------------------------------------
>>
>>
>> I don't know what missed in my codes and am wondering whether there are any
>> rules in Hadoop I should obey if I want to achieve my target.
>>
>> FYI, the myClassfsMount() and myClassfsUmount() will open a socket to call
>> services from a daemon. I would better if this design did not cause the
>> fail in
>> my codes.
>>
>>
>> Any comments?
>>
>>
>> Thanks in advance,
>>
>> Ian
>>
>
>
>
> --
> Alpha Chapters of my book on Hadoop are available
> http://www.apress.com/book/view/9781430219422
>
