Github user coderfi commented on the pull request:
https://github.com/apache/spark/pull/5876#issuecomment-104503508
Has this been tested with mapr4?
When running under python, I am running into an exception
Caused by: java.lang.UnsatisfiedLinkError: Native Library
/tmp/mapr-root-libMapRClient.4.0.2-mapr.so already loaded in another classloader
at java.lang.ClassLoader.loadLibrary1(ClassLoader.java:1931)
at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1890)
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1851)
at java.lang.Runtime.load0(Runtime.java:795)
at java.lang.System.load(System.java:1062)
at com.mapr.fs.shim.LibraryLoader.load(LibraryLoader.java:29)
This occurs under the following scenarios:
create a spark context + hive context
run a hive query
stop the spark context
create another spark context + hive context again
run a hive query
I get the Native Library exception
From what I can tell, when the ClassLoader is hitting a MAPR class which is
attempting to load the native library (I presume, as part of a static
initializer).
Unfortunately, the JVM prohibits this the second time around. :(
I could file a ticket with a full stack trace if you'd like.
thanks,
Fi
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
