[
https://issues.apache.org/jira/browse/IMPALA-11342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17569482#comment-17569482
]
ASF subversion and git services commented on IMPALA-11342:
----------------------------------------------------------
Commit 112f887ea5219e5835d3ca0403faa0ba9ec49115 in impala's branch
refs/heads/master from Csaba Ringhofer
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=112f887ea ]
IMPALA-11342: Fix class loading in Hive UDFs' constructors
Loading new classes from the same jar in the constructor of UDFs
did not work in the catalog because the URLClassLoader was closed
too early. Extended the lifecycle of the class loader a bit to
let the catalog finish all initialisation.
Note that the instantiation of legacy Hive UDFs doesn't seem
necessary in the catalog, we can get all relevant info from
the class. Generic UDFs do need to be instantiated to be able
to call initialize().
Testing:
- added new classes to load in test UDFs and loaded these
in constructor / initialize()
- ran the Hive UDF ee tests
Change-Id: If16e38b8fc3b2577a5d32104ea9e6948b9562e24
Reviewed-on: http://gerrit.cloudera.org:8080/18611
Reviewed-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
Tested-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
> Hive UDFs are unable to use classes from the same Jar in catalogd
> -----------------------------------------------------------------
>
> Key: IMPALA-11342
> URL: https://issues.apache.org/jira/browse/IMPALA-11342
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
> Reporter: Csaba Ringhofer
> Assignee: Csaba Ringhofer
> Priority: Major
>
> While it is a known limitations that Hive UDFs in Impala cannot load
> dependencies from classpath, they should be able to load classes from the
> same jar (and thus creating an shaded jar with the dependencies should be a
> valid workaround). Meanwhile I got an error when the parent class of the UDF
> was a class defined in another package in the jar.
> The exception occurred when catalogd tried to instantiate the UDF class:
> {code}
> I0608 14:50:21.520296 22464 jni-util.cc:286] java.lang.NoClassDefFoundError:
> com/esri/core/geometry/Point
> at java.lang.Class.getDeclaredConstructors0(Native Method)
> at java.lang.Class.privateGetDeclaredConstructors(Class.java:2671)
> at java.lang.Class.getConstructor0(Class.java:3075)
> at java.lang.Class.getConstructor(Class.java:1825)
> at
> org.apache.impala.hive.executor.HiveLegacyJavaFunction.instantiateUDFInstance(HiveLegacyJavaFunction.java:115)
> at
> org.apache.impala.hive.executor.HiveLegacyJavaFunction.<init>(HiveLegacyJavaFunction.java:74)
> at
> org.apache.impala.hive.executor.HiveJavaFunctionFactoryImpl.create(HiveJavaFunctionFactoryImpl.java:50)
> at
> org.apache.impala.hive.executor.HiveJavaFunctionFactoryImpl.create(HiveJavaFunctionFactoryImpl.java:67)
> at
> org.apache.impala.service.CatalogOpExecutor.createFunction(CatalogOpExecutor.java:2117)
> at
> org.apache.impala.service.CatalogOpExecutor.execDdlRequest(CatalogOpExecutor.java:453)
> at org.apache.impala.service.JniCatalog.execDdl(JniCatalog.java:245)
> Caused by: java.lang.ClassNotFoundException: com.esri.core.geometry.Point
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> ... 11 more
> {code}
> The issue seems to be cause by closing the UrlClassLoader too early, after
> loading the class but before doing the actual instantiation:
> https://github.com/apache/impala/blob/13bbff4e4e5fc5d459cc6f7a5512f84ceba897cd/fe/src/main/java/org/apache/impala/hive/executor/HiveUdfLoader.java#L71
> Note that this is an issue only in the catalogd, not the executors, as the
> class loader is kept open longer in that case.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
