[
https://issues.apache.org/jira/browse/HIVE-1157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12848261#action_12848261
]
Edward Capriolo commented on HIVE-1157:
---------------------------------------
{noformat}
[edw...@ec hive]$ ant -Dtestcase=TestAddJarFromHDFS test
<testcase classname="org.apache.hadoop.hive.ql.session.TestAddJarFromHDFS"
name="testAddJarFromHDFS" time="6.73">
<error type="java.lang.NullPointerException">java.lang.NullPointerException
at
org.apache.hadoop.hive.ql.session.SessionState$JarResourceHook.preHook(SessionState.java:391)
at
org.apache.hadoop.hive.ql.session.SessionState.add_resource(SessionState.java:474)
at
org.apache.hadoop.hive.ql.processors.AddResourceProcessor.run(AddResourceProcessor.java:52)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:173)
at
org.apache.hadoop.hive.ql.session.TestAddJarFromHDFS.testAddJarFromHDFS(TestAddJarFromHDFS.java:71)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at junit.framework.TestCase.runTest(TestCase.java:154)
at junit.framework.TestCase.runBare(TestCase.java:127)
at junit.framework.TestResult$1.protect(TestResult.java:106)
at junit.framework.TestResult.runProtected(TestResult.java:124)
at junit.framework.TestResult.run(TestResult.java:109)
at junit.framework.TestCase.run(TestCase.java:118)
at junit.framework.TestSuite.runTest(TestSuite.java:208)
at junit.framework.TestSuite.run(TestSuite.java:203)
at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:422)
at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:931)
at
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:785)
</error>
</testcase>
<system-out><![CDATA[Starting DataNode 0 with dfs.data.dir:
build/test/data/dfs/data/data1,build/test/data/dfs/data/data2
]]></system-out>
<system-err><![CDATA[Waiting for the Mini HDFS Cluster to start...
]]></system-err>
</testsuite>
{noformat}
{noformat}
String jarURI = fs.getUri().toString() + "/addJarFromHdfs.jar";
int ret = cliDriver.processCmd("ADD JAR " + jarURI);
{noformat}
This is a clean checkout and build. I will try to trace this down more.
> UDFs can't be loaded via "add jar" when jar is on HDFS
> ------------------------------------------------------
>
> Key: HIVE-1157
> URL: https://issues.apache.org/jira/browse/HIVE-1157
> Project: Hadoop Hive
> Issue Type: Improvement
> Components: Query Processor
> Reporter: Philip Zeyliger
> Priority: Minor
> Attachments: hive-1157.patch.txt, HIVE-1157.v2.patch.txt
>
>
> As discussed on the mailing list, it would be nice if you could use UDFs that
> are on jars on HDFS. The proposed implementation would be for "add jar" to
> recognize that the target file is on HDFS, copy it locally, and load it into
> the classpath.
> {quote}
> Hi folks,
> I have a quick question about UDF support in Hive. I'm on the 0.5 branch.
> Can you use a UDF where the jar which contains the function is on HDFS, and
> not on the local filesystem. Specifically, the following does not seem to
> work:
> # This is Hive 0.5, from svn
> $bin/hive
> Hive history file=/tmp/philip/hive_job_log_philip_201002081541_370227273.txt
> hive> add jar hdfs://localhost/FooTest.jar;
>
> Added hdfs://localhost/FooTest.jar to class path
> hive> create temporary function cube as 'com.cloudera.FooTestUDF';
>
> FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.FunctionTask
> Does this work for other people? I could probably fix it by changing "add
> jar" to download remote jars locally, when necessary (to load them into the
> classpath), or update URLClassLoader (or whatever is underneath there) to
> read directly from HDFS, which seems a bit more fragile. But I wanted to
> make sure that my interpretation of what's going on is right before I have at
> it.
> Thanks,
> -- Philip
> {quote}
> {quote}
> Yes that's correct. I prefer to download the jars in "add jar".
> Zheng
> {quote}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
