[ https://issues.apache.org/jira/browse/HIVE-26614?focusedWorklogId=815140&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-815140 ]
ASF GitHub Bot logged work on HIVE-26614: ----------------------------------------- Author: ASF GitHub Bot Created on: 10/Oct/22 09:04 Start Date: 10/Oct/22 09:04 Worklog Time Spent: 10m Work Description: deniskuzZ commented on code in PR #3652: URL: https://github.com/apache/hive/pull/3652#discussion_r991035653 ########## ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java: ########## @@ -4158,7 +4158,7 @@ public static void addDependencyJars(Configuration conf, Class<?>... classes) if (!localFs.exists(new Path(path))) { throw new RuntimeException("Could not validate jar file " + path + " for class " + clazz); } - jars.add(path); + jars.add(localFs.makeQualified(new Path(path)).toString()); Review Comment: could we use the FileUtils helper method? ```` FileUtils.makeQualified(path, conf).toString() ```` Issue Time Tracking ------------------- Worklog Id: (was: 815140) Time Spent: 50m (was: 40m) > Fix adding custom jars in Job Classpath > --------------------------------------- > > Key: HIVE-26614 > URL: https://issues.apache.org/jira/browse/HIVE-26614 > Project: Hive > Issue Type: Bug > Reporter: Ayush Saxena > Assignee: Ayush Saxena > Priority: Major > Labels: pull-request-available > Time Spent: 50m > Remaining Estimate: 0h > > Custom added Jars in LocalFs throws FNF: > {noformat} > ERROR : Job Submission failed with exception > 'java.io.FileNotFoundException(File does not exist: > hdfs://localhost:9000/Users/ayushsaxena/code/hive-os/hive/packaging/target/apache-hive-4.0.0-alpha-2-SNAPSHOT-bin/apache-hive-4.0.0-alpha-2-SNAPSHOT-bin/lib/hive-iceberg-handler-4.0.0-alpha-2-SNAPSHOT.jar)' > java.io.FileNotFoundException: File does not exist: > hdfs://localhost:9000/Users/ayushsaxena/code/hive-os/hive/packaging/target/apache-hive-4.0.0-alpha-2-SNAPSHOT-bin/apache-hive-4.0.0-alpha-2-SNAPSHOT-bin/lib/hive-iceberg-handler-4.0.0-alpha-2-SNAPSHOT.jar > at > org.apache.hadoop.hdfs.DistributedFileSystem$29.doCall(DistributedFileSystem.java:1756) > at > org.apache.hadoop.hdfs.DistributedFileSystem$29.doCall(DistributedFileSystem.java:1749) > at > org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) > at > org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1764) > at org.apache.hadoop.fs.FileSystem.resolvePath(FileSystem.java:958) > at > org.apache.hadoop.mapreduce.v2.util.MRApps.addToClasspathIfNotJar(MRApps.java:342) > at > org.apache.hadoop.mapreduce.v2.util.MRApps.setClasspath(MRApps.java:275) > at > org.apache.hadoop.mapred.YARNRunner.setupContainerLaunchContextForAM(YARNRunner.java:525) > at > org.apache.hadoop.mapred.YARNRunner.createApplicationSubmissionContext(YARNRunner.java:584) > at org.apache.hadoop.mapred.YARNRunner.submitJob(YARNRunner.java:326) > at > org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:251) > at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1571) > at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1568) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1878) > at org.apache.hadoop.mapreduce.Job.submit(Job.java:1568) > at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:576) > at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:571) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1878) > at > org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:571) > at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:562) > at > org.apache.hadoop.hive.ql.exec.mr.ExecDriver.execute(ExecDriver.java:416) > at > org.apache.hadoop.hive.ql.exec.mr.MapRedTask.execute(MapRedTask.java:158){noformat} > Some Applications do consider every Jar Path to be in LocalFileSystem but > rest follow the standard Hadoop practice to check the Fs.DefaultFs. We should > better make the path qualified -- This message was sent by Atlassian Jira (v8.20.10#820010)