[ https://issues.apache.org/jira/browse/KYLIN-4268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
ZhouKang updated KYLIN-4268: ---------------------------- Description: When we update hadoop package, such as hbase and hive. The job will fill cause by: {code:java} // code placeholder Exception in thread "main" java.io.FileNotFoundException: File file:/home/work/app/kylin-all-2.5.2-2.5.2-HBase2-SNAPSHOT/hbase-2.2.0-3.1.0/lib/hbase-common-2.2.0-3.1.0.jar does not exist at org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:534) at org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:747) at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:524) at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:416) at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:337) at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:289) at org.apache.spark.deploy.yarn.Client.copyFileToRemote(Client.scala:406) at org.apache.spark.deploy.yarn.Client.org\$apache\$spark\$deploy\$yarn\$Client\$\$distribute\ {code} The root cause is kylin will generate the full command as soon as the job summited. And the parameter '--jars' use file's absolute path. there will be 2 ways to fix this problem. # keep the old package in the same path # change the generation of '--jars' to the stage of job execution. which is the better choose? was: When we update hadoop package, such as hbase and hive. The job will fill cause by: {code:java} // code placeholder Exception in thread "main" java.io.FileNotFoundException: File file:/home/work/app/kylin-all-2.5.2-2.5.2-HBase2-SNAPSHOT/hbase-2.2.0-3.1.0/lib/hbase-common-2.2.0-3.1.0.jar does not exist at org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:534) at org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:747) at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:524) at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:416) at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:337) at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:289) at org.apache.spark.deploy.yarn.Client.copyFileToRemote(Client.scala:406) at org.apache.spark.deploy.yarn.Client.org\$apache\$spark\$deploy\$yarn\$Client\$\$distribute\ {code} The root cause is kylin will generate the full command as soon as the job summited. And the parameter '--jars' use file's absolute path. there will be 2 ways to fix this problem. # keep the old package in the same path # change the generation of '--jars' to the stage of job execution. which is the better choose? > build job failed caused by java lib update > ------------------------------------------ > > Key: KYLIN-4268 > URL: https://issues.apache.org/jira/browse/KYLIN-4268 > Project: Kylin > Issue Type: Bug > Reporter: ZhouKang > Priority: Major > > When we update hadoop package, such as hbase and hive. The job will fill > cause by: > > {code:java} > // code placeholder > Exception in thread "main" java.io.FileNotFoundException: File > file:/home/work/app/kylin-all-2.5.2-2.5.2-HBase2-SNAPSHOT/hbase-2.2.0-3.1.0/lib/hbase-common-2.2.0-3.1.0.jar > does not exist > at > org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:534) > at > org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:747) > at > org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:524) > at > org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:416) > at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:337) > at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:289) > at > org.apache.spark.deploy.yarn.Client.copyFileToRemote(Client.scala:406) > at > org.apache.spark.deploy.yarn.Client.org\$apache\$spark\$deploy\$yarn\$Client\$\$distribute\ > {code} > > The root cause is kylin will generate the full command as soon as the job > summited. And the parameter '--jars' use file's absolute path. > > there will be 2 ways to fix this problem. > # keep the old package in the same path > # change the generation of '--jars' to the stage of job execution. > > which is the better choose? -- This message was sent by Atlassian Jira (v8.3.4#803005)