ZhouKang created KYLIN-4268:
-------------------------------
Summary: build job failed caused by java lib update
Key: KYLIN-4268
URL: https://issues.apache.org/jira/browse/KYLIN-4268
Project: Kylin
Issue Type: Bug
Reporter: ZhouKang
When we update hadoop package, such as hbase and hive. The job will fill cause
by:
{code:java}
// code placeholder
Exception in thread "main" java.io.FileNotFoundException: File
file:/home/work/app/kylin-all-2.5.2-2.5.2-HBase2-SNAPSHOT/hbase-2.2.0-3.1.0/lib/hbase-common-2.2.0-3.1.0.jar
does not exist
at
org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:534)
at
org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:747)
at
org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:524)
at
org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:416)
at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:337)
at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:289)
at
org.apache.spark.deploy.yarn.Client.copyFileToRemote(Client.scala:406)
at
org.apache.spark.deploy.yarn.Client.org\$apache\$spark\$deploy\$yarn\$Client\$\$distribute\
{code}
The root cause is kylin will generate the full command as soon as the job
summited. And the parameter '--jars' use file's absolute path.
there will be 2 ways to fix this problem.
# keep the old package in the same path
# change the generation of '--jars' to the stage of job execution.
which is the better choose?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
