[
https://issues.apache.org/jira/browse/KYLIN-1021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14962769#comment-14962769
]
fengYu commented on KYLIN-1021:
-------------------------------
Yes, This is simpler, Here is two way of deploying kylin env, one is having
consistent binary(hive/hbase jars) and configs across all nodes just like your
env, and another is hadoop nodes do not hive those binarys just like ours,
should we add a config property to distinguish those, or just distinguish it by
the property "kylin.job.mr.lib.dir".
> upload dependent jars of kylin to HDFS and set tmpjars
> ------------------------------------------------------
>
> Key: KYLIN-1021
> URL: https://issues.apache.org/jira/browse/KYLIN-1021
> Project: Kylin
> Issue Type: Improvement
> Affects Versions: v1.0
> Reporter: fengYu
> Assignee: liyang
> Attachments: 0001-hadoop-jar-files.patch
>
>
> As [~Shaofengshi] says in maillist : Regrading your question about the jar
> files located in local disk instead of HDFS, yes the hadoop/hive/hbase jars
> should exist in local disk on each machine of the hadoop cluster, with the
> same locations; Kylin will not upload those jars; Please check and ensure the
> consistency of your hadoop cluster.
> However, our hadoop cluster is managed by hadoop administrator, we have no
> right to login those machine, even though we have the right, copy all files
> to hundreds of machine will be a painful job(I do not know is there any tools
> can do the job well).
> By the way, I can not get any tips about you measure(If you has the document，
> tell me)...
> I change my source code and create a directory in kylin tmp
> directory(kylin.hdfs.working.dir/kylin_metadata) and upload all jars to the
> directory if the directory is empty(it only happened at the first time) when
> submitting a mapreduce job, then set those locations to tmpjars of the
> mapreduce job(just like kylin set tmpfiles before submit job), This is
> automated and make kylin deploying easier..
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
