[
https://issues.apache.org/jira/browse/YARN-5667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15523418#comment-15523418
]
Sangjin Lee commented on YARN-5667:
-----------------------------------
In principle, no. You'll find that we've already isolated the hbase test code
into a separate module (hadoop-yarn-server-timelineservice-hbase-tests). I
considered moving the non-test hbase code there too, but stopped. The main
reason is bit complicated. There are two pieces of non-test hbase code that is
under timeline service: hbase client code and hbase co-processor code.
The major difference between them is that the hbase client code needs to run
under the hadoop yarn trunk dependencies (as that's where it needs to run)
whereas the co-processor code needs to run under the hbase dependencies. The
hbase tests require the hbase dependencies because of the testing utility
dependency.
So there is not only a potential for circular dependency, but also a diamond
dependency. If we were to separate the hbase code out of the timeline service
module, then we may need to create 2 more modules, one for hbase client and
another for hbase co-processor. But they have dependencies between them still.
So we'd need to untangle them.
I know it's bit dense, but please feel free to reach out to me so we can
discuss more.
> Move HBase backend code in ATS v2 into its separate module
> -----------------------------------------------------------
>
> Key: YARN-5667
> URL: https://issues.apache.org/jira/browse/YARN-5667
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: yarn
> Reporter: Haibo Chen
> Assignee: Haibo Chen
>
> The HBase backend code currently lives along with the core ATS v2 code in
> hadoop-yarn-server-timelineservice module. Because Resource Manager depends
> on hadoop-yarn-server-timelineservice, an unnecessary dependency of the RM
> module on HBase modules is introduced (HBase backend is pluggable, so we do
> not need to directly pull in HBase jars).
> In our internal effort to try ATS v2 with HBase 2.0 which depends on Hadoop
> 3, we encountered a circular dependency during our builds between HBase2.0
> and Hadoop3 artifacts.
> {code}
> hadoop-mapreduce-client-common, hadoop-yarn-client,
> hadoop-yarn-server-resourcemanager, hadoop-yarn-server-timelineservice,
> hbase-server, hbase-prefix-tree, hbase-hadoop2-compat,
> hadoop-mapreduce-client-jobclient, hadoop-mapreduce-client-common]
> {code}
> This jira proposes we move all HBase-backend-related code from
> hadoop-yarn-server-timelineservice into its own module (possible name is
> yarn-server-timelineservice-storage) so that core RM modules do not depend on
> HBase modules any more.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
