[ https://issues.apache.org/jira/browse/YARN-5667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15570498#comment-15570498 ]
Sangjin Lee commented on YARN-5667: ----------------------------------- Took a closer look at the patch (mostly at the pom changes). I prefer that we are 100% accurate in terms of including all direct references (classes referring to classes in a dependency directly) as dependencies. I run mvn dependency:analyze and ideally there should be no "used undeclared" dependencies. That's just goo hygiene. I do see quite a few used undeclared dependencies in the timelineservice, timelineservice-hbase and timelineservice-hbase-tests modules. Could you please look into them and restore them accurately? In some more detail, (timelineservice) - yarn-api should not be removed (the entity API is all here) - yarn-server-common should not be removed - yarn-common should not be removed (e.g. YarnRPC) - jersey-core should not be removed (javax.ws.rs package) (timelineservice-hbase) - should add the license - yarn-api should be added (the entity API is all here) - yarn-server-applicationhistoryserver should be added (GenericObjectMapper) - hadoop-common should be added - hadoop- annotations should be added - hadoop-yarn-common should be added (timelineservice-hbase-tests) - similarly, we should not remove the dependencies here: they were carefully analyzed and added for direct dependencies - also, we still need to add timelineservice (there is a dependency to that) > Move HBase backend code in ATS v2 into its separate module > ----------------------------------------------------------- > > Key: YARN-5667 > URL: https://issues.apache.org/jira/browse/YARN-5667 > Project: Hadoop YARN > Issue Type: Sub-task > Components: yarn > Reporter: Haibo Chen > Assignee: Haibo Chen > Attachments: New module structure.png, part1.yarn5667.prelim.patch, > part2.yarn5667.prelim.patch, part3.yarn5667.prelim.patch, > part4.yarn5667.prelim.patch, part5.yarn5667.prelim.patch, > pt1.yarn5667.001.patch, pt2.yarn5667.001.patch, pt3.yarn5667.001.patch, > pt4.yarn5667.001.patch, pt5.yarn5667.001.patch, pt6.yarn5667.001.patch, > pt9.yarn5667.001.patch, yarn5667-001.tar.gz > > > The HBase backend code currently lives along with the core ATS v2 code in > hadoop-yarn-server-timelineservice module. Because Resource Manager depends > on hadoop-yarn-server-timelineservice, an unnecessary dependency of the RM > module on HBase modules is introduced (HBase backend is pluggable, so we do > not need to directly pull in HBase jars). > In our internal effort to try ATS v2 with HBase 2.0 which depends on Hadoop > 3, we encountered a circular dependency during our builds between HBase2.0 > and Hadoop3 artifacts. > {code} > hadoop-mapreduce-client-common, hadoop-yarn-client, > hadoop-yarn-server-resourcemanager, hadoop-yarn-server-timelineservice, > hbase-server, hbase-prefix-tree, hbase-hadoop2-compat, > hadoop-mapreduce-client-jobclient, hadoop-mapreduce-client-common] > {code} > This jira proposes we move all HBase-backend-related code from > hadoop-yarn-server-timelineservice into its own module (possible name is > yarn-server-timelineservice-storage) so that core RM modules do not depend on > HBase modules any more. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org