> On Jan. 8, 2016, 1:49 p.m., Jonathan Hurley wrote: > > Why go through all of these code changes to accomplish this? It seems like > > you could just have removed the `parallel-scheduler` element from the > > upgrade XML: > > > > ``` > > <group xsi:type="restart" name="HIVE" title="Hive"> > > <service-check>false</service-check> > > <skippable>true</skippable> > > <supports-auto-skip-failure>false</supports-auto-skip-failure> > > <parallel-scheduler/> > > <service name="HIVE"> > > <component>HIVE_METASTORE</component> > > <component>HIVE_SERVER</component> > > <component>WEBHCAT_SERVER</component> > > <component>HIVE_CLIENT</component> > > <component>HCAT</component> > > </service> > > </group> > > ``` > > Nate Cole wrote: > +1 for this > > Alejandro Fernandez wrote: > I want the group to still be in parallel, but only Hive Server and > WebHCat server to be serial. Breaking it up into 2 separate groups seemed too > clunky. If the default behavior of parallel-scheduler changes, then I still > ensure that those 2 components run sequentially. > > Jonathan Hurley wrote: > That's what the groupings are for though. I'd expect this kind of change > to be an XML-only change since it's so simple. Seems like we're introducing a > lot of new code which overcomplicates the solution. > > Alejandro Fernandez wrote: > I thought about that, but didn't want the EU/RU wizard to then show > multiple groups for Hive. > > Jonathan Hurley wrote: > I think this is a matter of trading off between a cleaner XML with a > single group vs having additional complex code for calculating the maximum > degree of parallelism. My feeling is that even though it's a neat feature, > it's not something required to fix this problem. Fixing it in the XML seems > to be more intuitive since it will be represented to the user as 2 different > groups. > > If others feel strongly that we should keep Hive as a single group, then > I'm OK with it. I just didn't think this much code was necessary to fix the > problem. > > Nate Cole wrote: > I would prefer keeping the code simpler. The whole point of the upgrade > pack "verbosity" is to make it clear to someone viewing it what is going to > happen. That there's two hive groups being shown isn't a big deal. "Hive" > and "Hive Clients." > > Jayush Luniya wrote: > While it would have been better to have an ability to override, I agree > that it is less readable based on how the upgrade pack needs to be authored. > It would have been better if groups could have sub-groups and we can set > overrides in sub-groups. That way we wouldn't see separate groups but > internally override group-level properties. For now maybe we can just keep it > simple and leave it as separate groups. > > <group name="Hive"> > <parallel-scheduler> > <max-degree-of-parallelism>100</max-degree-of-parallelism> > </parallel-scheduler> > <sub-group name="Hive"> > <parallel-scheduler> > <max-degree-of-parallelism>1</max-degree-of-parallelism> > </parallel-scheduler> > <sub-group> > <sub-group name="Hive Client"> > </sub-group> > </group>
RE: E.g., Hive Server copies tarballs, which can only run one at a time. Hive MetaStore updates the schema, and only one can run at a time. Wouldnt this been an issue during clean installs too? (assuming blueprint based HA deployments) - Jayush ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/42038/#review113472 ----------------------------------------------------------- On Jan. 7, 2016, 10:18 p.m., Alejandro Fernandez wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/42038/ > ----------------------------------------------------------- > > (Updated Jan. 7, 2016, 10:18 p.m.) > > > Review request for Ambari, Dmytro Grinenko, Dmitro Lisnichenko, Jonathan > Hurley, Jayush Luniya, and Nate Cole. > > > Bugs: AMBARI-14581 > https://issues.apache.org/jira/browse/AMBARI-14581 > > > Repository: ambari > > > Description > ------- > > During an Express Upgrade, all of the tasks for Hive Server and Hive > MetaStore run in parallel, so if these components have multiple hosts, then > running in parallel can cause errors during the upgrade. > E.g., Hive Server copies tarballs, which can only run one at a time. Hive > MetaStore updates the schema, and only one can run at a time. > > > Diffs > ----- > > > ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/Grouping.java > aa852bd > > ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/StageWrapperBuilder.java > b0df83f > > ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/Task.java > 60091a6 > > ambari-server/src/main/resources/stacks/HDP/2.1/upgrades/nonrolling-upgrade-2.3.xml > a46a5cd > > ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.2.xml > e57b13f > > ambari-server/src/main/resources/stacks/HDP/2.2/upgrades/nonrolling-upgrade-2.3.xml > b1c3739 > > ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.3.xml > ba2578b > > ambari-server/src/test/java/org/apache/ambari/server/state/UpgradeHelperTest.java > 7e61942 > > ambari-server/src/test/java/org/apache/ambari/server/state/stack/UpgradePackTest.java > 15be8b4 > > ambari-server/src/test/resources/stacks/HDP/2.0.7/services/HIVE/metainfo.xml > 35a4d05 > > ambari-server/src/test/resources/stacks/HDP/2.0.7/services/YARN/metainfo.xml > beee6a0 > > ambari-server/src/test/resources/stacks/HDP/2.1.1/upgrades/upgrade_test_nonrolling.xml > 9f57ca8 > > Diff: https://reviews.apache.org/r/42038/diff/ > > > Testing > ------- > > Ran unit tests in UpgradeHelperTest.java and UpgradePackTest.java > Verified with EU from HDP 2.2->2.3 with multiple Hive MetaStores. > > Waiting for unit test results. > > > Thanks, > > Alejandro Fernandez > >
