Re: Review Request 64809: AMBARI-22685 Stack OS/repos removed from cluster installation UI showed up on the Versions tab after cluster is installed
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64809/#review194447 --- Ship it! Ship It! - Tim Thorpe On Dec. 22, 2017, 2:48 p.m., Di Li wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64809/ > --- > > (Updated Dec. 22, 2017, 2:48 p.m.) > > > Review request for Ambari and Tim Thorpe. > > > Bugs: AMBARI-22685 > https://issues.apache.org/jira/browse/AMBARI-22685 > > > Repository: ambari > > > Description > --- > > Even when a user had remove say Diebian , SLES OS/repos from cluster > installation UI after selecting a stack, they still showed up on the Versions > tab after cluster was installed. > > > Diffs > - > > ambari-web/app/controllers/installer.js f49fc11 > ambari-web/app/controllers/wizard/step8_controller.js e64a09a > > > Diff: https://reviews.apache.org/r/64809/diff/1/ > > > Testing > --- > > existing unit tests, patch a trunk cluster with the changes, remove some of > the OS from the stack page during cluster installation, run thru cluster > install. From Ambari web UI, on the Versions tab, check os/repos shown on the > cluster, verify only the ones selected during the installation time remain. > > > Thanks, > > Di Li > >
Re: Review Request 64781: AMBARI-22680 Ambari web UI does not fail an invalid repo base URL during cluster installation
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64781/#review194347 --- Ship it! Ship It! - Tim Thorpe On Dec. 21, 2017, 4:29 p.m., Di Li wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64781/ > --- > > (Updated Dec. 21, 2017, 4:29 p.m.) > > > Review request for Ambari and Tim Thorpe. > > > Bugs: AMBARI-22680 > https://issues.apache.org/jira/browse/AMBARI-22680 > > > Repository: ambari > > > Description > --- > > This is because the Ambari server backend never validated anything. It just > simply sent back HTTP 200 as a dummy response. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/RepositoryResourceProvider.java > edec1ee > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/RepositoryResourceProviderTest.java > abdef9b > > > Diff: https://reviews.apache.org/r/64781/diff/1/ > > > Testing > --- > > unit test, patch a trunk cluster with change, provide an invalid url during > cluster install, verify the UI stops users from proceeding to the next page > with an error about the URL being invalid. verify the UI moves to the next > page when the urls are correct. > > > Thanks, > > Di Li > >
Re: Review Request 64771: AMBARI-22681 Ambari web UI failed to update stack repo directly from the Versions tab
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/64771/#review194338 --- Ship it! Ship It! - Tim Thorpe On Dec. 21, 2017, 2:05 p.m., Di Li wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/64771/ > --- > > (Updated Dec. 21, 2017, 2:05 p.m.) > > > Review request for Ambari and Tim Thorpe. > > > Bugs: AMBARI-22681 > https://issues.apache.org/jira/browse/AMBARI-22681 > > > Repository: ambari > > > Description > --- > > REST API calls from the web got rejected by the ambari server with a missing > property error. > > > Diffs > - > > ambari-web/app/controllers/main/admin/stack_and_upgrade_controller.js > 2f0cb68 > ambari-web/app/utils/ajax/ajax.js 3f85dfdc > > > Diff: https://reviews.apache.org/r/64771/diff/1/ > > > Testing > --- > > existing JS unit tests. Patch a trunk cluster with the changes and verify the > UI now can save URLs without having to go to the admin view. > > > Thanks, > > Di Li > >
Re: Review Request 64311: AMBARI-22574 Failed to restart services on PPC cluster post Ambari upgrade for IOP/HDP migration
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64311/#review192887
---
Ship it!
Ship It!
- Tim Thorpe
On Dec. 5, 2017, 5:39 p.m., Di Li wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64311/
> ---
>
> (Updated Dec. 5, 2017, 5:39 p.m.)
>
>
> Review request for Ambari, Jonathan Hurley and Tim Thorpe.
>
>
> Bugs: AMBARI-22574
> https://issues.apache.org/jira/browse/AMBARI-22574
>
>
> Repository: ambari
>
>
> Description
> ---
>
> The cluster started as IOP 4.2.5 on PPC with NameNode HA enabled. I upgraded
> Ambari to Ambari 2.6.1. The NN HA setting does not matter to the issue.
> After the Ambari upgrade, I can't restart services by clicking the "Restart"
> button on each service with that "restart needed" icon. Nor can I abort the
> requests.
> Ambari server log contains the following error on unable to find redhat-ppc7
> os type.
> java.lang.RuntimeException:
> org.apache.ambari.server.controller.spi.SystemException: Operating System
> matching redhat-ppc7 could not be found
> at
> org.apache.ambari.server.actionmanager.ExecutionCommandWrapper.getExecutionCommand(ExecutionCommandWrapper.java:253)
> at
> org.apache.ambari.server.actionmanager.ActionScheduler.abortOperationsForStage(ActionScheduler.java:880)
> at
> org.apache.ambari.server.actionmanager.ActionScheduler.processCancelledRequestsList(ActionScheduler.java:1172)
> at
> org.apache.ambari.server.actionmanager.ActionScheduler.doWork(ActionScheduler.java:333)
> at
> org.apache.ambari.server.actionmanager.ActionScheduler.run(ActionScheduler.java:310)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.ambari.server.controller.spi.SystemException: Operating
> System matching redhat-ppc7 could not be found
> at
> org.apache.ambari.server.state.stack.upgrade.RepositoryVersionHelper.getOSEntityForHost(RepositoryVersionHelper.java:422)
> at
> org.apache.ambari.server.actionmanager.ExecutionCommandWrapper.getExecutionCommand(ExecutionCommandWrapper.java:247)
> repo_version table had "redhat7" as the os type in the repository
> information. I worked around the issue by updating repo_version table to have
> redhat-ppc7 as the os type via the following SQL query:
> update repo_version set
> repositories='[{"OperatingSystems/ambari_managed_repositories":"true","repositories":[
> {"Repositories/repo_id":"IOP-4.2.5","Repositories/base_url":"http://birepo-build.svl.ibm.com/repos/IOP/RHEL7/ppc64le/4.2.5.0/20170602_1749","Repositories/repo_name":"IOP"}
> ,
> {"Repositories/repo_id":"IOP-UTILS-1.3","Repositories/base_url":"http://birepo-build.svl.ibm.com/repos/IOP-UTILS/RHEL7/ppc64le/1.3","Repositories/repo_name":"IOP-UTILS"}
> ],"OperatingSystems/os_type":"redhat-ppc7"}]' where repo_version_id = 1;
>
>
> Diffs
> -
>
>
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog261.java
> ba95833
>
>
> Diff: https://reviews.apache.org/r/64311/diff/3/
>
>
> Testing
> ---
>
> upgrade ambari server rpms on an IOP 4.2.5 PPC cluster, replace ambari-server
> jar with the one built with the fixes, run ambari-server upgrade, verify
> ambari database has the repositories info in repo_version is updated from
> redhat7 to redhat-ppc7, verify post ambari upgrade, I could restart services
> without having ambari server threw exceptions.
>
>
> Thanks,
>
> Di Li
>
>
Re: Review Request 64311: AMBARI-22574 Failed to restart services on PPC cluster post Ambari upgrade for IOP/HDP migration
> On Dec. 5, 2017, 1:01 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog261.java
> > Lines 154 (patched)
> > <https://reviews.apache.org/r/64311/diff/2/?file=1908245#file1908245line154>
> >
> > We really shouldn't be hardcoding stack names and versions like this.
> >
> > what change caused this problem? there must be a better workaround
>
> Di Li wrote:
> Hello Jonathan,
>
> This is the code change that caused the err >> "AMBARI-22318
> repositoryFile entity populating for wrong repository for RU (dgrinenko)"
>
> BigInsights 4.2.5 uses "redhat7" as the os type in the repo_version table
> instead of "redhat-ppc7" expected by Dmytro's code. Since this blocks the
> IOP/HDP migration, it has to happen at the ambari-upgrade time and needs to
> deal with BigInsights 4.2.5 repositories.
Since the next release of 2.6 is supposed to be the last that supports
migration, this code should only be there for a short time.
- Tim
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64311/#review192856
---
On Dec. 4, 2017, 8:08 p.m., Di Li wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64311/
> -----------
>
> (Updated Dec. 4, 2017, 8:08 p.m.)
>
>
> Review request for Ambari, Jonathan Hurley and Tim Thorpe.
>
>
> Bugs: AMBARI-22574
> https://issues.apache.org/jira/browse/AMBARI-22574
>
>
> Repository: ambari
>
>
> Description
> ---
>
> The cluster started as IOP 4.2.5 on PPC with NameNode HA enabled. I upgraded
> Ambari to Ambari 2.6.1. The NN HA setting does not matter to the issue.
> After the Ambari upgrade, I can't restart services by clicking the "Restart"
> button on each service with that "restart needed" icon. Nor can I abort the
> requests.
> Ambari server log contains the following error on unable to find redhat-ppc7
> os type.
> java.lang.RuntimeException:
> org.apache.ambari.server.controller.spi.SystemException: Operating System
> matching redhat-ppc7 could not be found
> at
> org.apache.ambari.server.actionmanager.ExecutionCommandWrapper.getExecutionCommand(ExecutionCommandWrapper.java:253)
> at
> org.apache.ambari.server.actionmanager.ActionScheduler.abortOperationsForStage(ActionScheduler.java:880)
> at
> org.apache.ambari.server.actionmanager.ActionScheduler.processCancelledRequestsList(ActionScheduler.java:1172)
> at
> org.apache.ambari.server.actionmanager.ActionScheduler.doWork(ActionScheduler.java:333)
> at
> org.apache.ambari.server.actionmanager.ActionScheduler.run(ActionScheduler.java:310)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.ambari.server.controller.spi.SystemException: Operating
> System matching redhat-ppc7 could not be found
> at
> org.apache.ambari.server.state.stack.upgrade.RepositoryVersionHelper.getOSEntityForHost(RepositoryVersionHelper.java:422)
> at
> org.apache.ambari.server.actionmanager.ExecutionCommandWrapper.getExecutionCommand(ExecutionCommandWrapper.java:247)
> repo_version table had "redhat7" as the os type in the repository
> information. I worked around the issue by updating repo_version table to have
> redhat-ppc7 as the os type via the following SQL query:
> update repo_version set
> repositories='[{"OperatingSystems/ambari_managed_repositories":"true","repositories":[
> {"Repositories/repo_id":"IOP-4.2.5","Repositories/base_url":"http://birepo-build.svl.ibm.com/repos/IOP/RHEL7/ppc64le/4.2.5.0/20170602_1749","Repositories/repo_name":"IOP"}
> ,
> {"Repositories/repo_id":"IOP-UTILS-1.3","Repositories/base_url":"http://birepo-build.svl.ibm.com/repos/IOP-UTILS/RHEL7/ppc64le/1.3","Repositories/repo_name":"IOP-UTILS"}
> ],"OperatingSystems/os_type":"redhat-ppc7"}]' where repo_version_id = 1;
>
>
> Diffs
> -
>
>
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog261.java
> ba95833
>
>
> Diff: https://reviews.apache.org/r/64311/diff/2/
>
>
> Testing
> ---
>
> upgrade ambari server rpms on an IOP 4.2.5 PPC cluster, replace ambari-server
> jar with the one built with the fixes, run ambari-server upgrade, verify
> ambari database has the repositories info in repo_version is updated from
> redhat7 to redhat-ppc7, verify post ambari upgrade, I could restart services
> without having ambari server threw exceptions.
>
>
> Thanks,
>
> Di Li
>
>
Re: Review Request 64311: AMBARI-22574 Failed to restart services on PPC cluster post Ambari upgrade for IOP/HDP migration
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64311/#review192750
---
Ship it!
Ship It!
- Tim Thorpe
On Dec. 4, 2017, 7:20 p.m., Di Li wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64311/
> ---
>
> (Updated Dec. 4, 2017, 7:20 p.m.)
>
>
> Review request for Ambari, Jonathan Hurley and Tim Thorpe.
>
>
> Bugs: AMBARI-22574
> https://issues.apache.org/jira/browse/AMBARI-22574
>
>
> Repository: ambari
>
>
> Description
> ---
>
> The cluster started as IOP 4.2.5 on PPC with NameNode HA enabled. I upgraded
> Ambari to Ambari 2.6.1. The NN HA setting does not matter to the issue.
> After the Ambari upgrade, I can't restart services by clicking the "Restart"
> button on each service with that "restart needed" icon. Nor can I abort the
> requests.
> Ambari server log contains the following error on unable to find redhat-ppc7
> os type.
> java.lang.RuntimeException:
> org.apache.ambari.server.controller.spi.SystemException: Operating System
> matching redhat-ppc7 could not be found
> at
> org.apache.ambari.server.actionmanager.ExecutionCommandWrapper.getExecutionCommand(ExecutionCommandWrapper.java:253)
> at
> org.apache.ambari.server.actionmanager.ActionScheduler.abortOperationsForStage(ActionScheduler.java:880)
> at
> org.apache.ambari.server.actionmanager.ActionScheduler.processCancelledRequestsList(ActionScheduler.java:1172)
> at
> org.apache.ambari.server.actionmanager.ActionScheduler.doWork(ActionScheduler.java:333)
> at
> org.apache.ambari.server.actionmanager.ActionScheduler.run(ActionScheduler.java:310)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.ambari.server.controller.spi.SystemException: Operating
> System matching redhat-ppc7 could not be found
> at
> org.apache.ambari.server.state.stack.upgrade.RepositoryVersionHelper.getOSEntityForHost(RepositoryVersionHelper.java:422)
> at
> org.apache.ambari.server.actionmanager.ExecutionCommandWrapper.getExecutionCommand(ExecutionCommandWrapper.java:247)
> repo_version table had "redhat7" as the os type in the repository
> information. I worked around the issue by updating repo_version table to have
> redhat-ppc7 as the os type via the following SQL query:
> update repo_version set
> repositories='[{"OperatingSystems/ambari_managed_repositories":"true","repositories":[
> {"Repositories/repo_id":"IOP-4.2.5","Repositories/base_url":"http://birepo-build.svl.ibm.com/repos/IOP/RHEL7/ppc64le/4.2.5.0/20170602_1749","Repositories/repo_name":"IOP"}
> ,
> {"Repositories/repo_id":"IOP-UTILS-1.3","Repositories/base_url":"http://birepo-build.svl.ibm.com/repos/IOP-UTILS/RHEL7/ppc64le/1.3","Repositories/repo_name":"IOP-UTILS"}
> ],"OperatingSystems/os_type":"redhat-ppc7"}]' where repo_version_id = 1;
>
>
> Diffs
> -
>
>
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog261.java
> ba95833
>
>
> Diff: https://reviews.apache.org/r/64311/diff/1/
>
>
> Testing
> ---
>
> upgrade ambari server rpms on an IOP 4.2.5 PPC cluster, replace ambari-server
> jar with the one built with the fixes, run ambari-server upgrade, verify
> ambari database has the repositories info in repo_version is updated from
> redhat7 to redhat-ppc7, verify post ambari upgrade, I could restart services
> without having ambari server threw exceptions.
>
>
> Thanks,
>
> Di Li
>
>
Re: Review Request 63877: AMBARI-22459 IOP/HDP migration NN HA restart post Ambari upgrade fails
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63877/#review191176
---
Ship it!
Ship It!
- Tim Thorpe
On Nov. 16, 2017, 1:36 p.m., Di Li wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63877/
> ---
>
> (Updated Nov. 16, 2017, 1:36 p.m.)
>
>
> Review request for Ambari, Jonathan Hurley and Tim Thorpe.
>
>
> Bugs: AMBARI-22459
> https://issues.apache.org/jira/browse/AMBARI-22459
>
>
> Repository: ambari
>
>
> Description
> ---
>
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/stack_select.py",
> line 109, in get_package_name
> package = get_packages(PACKAGE_SCOPE_STACK_SELECT, service_name,
> component_name)
> File
> "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/stack_select.py",
> line 234, in get_packages
> raise Fail("The package
> {0}
> is not supported by this version of the stack-select tool.".format(package))
> resource_management.core.exceptions.Fail: The package hadoop-hdfs-zkfc is not
> supported by this version of the stack-select tool.
>
>
> Diffs
> -
>
>
> ambari-server/src/main/resources/stacks/BigInsights/4.0/properties/stack_packages.json
> f890d66
>
>
> Diff: https://reviews.apache.org/r/63877/diff/1/
>
>
> Testing
> ---
>
> Have an IOP 4.2.5 NN HA cluster, modify the file after yum upgrade of ambari
> rpms to Ambari 2.6, then run ambari-server upgrade, then try to restart HDFS,
> verify HDFS can restart. Proceed with EU, verify EU is successful.
>
>
> Thanks,
>
> Di Li
>
>
Re: Review Request 63800: AMBARI-22431 Able to add config type if EU/RU of the same stack (minor version upgrade)
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63800/#review190961 --- Ship it! Ship It! - Tim Thorpe On Nov. 14, 2017, 4:46 p.m., Di Li wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63800/ > --- > > (Updated Nov. 14, 2017, 4:46 p.m.) > > > Review request for Ambari, Jonathan Hurley and Tim Thorpe. > > > Bugs: AMBARI-22431 > https://issues.apache.org/jira/browse/AMBARI-22431 > > > Repository: ambari > > > Description > --- > > EU/RU does not create/load default of new config types when the source stack > and target stack is the same. This JIRA introduces a new config task type for > creating/loading defaults of a new config type when stack does not change > during EU/RU. This is particularly important for EU/RU of custom services > where their stack may not change at all. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java > 66f5bf9 > > ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/CreateAndConfigureAction.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/state/stack/UpgradePack.java > 56f13ab > > ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ClusterGrouping.java > 63d0993 > > ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/CreateAndConfigureTask.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/Task.java > 6ab2fd2 > ambari-server/src/main/resources/upgrade-pack.xsd 21606bd > > ambari-server/src/test/java/org/apache/ambari/server/serveraction/upgrades/CreateAndConfigureActionTest.java > PRE-CREATION > > > Diff: https://reviews.apache.org/r/63800/diff/1/ > > > Testing > --- > > unit tests, patch Ambari 2.6 cluster with the changes, upgrade from HDP 2.6.2 > to 2.6.3, where zookeeper was modified to have zookeeper-fake that needs to > be created and configured during EU. > > > File Attachments > > > config_type_tests_hdp_2.6 > > https://reviews.apache.org/media/uploaded/files/2017/11/14/06efe434-a4fa-4b74-bede-7293bb6e1bf5__config_type_tests_hdp_2.6.zip > > > Thanks, > > Di Li > >
Re: Review Request 63640: AMBARI-22378 BI 4.2.5 repoinfo misses the redhat-ppc7 section
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63640/#review190455 --- Ship it! Ship It! - Tim Thorpe On Nov. 7, 2017, 5:23 p.m., Di Li wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63640/ > --- > > (Updated Nov. 7, 2017, 5:23 p.m.) > > > Review request for Ambari, Jonathan Hurley and Tim Thorpe. > > > Bugs: AMBARI-22378 > https://issues.apache.org/jira/browse/AMBARI-22378 > > > Repository: ambari > > > Description > --- > > BI 4.2.5 repoinfo misses the redhat-ppc7 section, where it currently only has > a redhat6 and a redhat7 section. > > > Diffs > - > > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/repos/repoinfo.xml > d1b155d > > > Diff: https://reviews.apache.org/r/63640/diff/1/ > > > Testing > --- > > manually tested it by patching an Ambari 2.6.0 cluster with changes > > > Thanks, > > Di Li > >
Re: Review Request 63513: AMBARI-22337 each service should be able to implement server actions, package them add a jar to be loaded during EU
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63513/#review189951 --- Ship it! Ship It! - Tim Thorpe On Nov. 2, 2017, 7:30 p.m., Di Li wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63513/ > --- > > (Updated Nov. 2, 2017, 7:30 p.m.) > > > Review request for Ambari, Jonathan Hurley and Tim Thorpe. > > > Bugs: AMBARI-22337 > https://issues.apache.org/jira/browse/AMBARI-22337 > > > Repository: ambari > > > Description > --- > > xsi:type="server_action" tasks defined in EU/RU upgrade pack xml files > currently can only classes from Ambari source code. This limites the server > action that custom services / services from mpacks can perform. This JIRA > proposes a way to allow each service to implement server action classes, > package them in a jar to be loaded during EU > 1. Each service can have a server_actions (support stack inheritance) such as > /var/lib/ambari-server/resources/stacks/HDP/2.5/services/ZOOKEEPER/server_actions. > > a) The dir contains jar for the server action classes. > /var/lib/ambari-server/resources/stacks/HDP/2.5/services/ZOOKEEPER/server_actions > [root@~ server_actions]# ll > total 8 > rw-rr- 1 root root 7510 Oct 30 10:49 test_full.jar > 2. upgrade pack can then invoke the server action as shown below > > > > class="org.apache.ambari.server.serveraction.upgrades.SATestWithoutService"/> > > > > > class="org.apache.ambari.server.serveraction.upgrades.SATestWithService"> > zkpr test > > > > > Diffs > - > > ambari-server/pom.xml a86acf5 > > ambari-server/src/main/java/org/apache/ambari/server/serveraction/ServerActionExecutor.java > 50e3cfe > > ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceDirectory.java > 119163e > > ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java > 3b3d52c > ambari-server/src/main/java/org/apache/ambari/server/state/ServiceInfo.java > 8fe6583 > > ambari-server/src/test/java/org/apache/ambari/server/stack/ServiceModuleTest.java > dbdd043 > > ambari-server/src/test/java/org/apache/ambari/server/stack/StackManagerExtensionTest.java > 6617b33 > > > Diff: https://reviews.apache.org/r/63513/diff/2/ > > > Testing > --- > > unit tests, patch a trunk cluster with upgrade.xml changes and java changes, > run EU, verify the external server action classes ran and were able to > provide response back to EU. > > > Thanks, > > Di Li > >
Re: Review Request 63513: AMBARI-22337 each service should be able to implement server actions, package them add a jar to be loaded during EU
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/63513/#review189923 --- Fix it, then Ship it! Remove whitespace ambari-server/src/main/java/org/apache/ambari/server/serveraction/ServerActionExecutor.java Lines 593 (patched) <https://reviews.apache.org/r/63513/#comment267149> Should this be moved to a debug message? ambari-server/src/main/java/org/apache/ambari/server/serveraction/ServerActionExecutor.java Lines 600 (patched) <https://reviews.apache.org/r/63513/#comment267148> Should this be moved to a debug message? ambari-server/src/main/java/org/apache/ambari/server/serveraction/ServerActionExecutor.java Lines 616 (patched) <https://reviews.apache.org/r/63513/#comment267150> Should this be moved to a debug message? ambari-server/src/main/java/org/apache/ambari/server/serveraction/ServerActionExecutor.java Lines 619 (patched) <https://reviews.apache.org/r/63513/#comment267151> Should this be moved to a debug message? ambari-server/src/main/java/org/apache/ambari/server/serveraction/ServerActionExecutor.java Lines 623 (patched) <https://reviews.apache.org/r/63513/#comment267152> Should this be moved to a debug message? ambari-server/src/main/java/org/apache/ambari/server/serveraction/ServerActionExecutor.java Lines 646 (patched) <https://reviews.apache.org/r/63513/#comment267153> Should this be moved to a debug message? ambari-server/src/main/java/org/apache/ambari/server/serveraction/ServerActionExecutor.java Lines 664 (patched) <https://reviews.apache.org/r/63513/#comment267154> Should this be moved to a debug message? - Tim Thorpe On Nov. 2, 2017, 3:12 p.m., Di Li wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/63513/ > --- > > (Updated Nov. 2, 2017, 3:12 p.m.) > > > Review request for Ambari, Jonathan Hurley and Tim Thorpe. > > > Bugs: AMBARI-22337 > https://issues.apache.org/jira/browse/AMBARI-22337 > > > Repository: ambari > > > Description > --- > > xsi:type="server_action" tasks defined in EU/RU upgrade pack xml files > currently can only classes from Ambari source code. This limites the server > action that custom services / services from mpacks can perform. This JIRA > proposes a way to allow each service to implement server action classes, > package them in a jar to be loaded during EU > 1. Each service can have a server_actions (support stack inheritance) such as > /var/lib/ambari-server/resources/stacks/HDP/2.5/services/ZOOKEEPER/server_actions. > > a) The dir contains jar for the server action classes. > /var/lib/ambari-server/resources/stacks/HDP/2.5/services/ZOOKEEPER/server_actions > [root@~ server_actions]# ll > total 8 > rw-rr- 1 root root 7510 Oct 30 10:49 test_full.jar > 2. upgrade pack can then invoke the server action as shown below > > > > class="org.apache.ambari.server.serveraction.upgrades.SATestWithoutService"/> > > > > > class="org.apache.ambari.server.serveraction.upgrades.SATestWithService"> > zkpr test > > > > > Diffs > - > > ambari-server/pom.xml a86acf5 > > ambari-server/src/main/java/org/apache/ambari/server/serveraction/ServerActionExecutor.java > 50e3cfe > > ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceDirectory.java > 119163e > > ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java > 3b3d52c > ambari-server/src/main/java/org/apache/ambari/server/state/ServiceInfo.java > 8fe6583 > > ambari-server/src/test/java/org/apache/ambari/server/stack/ServiceModuleTest.java > dbdd043 > > ambari-server/src/test/java/org/apache/ambari/server/stack/StackManagerExtensionTest.java > 6617b33 > > > Diff: https://reviews.apache.org/r/63513/diff/1/ > > > Testing > --- > > unit tests, patch a trunk cluster with upgrade.xml changes and java changes, > run EU, verify the external server action classes ran and were able to > provide response back to EU. > > > Thanks, > > Di Li > >
Review Request 62943: AMBARI-22220 - Should be able to switch the extension version to which a stack version is linked
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62943/ --- Review request for Ambari, Di Li, Jonathan Hurley, and Jayush Luniya. Bugs: AMBARI-0 https://issues.apache.org/jira/browse/AMBARI-0 Repository: ambari Description --- Currently the only way to switch the extension version to which a stack version is linked is to perform the following actions: Stop the extension services Reregister (delete) the extension services Unlink the old extension version Link the new extension version Add the extension services back The REST API should allow us to do an update action on the extension link. In this manner, if we were running an upgrade from HDP 2.6.1 to 2.6.3 (or something similar) which both use the HDP 2.6 stack version then we could perform the extension link switch as a manual step during the upgrade process. Diffs - ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementController.java f0f13e1 ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java b0eb8ac ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementHelper.java 0c8edfe ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ExtensionLinkResourceProvider.java 9b894ff ambari-server/src/main/java/org/apache/ambari/server/stack/ExtensionHelper.java 91dc870 ambari-server/src/test/java/org/apache/ambari/server/stack/StackManagerExtensionTest.java cef30b5 ambari-server/src/test/resources/extensions/EXT/0.2/metainfo.xml c95a20f ambari-server/src/test/resources/extensions/EXT/0.4/metainfo.xml PRE-CREATION ambari-server/src/test/resources/extensions/EXT/0.4/services/OOZIE2/metainfo.xml PRE-CREATION ambari-server/src/test/resources/extensions/EXT/0.4/services/OOZIE2/themes/broken_theme.json PRE-CREATION Diff: https://reviews.apache.org/r/62943/diff/1/ Testing --- mvn test -Dtest=StackManagerCommonServicesTest,StackManagerTest,StackManagerMiscTest,StackManagerExtensionTest Manual testing both valid and invalid extension version updates Thanks, Tim Thorpe
Re: Review Request 62778: AMBARI-22137 - Different stack versions should be able to link to different extension versions
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62778/ --- (Updated Oct. 5, 2017, 6:06 p.m.) Review request for Ambari, Di Li, Jonathan Hurley, and Jayush Luniya. Changes --- Removed commented out code Bugs: AMBARI-22137 https://issues.apache.org/jira/browse/AMBARI-22137 Repository: ambari Description --- Currently if you have extension version 0.1 linked to HDP 2.5, you can't link extension version 0.2 to HDP 2.6. Attempting to make that link will fail with an exception stating that the services included in extension version 0.2 are already in HDP 2.6 (inherited from HDP 2.5 via the 0.1 version of the extension). Diffs (updated) - ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementHelper.java 2dd6f12 ambari-server/src/main/java/org/apache/ambari/server/stack/ExtensionHelper.java 86e532a ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 6dc2b93 ambari-server/src/test/java/org/apache/ambari/server/stack/StackManagerExtensionTest.java 20c8f40 ambari-server/src/test/resources/extensions/EXT/0.2/metainfo.xml 0d37b3e ambari-server/src/test/resources/extensions/EXT/0.3/metainfo.xml d827314 ambari-server/src/test/resources/stacks_with_extensions/HDP/0.4/metainfo.xml PRE-CREATION ambari-server/src/test/resources/stacks_with_extensions/HDP/0.4/repos/repoinfo.xml PRE-CREATION ambari-server/src/test/resources/stacks_with_extensions/HDP/0.4/services/HBASE/metainfo.xml PRE-CREATION ambari-server/src/test/resources/stacks_with_extensions/HDP/0.4/services/HDFS/configuration/global.xml PRE-CREATION ambari-server/src/test/resources/stacks_with_extensions/HDP/0.4/services/HDFS/configuration/hadoop-env.xml PRE-CREATION ambari-server/src/test/resources/stacks_with_extensions/HDP/0.4/services/HDFS/configuration/hbase-site.xml PRE-CREATION ambari-server/src/test/resources/stacks_with_extensions/HDP/0.4/services/HDFS/configuration/hdfs-log4j.xml PRE-CREATION ambari-server/src/test/resources/stacks_with_extensions/HDP/0.4/services/HDFS/configuration/hdfs-site.xml PRE-CREATION ambari-server/src/test/resources/stacks_with_extensions/HDP/0.4/services/HDFS/metainfo.xml PRE-CREATION ambari-server/src/test/resources/stacks_with_extensions/HDP/0.4/services/HDFS/package/dummy-script.py PRE-CREATION ambari-server/src/test/resources/stacks_with_extensions/HDP/0.4/services/HIVE/metainfo.xml PRE-CREATION ambari-server/src/test/resources/stacks_with_extensions/HDP/0.4/services/MAPREDUCE/metainfo.xml PRE-CREATION ambari-server/src/test/resources/stacks_with_extensions/HDP/0.4/services/ZOOKEEPER/metainfo.xml PRE-CREATION Diff: https://reviews.apache.org/r/62778/diff/2/ Changes: https://reviews.apache.org/r/62778/diff/1-2/ Testing --- Manual testing with different extension and stack versions. mvn test -DskipPythonTests -Dtest=StackManagerCommonServicesTest,StackManagerTest,StackManagerMiscTest,StackManagerExtensionTest Thanks, Tim Thorpe
Re: Review Request 62787: AMBARI-22136 Enable server_action tasks defined in EU/RU upgrade pack xml files to take parameters
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62787/#review187197 --- Ship it! Ship It! - Tim Thorpe On Oct. 5, 2017, 3:17 p.m., Di Li wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/62787/ > --- > > (Updated Oct. 5, 2017, 3:17 p.m.) > > > Review request for Ambari, Jonathan Hurley and Tim Thorpe. > > > Bugs: AMBARI-22136 > https://issues.apache.org/jira/browse/AMBARI-22136 > > > Repository: ambari > > > Description > --- > > xsi:type="server_action" tasks defined in EU/RU upgrade pack xml files > currently can only take impClass name, message, scope and summary as > parameters. Since no custom parameters can pass in, each server_action is > basically a special case where all stack/service information must be > hardcoded in the Java logic. > This JIRA proposes a way to pass in custom parameters as shown below > (multiple "parameter" elements). This will allow developers to create general > cased server_action that behaves accordingly to parameters passed in. > class="org.apache.ambari.server.serveraction.upgrades.VerifyExtensionLinkAction"> > a > b > MyStack > 1.2 > > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/actionmanager/Stage.java > f466ce9 > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java > 33ce25e > > ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ServerSideActionTask.java > c593c04 > > ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/TaskParameter.java > PRE-CREATION > ambari-server/src/main/resources/upgrade-pack.xsd 21606bd > > > Diff: https://reviews.apache.org/r/62787/diff/2/ > > > Testing > --- > > Manual test on HDP upgrade from Ambari 2.5.2 to trunk (3.0.0.0) > > Updated HDP 2.6.2 express ugprade pack xml with parameter sections for > server_action tasks and verify the parameters were available in the action > class > > title="Update Target Repositories"> > > class="org.apache.ambari.server.serveraction.upgrades.UpdateDesiredRepositoryAction" > > > + a > + b > + c > > > > 04 Oct 2017 11:32:18,589 INFO [Server Action Executor Worker 54] > UpdateDesiredRepositoryAction:82 - BEGIN - Map<String, String> > commandParameters = getCommandParameters(); > 04 Oct 2017 11:32:18,590 INFO [Server Action Executor Worker 54] > UpdateDesiredRepositoryAction:84 - a: a > 04 Oct 2017 11:32:18,590 INFO [Server Action Executor Worker 54] > UpdateDesiredRepositoryAction:84 - b: b > 04 Oct 2017 11:32:18,590 INFO [Server Action Executor Worker 54] > UpdateDesiredRepositoryAction:84 - c: c > 04 Oct 2017 11:32:18,590 INFO [Server Action Executor Worker 54] > UpdateDesiredRepositoryAction:86 - END - Map<String, String> > commandParameters = getCommandParameters(); > > > Thanks, > > Di Li > >
Re: Review Request 62787: AMBARI-22136 Enable server_action tasks defined in EU/RU upgrade pack xml files to take parameters
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62787/#review187181 --- ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ExecuteHostType.java Lines 60 (patched) <https://reviews.apache.org/r/62787/#comment264090> Remove this - Tim Thorpe On Oct. 5, 2017, 1:24 p.m., Di Li wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/62787/ > --- > > (Updated Oct. 5, 2017, 1:24 p.m.) > > > Review request for Ambari, Jonathan Hurley and Tim Thorpe. > > > Bugs: AMBARI-22136 > https://issues.apache.org/jira/browse/AMBARI-22136 > > > Repository: ambari > > > Description > --- > > xsi:type="server_action" tasks defined in EU/RU upgrade pack xml files > currently can only take impClass name, message, scope and summary as > parameters. Since no custom parameters can pass in, each server_action is > basically a special case where all stack/service information must be > hardcoded in the Java logic. > This JIRA proposes a way to pass in custom parameters as shown below > (multiple "parameter" elements). This will allow developers to create general > cased server_action that behaves accordingly to parameters passed in. > class="org.apache.ambari.server.serveraction.upgrades.VerifyExtensionLinkAction"> > a > b > MyStack > 1.2 > > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/actionmanager/Stage.java > f466ce9 > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java > 33ce25e > > ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ExecuteHostType.java > aa6430a > > ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ServerSideActionTask.java > c593c04 > > ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/TaskParameter.java > PRE-CREATION > ambari-server/src/main/resources/upgrade-pack.xsd 21606bd > > > Diff: https://reviews.apache.org/r/62787/diff/1/ > > > Testing > --- > > Manual test on HDP upgrade from Ambari 2.5.2 to trunk (3.0.0.0) > > Updated HDP 2.6.2 express ugprade pack xml with parameter sections for > server_action tasks and verify the parameters were available in the action > class > > title="Update Target Repositories"> > > class="org.apache.ambari.server.serveraction.upgrades.UpdateDesiredRepositoryAction" > > > + a > + b > + c > > > > 04 Oct 2017 11:32:18,589 INFO [Server Action Executor Worker 54] > UpdateDesiredRepositoryAction:82 - BEGIN - Map<String, String> > commandParameters = getCommandParameters(); > 04 Oct 2017 11:32:18,590 INFO [Server Action Executor Worker 54] > UpdateDesiredRepositoryAction:84 - a: a > 04 Oct 2017 11:32:18,590 INFO [Server Action Executor Worker 54] > UpdateDesiredRepositoryAction:84 - b: b > 04 Oct 2017 11:32:18,590 INFO [Server Action Executor Worker 54] > UpdateDesiredRepositoryAction:84 - c: c > 04 Oct 2017 11:32:18,590 INFO [Server Action Executor Worker 54] > UpdateDesiredRepositoryAction:86 - END - Map<String, String> > commandParameters = getCommandParameters(); > > > Thanks, > > Di Li > >
Review Request 62778: AMBARI-22137 - Different stack versions should be able to link to different extension versions
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/62778/ --- Review request for Ambari, Di Li, Jonathan Hurley, and Jayush Luniya. Bugs: AMBARI-22137 https://issues.apache.org/jira/browse/AMBARI-22137 Repository: ambari Description --- Currently if you have extension version 0.1 linked to HDP 2.5, you can't link extension version 0.2 to HDP 2.6. Attempting to make that link will fail with an exception stating that the services included in extension version 0.2 are already in HDP 2.6 (inherited from HDP 2.5 via the 0.1 version of the extension). Diffs - ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementHelper.java 2dd6f12 ambari-server/src/main/java/org/apache/ambari/server/stack/ExtensionHelper.java 86e532a ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 6dc2b93 ambari-server/src/test/java/org/apache/ambari/server/stack/StackManagerExtensionTest.java 20c8f40 ambari-server/src/test/resources/extensions/EXT/0.2/metainfo.xml 0d37b3e ambari-server/src/test/resources/extensions/EXT/0.3/metainfo.xml d827314 ambari-server/src/test/resources/stacks_with_extensions/HDP/0.4/metainfo.xml PRE-CREATION ambari-server/src/test/resources/stacks_with_extensions/HDP/0.4/repos/repoinfo.xml PRE-CREATION ambari-server/src/test/resources/stacks_with_extensions/HDP/0.4/services/HBASE/metainfo.xml PRE-CREATION ambari-server/src/test/resources/stacks_with_extensions/HDP/0.4/services/HDFS/configuration/global.xml PRE-CREATION ambari-server/src/test/resources/stacks_with_extensions/HDP/0.4/services/HDFS/configuration/hadoop-env.xml PRE-CREATION ambari-server/src/test/resources/stacks_with_extensions/HDP/0.4/services/HDFS/configuration/hbase-site.xml PRE-CREATION ambari-server/src/test/resources/stacks_with_extensions/HDP/0.4/services/HDFS/configuration/hdfs-log4j.xml PRE-CREATION ambari-server/src/test/resources/stacks_with_extensions/HDP/0.4/services/HDFS/configuration/hdfs-site.xml PRE-CREATION ambari-server/src/test/resources/stacks_with_extensions/HDP/0.4/services/HDFS/metainfo.xml PRE-CREATION ambari-server/src/test/resources/stacks_with_extensions/HDP/0.4/services/HDFS/package/dummy-script.py PRE-CREATION ambari-server/src/test/resources/stacks_with_extensions/HDP/0.4/services/HIVE/metainfo.xml PRE-CREATION ambari-server/src/test/resources/stacks_with_extensions/HDP/0.4/services/MAPREDUCE/metainfo.xml PRE-CREATION ambari-server/src/test/resources/stacks_with_extensions/HDP/0.4/services/ZOOKEEPER/metainfo.xml PRE-CREATION Diff: https://reviews.apache.org/r/62778/diff/1/ Testing --- Manual testing with different extension and stack versions. mvn test -DskipPythonTests -Dtest=StackManagerCommonServicesTest,StackManagerTest,StackManagerMiscTest,StackManagerExtensionTest Thanks, Tim Thorpe
Re: Review Request 61948: AMBARI-21830 PowerPC - IOP/HDP migration, service restart fails post Ambari upgrade
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/61948/#review183937 --- Ship it! Ship It! - Tim Thorpe On Aug. 28, 2017, 3:35 p.m., Di Li wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/61948/ > --- > > (Updated Aug. 28, 2017, 3:35 p.m.) > > > Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Sid Wagle, > and Tim Thorpe. > > > Bugs: AMBARI-21830 > https://issues.apache.org/jira/browse/AMBARI-21830 > > > Repository: ambari > > > Description > --- > > BI 4.2.5 repoinfo.xml shipped by Ambari 2.5.2 only includes repo urls for x > no p "redhat-ppc7", so restarting services ( with staled configs) failed post > Ambari upgrade. > > > Diffs > - > > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/repos/repoinfo.xml > d1b155d > > > Diff: https://reviews.apache.org/r/61948/diff/1/ > > > Testing > --- > > the urls included in the repoinfo.xml should not matter, the only thing > required by Ambari for restarting services ( generating the repo_info data in > the command-[taskid].json) is the repo id "redhat-ppc7" > > patched a cluster with the new section in repoinfo.xml, restart Ambari ( > already upgraded), veify now can restart HDFS where it can install the > Hadoop client > > > Thanks, > > Di Li > >
Re: Review Request 61861: Post Ambari upgrade on a migration cluster Oozie failed to restart when LZO compression is enabled
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/61861/#review183663 --- Ship it! Ship It! - Tim Thorpe On Aug. 23, 2017, 8:25 p.m., Sid Wagle wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/61861/ > --- > > (Updated Aug. 23, 2017, 8:25 p.m.) > > > Review request for Ambari, Di Li, Jonathan Hurley, Robert Levas, and Sumit > Mohanty. > > > Bugs: AMBARI-21794 > https://issues.apache.org/jira/browse/AMBARI-21794 > > > Repository: ambari > > > Description > --- > > Restart Oozie fails due lzo package installation failure. The rpm package > name is different for IOP stack vs HDP. > > > Diffs > - > > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/alerts/alert_check_oozie_server.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/files/oozieSmoke2.sh > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/files/prepareOozieHdfsDirectories.sh > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/files/wrap_ooziedb.sh > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/check_oozie_server_status.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/oozie.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/oozie_client.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/oozie_server.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/oozie_server_upgrade.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/oozie_service.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/params.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/params_linux.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/params_windows.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/service_check.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/scripts/status_params.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/templates/adminusers.txt.j2 > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/templates/oozie-log4j.properties.j2 > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/templates/oozie.conf.j2 > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/OOZIE/package/templates/zkmigrator_jaas.conf.j2 > PRE-CREATION > > > Diff: https://reviews.apache.org/r/61861/diff/1/ > > > Testing > --- > > Manual verification pending. > > > Thanks, > > Sid Wagle > >
Re: Review Request 61813: Storm Service should not have ranger 'ranger-admin-site' and 'ranger-ugsync-site' dependencies
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/61813/#review183480 --- Ship it! Ship It! - Tim Thorpe On Aug. 22, 2017, 1:42 p.m., Jay SenSharma wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/61813/ > --- > > (Updated Aug. 22, 2017, 1:42 p.m.) > > > Review request for Ambari, Jayush Luniya, Juanjo Marron, and Tim Thorpe. > > > Bugs: AMBARI-21779 > https://issues.apache.org/jira/browse/AMBARI-21779 > > > Repository: ambari > > > Description > --- > > Storm Service should not include the following "" > > ranger-ugsync-site > ranger-admin-site > > Else making any configuration change in Ranger service (for example: "Ranger > User Info") will ask Storm service to be restarted. > > Looks like as part of JIRA "AMBARI-16250" this dependency was added in bulk > by mistake. > > > Diffs > - > > ambari-server/src/main/resources/common-services/STORM/0.9.1/metainfo.xml > 0c62c1a > > ambari-server/src/main/resources/common-services/STORM/1.0.1.3.0/metainfo.xml > 1bc23e4 > ambari-server/src/main/resources/common-services/STORM/1.0.1/metainfo.xml > 084eac2 > ambari-server/src/main/resources/common-services/STORM/1.1.0/metainfo.xml > 94f5ca3 > > > Diff: https://reviews.apache.org/r/61813/diff/1/ > > > Testing > --- > > . > > > Thanks, > > Jay SenSharma > >
Re: Review Request 61415: AMBARI-21654 Post Ambari upgrade, BI value add 4.2 services should still be able to restart
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/61415/#review182199 --- Ship it! Ship It! - Tim Thorpe On Aug. 3, 2017, 8:20 p.m., Di Li wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/61415/ > --- > > (Updated Aug. 3, 2017, 8:20 p.m.) > > > Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Sid Wagle, > and Tim Thorpe. > > > Bugs: AMBARI-21654 > https://issues.apache.org/jira/browse/AMBARI-21654 > > > Repository: ambari > > > Description > --- > > Ensure backword compatibility for older BI value add services. These services > do not get updated service definitions so their Python API references are old. > > > Diffs > - > > > ambari-common/src/main/python/resource_management/libraries/functions/version.py > 7dfa6b8 > > > Diff: https://reviews.apache.org/r/61415/diff/1/ > > > Testing > --- > > patch a cluster, now BI value add services (service definitions got copied > over automatically by the Ambari upgrade process, their service defintions > are NOT part of the stack shipped in Ambari) can restart now. > > > Thanks, > > Di Li > >
Re: Review Request 61415: AMBARI-21654 Post Ambari upgrade, BI value add 4.2 services should still be able to restart
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/61415/#review182157 --- Ship it! Ship It! - Tim Thorpe On Aug. 3, 2017, 8:20 p.m., Di Li wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/61415/ > --- > > (Updated Aug. 3, 2017, 8:20 p.m.) > > > Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Sid Wagle, > and Tim Thorpe. > > > Bugs: AMBARI-21654 > https://issues.apache.org/jira/browse/AMBARI-21654 > > > Repository: ambari > > > Description > --- > > Ensure backword compatibility for older BI value add services. These services > do not get updated service definitions so their Python API references are old. > > > Diffs > - > > > ambari-common/src/main/python/resource_management/libraries/functions/version.py > 7dfa6b8 > > > Diff: https://reviews.apache.org/r/61415/diff/1/ > > > Testing > --- > > patch a cluster, now BI value add services (service definitions got copied > over automatically by the Ambari upgrade process, their service defintions > are NOT part of the stack shipped in Ambari) can restart now. > > > Thanks, > > Di Li > >
Re: Review Request 61403: AMBARI-21554. Regenerating keytabs fails in Stack Advisor after upgrading Ambari
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/61403/#review182101 --- Ship it! Ship It! - Tim Thorpe On Aug. 3, 2017, 12:54 p.m., Attila Doroszlai wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/61403/ > --- > > (Updated Aug. 3, 2017, 12:54 p.m.) > > > Review request for Ambari, Alejandro Fernandez, Di Li, Eric Yang, Jonathan > Hurley, Sid Wagle, and Tim Thorpe. > > > Bugs: AMBARI-21554 > https://issues.apache.org/jira/browse/AMBARI-21554 > > > Repository: ambari > > > Description > --- > > Fix: Add the custom directory where `stack_advisor_25` etc. are defined to > the Python path. > > Code taken from IBM Ambari 2.2. Added some explanatory comment. > > > Diffs > - > > ambari-server/src/main/resources/scripts/stack_advisor.py > 66ab248d071a197eaf12057e10a8f315b1aebd9b > > > Diff: https://reviews.apache.org/r/61403/diff/1/ > > > Testing > --- > > Tested manually according to steps to reproduce in the bug description. > > > Thanks, > > Attila Doroszlai > >
Re: Review Request 61377: Hive CLI does not work after upgrade
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/61377/#review181990 --- Ship it! Ship It! - Tim Thorpe On Aug. 2, 2017, 1:21 p.m., Attila Doroszlai wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/61377/ > --- > > (Updated Aug. 2, 2017, 1:21 p.m.) > > > Review request for Ambari, Alejandro Fernandez, Di Li, Jonathan Hurley, Sumit > Mohanty, Sid Wagle, and Tim Thorpe. > > > Repository: ambari > > > Description > --- > > During upgrade `hive.execution.engine` is set to `tez`, even though Tez is > not installed in the cluster. > > Change back `hive.execution.engine` to `mr`. > > > Diffs > - > > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/config-upgrade.xml > 52a6cdcd4b3ee1c9691fdb76da84cf23057e30f1 > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/nonrolling-upgrade-to-hdp-2.6.xml > 1fe0382932dd44d40e5e61c2d40e5c26482abe24 > > ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/config-upgrade.xml > 17e80b5045ae4f790cc172e8a5b6ce4b33d344d5 > > ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/nonrolling-upgrade-to-hdp-2.6.xml > 55736cf613a767e7c5d2786d4db1774ccc1854d7 > > > Diff: https://reviews.apache.org/r/61377/diff/1/ > > > Testing > --- > > Tested stack upgrade on local cluster from both 4.2 and 4.2.5. > > > Thanks, > > Attila Doroszlai > >
Re: Review Request 61371: AMBARI-21624. HDFS restart failed post Ambari upgrade when lzo compression was enabled
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/61371/#review181982 --- Ship it! Ship It! - Tim Thorpe On Aug. 2, 2017, 1:12 p.m., Attila Doroszlai wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/61371/ > --- > > (Updated Aug. 2, 2017, 1:12 p.m.) > > > Review request for Ambari, Di Li, Sid Wagle, and Tim Thorpe. > > > Bugs: AMBARI-21624 > https://issues.apache.org/jira/browse/AMBARI-21624 > > > Repository: ambari > > > Description > --- > > The list of lzo packages for BI 4.2 is hard-coded in Ambari 2.5.2's stack > definition. Use the same list for BI 4.2.5. > > * Copied HDFS `package` from IBM Ambari 2.4.2's `common-services` > * Replaced `get_lzo_packages` call with list of packages. > * Also needed to replace `Script.get_force_https_protocol` calls due to the > change made in AMBARI-20857. > > > Diffs > - > > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/alerts/alert_checkpoint_time.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/alerts/alert_datanode_unmounted_data_dir.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/alerts/alert_ha_namenode_health.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/alerts/alert_metrics_deviation.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/alerts/alert_upgrade_finalized.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/files/checkWebUI.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/scripts/__init__.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/scripts/balancer-emulator/hdfs-command.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/scripts/datanode.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/scripts/datanode_upgrade.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/scripts/hdfs.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/scripts/hdfs_client.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/scripts/hdfs_datanode.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/scripts/hdfs_namenode.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/scripts/hdfs_nfsgateway.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/scripts/hdfs_rebalance.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/scripts/hdfs_snamenode.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/scripts/install_params.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/scripts/journalnode.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/scripts/journalnode_upgrade.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/scripts/namenode.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/scripts/namenode_ha_state.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/scripts/namenode_upgrade.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/scripts/nfsgateway.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/scripts/params.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/scripts/params_linux.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.
Re: Review Request 61371: AMBARI-21624. HDFS restart failed post Ambari upgrade when lzo compression was enabled
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/61371/#review181965 --- Ship it! Ship It! - Tim Thorpe On Aug. 2, 2017, 9:13 a.m., Attila Doroszlai wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/61371/ > --- > > (Updated Aug. 2, 2017, 9:13 a.m.) > > > Review request for Ambari, Di Li, Sid Wagle, and Tim Thorpe. > > > Bugs: AMBARI-21624 > https://issues.apache.org/jira/browse/AMBARI-21624 > > > Repository: ambari > > > Description > --- > > The list of lzo packages for BI 4.2 is hard-coded in Ambari 2.5.2's stack > definition. Use the same list for BI 4.2.5. > > * Copied HDFS `package` from IBM Ambari 2.4.2's `common-services` > * Replaced `get_lzo_packages` call with list of packages. > * Also needed to replace `Script.get_force_https_protocol` calls due to the > change made in AMBARI-20857. > > > Diffs > - > > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/alerts/alert_checkpoint_time.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/alerts/alert_datanode_unmounted_data_dir.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/alerts/alert_ha_namenode_health.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/alerts/alert_metrics_deviation.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/alerts/alert_upgrade_finalized.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/files/checkWebUI.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/scripts/__init__.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/scripts/balancer-emulator/balancer-err.log > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/scripts/balancer-emulator/balancer.log > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/scripts/balancer-emulator/hdfs-command.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/scripts/datanode.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/scripts/datanode_upgrade.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/scripts/hdfs.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/scripts/hdfs_client.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/scripts/hdfs_datanode.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/scripts/hdfs_namenode.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/scripts/hdfs_nfsgateway.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/scripts/hdfs_rebalance.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/scripts/hdfs_snamenode.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/scripts/install_params.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/scripts/journalnode.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/scripts/journalnode_upgrade.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/scripts/namenode.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/scripts/namenode_ha_state.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/scripts/namenode_upgrade.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/scripts/nfsgateway.py > PRE-CREATION > > ambari-server
Re: Review Request 61283: AMBARI-21627. Cross-stack upgrade from IOP to HDP, ranger audit properties need to be deleted
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61283/#review181964
---
Ship it!
Are these needed in BigInsights 4.2.5 as well?
- Tim Thorpe
On Aug. 1, 2017, 10:34 p.m., Alejandro Fernandez wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61283/
> ---
>
> (Updated Aug. 1, 2017, 10:34 p.m.)
>
>
> Review request for Ambari, Dmytro Grinenko, Di Li, Sumit Mohanty, Sid Wagle,
> and Tim Thorpe.
>
>
> Bugs: AMBARI-21627
> https://issues.apache.org/jira/browse/AMBARI-21627
>
>
> Repository: ambari
>
>
> Description
> ---
>
> STR:
>
> * Install BigInsights 4.2.0 with Ranger, Knox, Kafka, Hive, HBase, YARN, HDFS
> * Upgrade Ambari to 2.5.2
> * Install HDP 2.6
> * Perform EU to HDP 2.6
>
>
> All of the config types like ranger-*-audit.xml have configs that need to be
> deleted that begin with xasecure.audit.destination.db.*
> E.g.,
> ```
> xasecure.audit.destination.db: "true",
> xasecure.audit.destination.db.batch.filespool.dir:
> "/var/log/hbase/audit/db/spool",
> xasecure.audit.destination.db.jdbc.driver: "{{jdbc_driver}}",
> xasecure.audit.destination.db.jdbc.url: "{{audit_jdbc_url}}",
> xasecure.audit.destination.db.password:
> "SECRET:ranger-hbase-audit:3:xasecure.audit.destination.db.password",
> xasecure.audit.destination.db.user: "{{xa_audit_db_user}}",
> ```
>
>
> Diffs
> -
>
>
> ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/config-upgrade.xml
> 17e80b5
>
> ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/nonrolling-upgrade-to-hdp-2.6.xml
> 55736cf
>
>
> Diff: https://reviews.apache.org/r/61283/diff/1/
>
>
> Testing
> ---
>
> Inspected manually
>
>
> Thanks,
>
> Alejandro Fernandez
>
>
Re: Review Request 61371: AMBARI-21624. HDFS restart failed post Ambari upgrade when lzo compression was enabled
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/61371/#review181963 --- ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/scripts/balancer-emulator/balancer-err.log Lines 1 (patched) <https://reviews.apache.org/r/61371/#comment257801> This should either be removed to avoid the license file issue or added to the exception list in the pom. ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/scripts/balancer-emulator/balancer.log Lines 1 (patched) <https://reviews.apache.org/r/61371/#comment257800> This should either be removed to avoid the license file issue or added to the exception list in the pom. - Tim Thorpe On Aug. 2, 2017, 9:13 a.m., Attila Doroszlai wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/61371/ > --- > > (Updated Aug. 2, 2017, 9:13 a.m.) > > > Review request for Ambari, Di Li, Sid Wagle, and Tim Thorpe. > > > Bugs: AMBARI-21624 > https://issues.apache.org/jira/browse/AMBARI-21624 > > > Repository: ambari > > > Description > --- > > The list of lzo packages for BI 4.2 is hard-coded in Ambari 2.5.2's stack > definition. Use the same list for BI 4.2.5. > > * Copied HDFS `package` from IBM Ambari 2.4.2's `common-services` > * Replaced `get_lzo_packages` call with list of packages. > * Also needed to replace `Script.get_force_https_protocol` calls due to the > change made in AMBARI-20857. > > > Diffs > - > > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/alerts/alert_checkpoint_time.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/alerts/alert_datanode_unmounted_data_dir.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/alerts/alert_ha_namenode_health.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/alerts/alert_metrics_deviation.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/alerts/alert_upgrade_finalized.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/files/checkWebUI.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/scripts/__init__.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/scripts/balancer-emulator/balancer-err.log > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/scripts/balancer-emulator/balancer.log > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/scripts/balancer-emulator/hdfs-command.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/scripts/datanode.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/scripts/datanode_upgrade.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/scripts/hdfs.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/scripts/hdfs_client.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/scripts/hdfs_datanode.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/scripts/hdfs_namenode.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/scripts/hdfs_nfsgateway.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/scripts/hdfs_rebalance.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/scripts/hdfs_snamenode.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/scripts/install_params.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/scripts/journalnode.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HDFS/package/scripts/journalnode_upgra
Re: Review Request 61239: AMBARI-21611 Failed to install packages before EU on powerpc
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61239/#review181826
---
Ship it!
Ship It!
- Tim Thorpe
On July 31, 2017, 6:20 p.m., Di Li wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61239/
> ---
>
> (Updated July 31, 2017, 6:20 p.m.)
>
>
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Sid Wagle,
> and Tim Thorpe.
>
>
> Bugs: AMBARI-21611
> https://issues.apache.org/jira/browse/AMBARI-21611
>
>
> Repository: ambari
>
>
> Description
> ---
>
> Error message: Repositories for os type redhat-ppc7 are not defined.
> This is because:
> AmbariAgent reports os type (to be stored in the Hosts table) as redhat-ppc7.
> "-ppc" is a hardcoded post fix attached to the os type in os_check.py
> """
> os_check.py (AMBARI-21054)
> if _is_powerpc():
> operatingSystem += '-ppc'
> """
> When Ambari server backend generates the
> {os, repo}
> map, it uses os information from repo_versoin as the key . But later on uses
> os info from the Hosts table to query the map, thus the mismatch (redhat-ppc7
> vs redhat7).
> The repo_version table repository column however only uses redhat6/7 and does
> not distinguish ppc vs x86 ( nor it should )
>
>
> Diffs
> -
>
>
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProvider.java
> 633fe8c
>
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProviderTest.java
> 7018245
>
>
> Diff: https://reviews.apache.org/r/61239/diff/2/
>
>
> Testing
> ---
>
> one IOP ppc one IOP x86, upgrade Ambari, patch with server jar, run install
> packages after registering HDP 2.6.2 stack. verify the error did not appear
> any more for ppc, both ppc and x86 cluster were able to finish the install
> package step successfully
>
>
> Thanks,
>
> Di Li
>
>
Re: Review Request 61050: AMBARI-21555 Hive restart fails to restart MySQL after Ambari upgrade against IOP 4.2.5
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/61050/#review181145 --- Ship it! Ship It! - Tim Thorpe On July 21, 2017, 8:18 p.m., Di Li wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/61050/ > --- > > (Updated July 21, 2017, 8:18 p.m.) > > > Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Sid Wagle, > and Tim Thorpe. > > > Bugs: AMBARI-21555 > https://issues.apache.org/jira/browse/AMBARI-21555 > > > Repository: ambari > > > Description > --- > > IOP uses MariaDB instead of MySQL when a cluster is on RedHat 7. HDP > 2.6/Ambari 2.5.2 is yet to support MariaDB. A special flag is introduced to > switch daemon type for migrated clusters. Currently the flag is added during > EU (AMBARI-21351) to handle both IOP 4.2 and 4.2.5. We also need this flag to > be added during Ambari upgrade for IOP 4.2.5 to restart MySQL ( IOP 4.2 is > self-contained, thus not need this flag for restarting MySQL) > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog252.java > ea1b034 > > > Diff: https://reviews.apache.org/r/61050/diff/1/ > > > Testing > --- > > patch cluster with ambari-server jar, run ambari-server upgrade, verify > mariadb_redhat_support=true got added to hive-env post ambari upgrade. > > > Thanks, > > Di Li > >
Re: Review Request 61050: AMBARI-21555 Hive restart fails to restart MySQL after Ambari upgrade against IOP 4.2.5
> On July 21, 2017, 8:25 p.m., Sumit Mohanty wrote: > > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog252.java > > Lines 310 (patched) > > <https://reviews.apache.org/r/61050/diff/1/?file=1780886#file1780886line310> > > > > Should this have been done as part of the upgrade pack instead of > > Ambari upgrade? Hi Sumit, Currently the flag is added during EU (AMBARI-21351) to handle both IOP 4.2 and 4.2.5. We also need this flag to be added during Ambari upgrade for IOP 4.2.5 to restart MySQL prior to the EU (IOP 4.2 is self-contained, thus not need this flag for restarting MySQL) - Tim --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/61050/#review181141 --- On July 21, 2017, 8:18 p.m., Di Li wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/61050/ > --- > > (Updated July 21, 2017, 8:18 p.m.) > > > Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Sid Wagle, > and Tim Thorpe. > > > Bugs: AMBARI-21555 > https://issues.apache.org/jira/browse/AMBARI-21555 > > > Repository: ambari > > > Description > --- > > IOP uses MariaDB instead of MySQL when a cluster is on RedHat 7. HDP > 2.6/Ambari 2.5.2 is yet to support MariaDB. A special flag is introduced to > switch daemon type for migrated clusters. Currently the flag is added during > EU (AMBARI-21351) to handle both IOP 4.2 and 4.2.5. We also need this flag to > be added during Ambari upgrade for IOP 4.2.5 to restart MySQL ( IOP 4.2 is > self-contained, thus not need this flag for restarting MySQL) > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog252.java > ea1b034 > > > Diff: https://reviews.apache.org/r/61050/diff/1/ > > > Testing > --- > > patch cluster with ambari-server jar, run ambari-server upgrade, verify > mariadb_redhat_support=true got added to hive-env post ambari upgrade. > > > Thanks, > > Di Li > >
Re: Review Request 61049: AMBARI-21553 Oozie server failed to restart post IOP 4.2 migration
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/61049/#review181139 --- Ship it! Ship It! - Tim Thorpe On July 21, 2017, 7:59 p.m., Di Li wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/61049/ > --- > > (Updated July 21, 2017, 7:59 p.m.) > > > Review request for Ambari, Dmytro Grinenko, Di Li, Dmitro Lisnichenko, > Jonathan Hurley, Sumit Mohanty, Sid Wagle, and Tim Thorpe. > > > Bugs: AMBARI-21553 > https://issues.apache.org/jira/browse/AMBARI-21553 > > > Repository: ambari > > > Description > --- > > This is after the manual workaround of installing extjs from HDP-UTIL to > replace the one from IOP-UTIL - AMBARI-21463 > > This is due to the deprecated -Xms oozie_initial_heapsize property in > oozie-env > > > Diffs > - > > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/config-upgrade.xml > c8549b3 > > ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/config-upgrade.xml > bf6dd6b > > > Diff: https://reviews.apache.org/r/61049/diff/1/ > > > Testing > --- > > updated the config manually, oozie server restarted successfully > > > Thanks, > > Di Li > >
Re: Review Request 61039: AMBARI-21550 PHOENIX_QUERY_SERVER not upgraded in EU
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/61039/#review181125 --- Ship it! Ship It! - Tim Thorpe On July 21, 2017, 4:21 p.m., Di Li wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/61039/ > --- > > (Updated July 21, 2017, 4:21 p.m.) > > > Review request for Ambari, Dmytro Grinenko, Di Li, Dmitro Lisnichenko, > Jonathan Hurley, Sumit Mohanty, Sid Wagle, and Tim Thorpe. > > > Bugs: AMBARI-21550 > https://issues.apache.org/jira/browse/AMBARI-21550 > > > Repository: ambari > > > Description > --- > > PHOENIX_QUERY_SERVER is missing from the upgrade pack xml file, thus the > following error during finalize precheck. > """ > The following components were found to have version mismatches. Finalize will > not complete successfully: > raze3.fyre.ibm.com: HBASE/PHOENIX_QUERY_SERVER reports 4.2.5.0- > raze4.fyre.ibm.com: HBASE/PHOENIX_QUERY_SERVER reports 4.2.5.0- > raze2.fyre.ibm.com: HBASE/PHOENIX_QUERY_SERVER reports 4.2.5.0- > raze1.fyre.ibm.com: HBASE/PHOENIX_QUERY_SERVER reports 4.2.5.0- > """ > > > Diffs > - > > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/nonrolling-upgrade-to-hdp-2.6.xml > 57e731e > > > Diff: https://reviews.apache.org/r/61039/diff/1/ > > > Testing > --- > > to be tested in integrated tests > > > Thanks, > > Di Li > >
Re: Review Request 61000: AMBARI-21527 Restart of MR2 History Server failed due to wrong NameNode RPC address
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/61000/#review181059 --- Ship it! Ship It! - Tim Thorpe On July 20, 2017, 4:54 p.m., Di Li wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/61000/ > --- > > (Updated July 20, 2017, 4:54 p.m.) > > > Review request for Ambari, Attila Doroszlai, Alejandro Fernandez, Jonathan > Hurley, Sid Wagle, and Tim Thorpe. > > > Bugs: AMBARI-21527 > https://issues.apache.org/jira/browse/AMBARI-21527 > > > Repository: ambari > > > Description > --- > > dfs.namenode.rpc-address gets merged in during EU with default value > "localhost:8020". Both the host and port could be wrong and cause issues > during HDFS restart. NN restart for kerberos cluster and remote DN restart > for non secured cluster. > > This is because HDFS Py script looks for that property first and uses it if > it exists. This logic and the fact that the property (seems unnecessarily) > merged in during EU with "localhost:8020" as the value was what caused the > issues in our tests. > > > Diffs > - > > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/config-upgrade.xml > e476d57 > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/nonrolling-upgrade-to-hdp-2.6.xml > cbd0550 > > ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/config-upgrade.xml > dada6e2 > > ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/nonrolling-upgrade-to-hdp-2.6.xml > 3ea20ed > > > Diff: https://reviews.apache.org/r/61000/diff/1/ > > > Testing > --- > > tested as part of the EU > > > Thanks, > > Di Li > >
Re: Review Request 60986: AMBARI-21528. Zookeeper server has incorrect memory setting, missing m in Xmx value
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60986/#review181035
---
Ship it!
Ship It!
- Tim Thorpe
On July 20, 2017, 12:13 a.m., Alejandro Fernandez wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60986/
> ---
>
> (Updated July 20, 2017, 12:13 a.m.)
>
>
> Review request for Ambari, Di Li, Jonathan Hurley, Sumit Mohanty, Sid Wagle,
> and Tim Thorpe.
>
>
> Bugs: AMBARI-21528
> https://issues.apache.org/jira/browse/AMBARI-21528
>
>
> Repository: ambari
>
>
> Description
> ---
>
> Repro Steps:
>
> * Installed BI 4.2.0 cluster on IBM Ambari 2.2.2 with Zookeeper
> * Upgraded Ambari to 2.5.2.0-146
> * Registered HDP 2.6.2.0 repo, installed packages
> * Ran service checks
> * Started Express Upgrade
>
> Result: _Service Check ZooKeeper_ step failed with {{KeeperErrorCode =
> ConnectionLoss for /zk_smoketest}}
>
> This was caused by Zookeeper dying immediately during restart:
> ```
> Error occurred during initialization of VM
> Too small initial heap
> ```
>
> Before EU
> ```
> export JAVA_HOME=/usr/jdk64/java-1.8.0-openjdk-1.8.0.77-0.b03.el7_2.x86_64
> export ZOOKEEPER_HOME=/usr/iop/current/zookeeper-server
> export ZOO_LOG_DIR=/var/log/zookeeper
> export ZOOPIDFILE=/var/run/zookeeper/zookeeper_server.pid
> export SERVER_JVMFLAGS=-Xmx1024m
> export JAVA=$JAVA_HOME/bin/java
> export CLASSPATH=$CLASSPATH:/usr/share/zookeeper/*
> ```
>
> After EU
> ```
> export JAVA_HOME=/usr/jdk64/java-1.8.0-openjdk-1.8.0.77-0.b03.el7_2.x86_64
> export ZOOKEEPER_HOME=/usr/hdp/current/zookeeper-client
> export ZOO_LOG_DIR=/var/log/zookeeper
> export ZOOPIDFILE=/var/run/zookeeper/zookeeper_server.pid
> export SERVER_JVMFLAGS=-Xmx1024
> export JAVA=$JAVA_HOME/bin/java
> ```
>
> Note missing "m" in memory setting.
>
> zookeeper-env template contains,
> ```
> export SERVER_JVMFLAGS={{zk_server_heapsize}}
> ```
>
> In this cluster, zookeeper-env contains,
> zk_server_heapsize: "1024"
>
> While the params_linux.py file has some inconsistencies with appending the
> letter "m".
> ```
> zk_server_heapsize_value =
> str(default('configurations/zookeeper-env/zk_server_heapsize', "1024m"))
> zk_server_heapsize = format("-Xmx{zk_server_heapsize_value}")
> ```
>
> Instead, it should be,
> ```
> zk_server_heapsize_value =
> str(default('configurations/zookeeper-env/zk_server_heapsize', "1024"))
> zk_server_heapsize_value = zk_server_heapsize_value.strip()
> if len(zk_server_heapsize_value) > 0 and not
> zk_server_heapsize_value[-1].isdigit():
> zk_server_heapsize_value = zk_server_heapsize_value + "m"
> zk_server_heapsize = format("-Xmx{zk_server_heapsize_value}")
> ```
>
>
> Diffs
> -
>
>
> ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5/package/scripts/params_linux.py
> 0780d2e
>
>
> Diff: https://reviews.apache.org/r/60986/diff/2/
>
>
> Testing
> ---
>
> Python unit tests passed,
>
> --
> Total run:1161
> Total errors:0
> Total failures:0
> OK
>
>
> Thanks,
>
> Alejandro Fernandez
>
>
Review Request 60945: AMBARI-21509 - BigInsights/4.2 upgrade to HDP uses function take_snapshot but that doesn't exist in 4.2's hbase_upgrade.py
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/60945/ --- Review request for Ambari, Alejandro Fernandez, Di Li, and Jonathan Hurley. Bugs: AMBARI-21509 https://issues.apache.org/jira/browse/AMBARI-21509 Repository: ambari Description --- scripts/hbase_upgrade.py take_snapshot def snapshot(self, env): Diffs - ambari-server/src/main/resources/stacks/BigInsights/4.2/services/HBASE/package/scripts/hbase_upgrade.py 610f527 Diff: https://reviews.apache.org/r/60945/diff/1/ Testing --- modify the script and retry the upgrade from where it failed when the function wasn't found Thanks, Tim Thorpe
Re: Review Request 60866: AMBARI-21477: Remove Falcon proxy entries from Knox kerberos.json
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/60866/#review180526 --- Ship it! Ship It! - Tim Thorpe On July 14, 2017, 1:34 p.m., Di Li wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/60866/ > --- > > (Updated July 14, 2017, 1:34 p.m.) > > > Review request for Ambari, Dmytro Grinenko, Di Li, Dmitro Lisnichenko, > Jonathan Hurley, Sumit Mohanty, Sid Wagle, and Tim Thorpe. > > > Bugs: AMBARI-21477 > https://issues.apache.org/jira/browse/AMBARI-21477 > > > Repository: ambari > > > Description > --- > > Bi 4.2 stack Knox kerberos.json has falcon proxy entries that are not needed > anymore. remove them. > > > Diffs > - > > > ambari-server/src/main/resources/stacks/BigInsights/4.2/services/KNOX/kerberos.json > 8ee2acc > > > Diff: https://reviews.apache.org/r/60866/diff/1/ > > > Testing > --- > > to be tested in integration tests. > > > Thanks, > > Di Li > >
Re: Review Request 60814: AMBARI-21461 Spark thrift server restart fails during migration EU
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/60814/#review180426 --- Ship it! Ship It! - Tim Thorpe On July 12, 2017, 9 p.m., Di Li wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/60814/ > --- > > (Updated July 12, 2017, 9 p.m.) > > > Review request for Ambari, Dmytro Grinenko, Di Li, Dmitro Lisnichenko, > Jonathan Hurley, Sumit Mohanty, Sid Wagle, and Tim Thorpe. > > > Bugs: AMBARI-21461 > https://issues.apache.org/jira/browse/AMBARI-21461 > > > Repository: ambari > > > Description > --- > > Spark in IOP 4.2 BigInsights stack uses script name "thrift_server.py" > instead of "spark_thrift_server.py" used by the common service Spark and the > IOP 4.2.5 BigInsights stack. > This seemed to have caused the following error during migration EU where > Spark thrift server failed to restart. > """ > Caught an exception while executing custom service command: 'ambari_agent.AgentException.AgentException'>: 'Script > /var/lib/ambari-agent/cache/common-services/SPARK/1.2.1/package/scripts/thrift_server.py > does not exist'; 'Script > /var/lib/ambari-agent/cache/common-services/SPARK/1.2.1/package/scripts/thrift_server.py > does not exist' > """ > We should be able to fix it by renaming "thrift_server.py" used in BI 4.2 > stack to "spark_thrift_server.py" and updating metainfo.xml files accordingly. > > > Diffs > - > > > ambari-server/src/main/resources/stacks/BigInsights/4.0/services/SPARK/metainfo.xml > 02abc62 > > ambari-server/src/main/resources/stacks/BigInsights/4.0/services/SPARK/package/scripts/spark_thrift_server.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.0/services/SPARK/package/scripts/thrift_server.py > 39e15d3 > > ambari-server/src/main/resources/stacks/BigInsights/4.2/services/SPARK/metainfo.xml > 7b6bae1 > > ambari-server/src/main/resources/stacks/BigInsights/4.2/services/SPARK/package/scripts/spark_thrift_server.py > PRE-CREATION > > ambari-server/src/main/resources/stacks/BigInsights/4.2/services/SPARK/package/scripts/thrift_server.py > d1a8b67 > > > Diff: https://reviews.apache.org/r/60814/diff/1/ > > > Testing > --- > > to be tested in integration tests > > > Thanks, > > Di Li > >
Re: Review Request 60807: AMBARI-21453 Restart Hive after Install Packages fails with err on Condition with name 'should_install_mysl_connector'
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/60807/#review180330 --- Ship it! Ship It! - Tim Thorpe On July 12, 2017, 4:55 p.m., Di Li wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/60807/ > --- > > (Updated July 12, 2017, 4:55 p.m.) > > > Review request for Ambari, Dmytro Grinenko, Di Li, Dmitro Lisnichenko, > Jonathan Hurley, Sumit Mohanty, Sid Wagle, and Tim Thorpe. > > > Bugs: AMBARI-21453 > https://issues.apache.org/jira/browse/AMBARI-21453 > > > Repository: ambari > > > Description > --- > > Restart Hive after Install Packages fails with err on Condition with name > 'should_install_mysl_connector' > > > Diffs > - > > > ambari-server/src/main/resources/stacks/BigInsights/4.0/services/HIVE/metainfo.xml > 44e63e7 > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/HIVE/metainfo.xml > 8f703a5 > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/services/SQOOP/metainfo.xml > 05856fd > > ambari-server/src/main/resources/stacks/BigInsights/4.2/services/HIVE/metainfo.xml > d89a7d6 > > > Diff: https://reviews.apache.org/r/60807/diff/1/ > > > Testing > --- > > patch cluster, restart Hive > > > Thanks, > > Di Li > >
Re: Review Request 60691: AMBARI-21420 HBase master crashed during/post EU on an IOP4.2.5/HDP2.6 migrated cluster
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/60691/#review180181 --- Ship it! Ship It! - Tim Thorpe On July 10, 2017, 7:55 p.m., Di Li wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/60691/ > --- > > (Updated July 10, 2017, 7:55 p.m.) > > > Review request for Ambari, Dmytro Grinenko, Di Li, Dmitro Lisnichenko, > Jonathan Hurley, Sumit Mohanty, Sid Wagle, and Tim Thorpe. > > > Bugs: AMBARI-21420 > https://issues.apache.org/jira/browse/AMBARI-21420 > > > Repository: ambari > > > Description > --- > > HBase in IOP 4.2.5 is of higher version than the one shipped in HDP 2.6. > HBase master appeared as if it had started according to the Ambari action log > but later crashed with error: > 2017-07-05 09:28:08,952 FATAL [hats1:16000.activeMasterManager] > master.HMaster: Failed to become active masterjava.io.IOException: The > procedure class org.apache.hadoop.hbase.master.procedure.ServerCrashProcedure > must be accessible and have an empty constructorat > org.apache.hadoop.hbase.procedure2.Procedure.newInstance(Procedure.java:663)at > org.apache.hadoop.hbase.procedure2.Procedure.convert(Procedure.java:789)at > org.apache.hadoop.hbase.procedure2.store.wal.ProcedureWALFormatReader.read(ProcedureWALFormatReader.java:111)at > > org.apache.hadoop.hbase.procedure2.store.wal.ProcedureWALFormat.load(ProcedureWALFormat.java:81)at > > org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.load(WALProcedureStore.java:374)at > > org.apache.hadoop.hbase.procedure2.ProcedureExecutor.load(ProcedureExecutor.java:297)at > > org.apache.hadoop.hbase.procedure2.ProcedureExecutor.start(ProcedureExecutor.java:464)at > > org.apache.hadoop.hbase.master.HMaster.startProcedureExecutor(HMaster.java:1297)at > org.apach e.hadoop.hbase.master.HMaster.startServiceThreads(HMaster.java:1196) > Attach one or more files to this issue > Attachments > > > Diffs > - > > > ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/hbase_service.py > a1003dc > > ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/params_linux.py > 43c6036 > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/config-upgrade.xml > 42999b2 > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/upgrades/nonrolling-upgrade-to-hdp-2.6.xml > e9c8541 > > ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/config-upgrade.xml > f9e3e15 > > ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/nonrolling-upgrade-to-hdp-2.6.xml > e9c8541 > > > Diff: https://reviews.apache.org/r/60691/diff/5/ > > > Testing > --- > > will be tested in the integration tests. > > > Thanks, > > Di Li > >
Re: Review Request 60754: AMBARI-21439 Hive/MapRed2 restart fails after Ambari upgrade
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/60754/#review180180 --- Ship it! Ship It! - Tim Thorpe On July 10, 2017, 9:40 p.m., Di Li wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/60754/ > --- > > (Updated July 10, 2017, 9:40 p.m.) > > > Review request for Ambari, Dmytro Grinenko, Di Li, Dmitro Lisnichenko, > Jonathan Hurley, Sumit Mohanty, Sid Wagle, and Tim Thorpe. > > > Bugs: AMBARI-21439 > https://issues.apache.org/jira/browse/AMBARI-21439 > > > Repository: ambari > > > Description > --- > > The BI stacks copied to HDP uses old copy_to_hdfs Python API from > ambari-common that uses the now-renamed "host_system_prepped", It's renamed > to "skip". > This issue only happens on BI 4.2 GA clusters. References to the copy_to_hdfs > API need to use the new parameter - skip > > > Diffs > - > > > ambari-server/src/main/resources/stacks/BigInsights/4.0/services/YARN/package/scripts/historyserver.py > bb384e4 > > ambari-server/src/main/resources/stacks/BigInsights/4.2/services/HIVE/package/scripts/hive.py > f6fddd0 > > ambari-server/src/main/resources/stacks/BigInsights/4.2/services/HIVE/package/scripts/hive_server.py > 9c336eb > > ambari-server/src/main/resources/stacks/BigInsights/4.2/services/SLIDER/package/scripts/service_check.py > d56f612 > > ambari-server/src/main/resources/stacks/BigInsights/4.2/services/YARN/package/scripts/historyserver.py > 40ca374 > > > Diff: https://reviews.apache.org/r/60754/diff/1/ > > > Testing > --- > > patched a cluster with the changes, restart Hive, Yarn, run service check for > Slider > > > Thanks, > > Di Li > >
Re: Review Request 60574: AMBARI-21383 Update upgrade pack to set mariadb support property for hive
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/60574/#review179402 --- Ship it! Ship It! - Tim Thorpe On June 30, 2017, 7:15 p.m., Di Li wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/60574/ > --- > > (Updated June 30, 2017, 7:15 p.m.) > > > Review request for Ambari, Dmytro Grinenko, Di Li, Dmitro Lisnichenko, > Jonathan Hurley, Sumit Mohanty, Sid Wagle, and Tim Thorpe. > > > Bugs: AMBARI-21383 > https://issues.apache.org/jira/browse/AMBARI-21383 > > > Repository: ambari > > > Description > --- > > Update upgrade pack to set mariadb support property for hive > > > Diffs > - > > > ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/config-upgrade.xml > 540c017 > > ambari-server/src/main/resources/stacks/BigInsights/4.2/upgrades/nonrolling-upgrade-to-hdp-2.6.xml > e8a1e33 > > > Diff: https://reviews.apache.org/r/60574/diff/1/ > > > Testing > --- > > integration test later on. > > > Thanks, > > Di Li > >
Re: Review Request 60537: AMBARI-21373 Remove clean repo logic not suitable for Apache Ambari Python API
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/60537/#review179262 --- Ship it! Ship It! - Tim Thorpe On June 29, 2017, 1:39 p.m., Di Li wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/60537/ > --- > > (Updated June 29, 2017, 1:39 p.m.) > > > Review request for Ambari, Dmytro Grinenko, Di Li, Dmitro Lisnichenko, > Jonathan Hurley, Sumit Mohanty, Sid Wagle, and Tim Thorpe. > > > Bugs: AMBARI-21373 > https://issues.apache.org/jira/browse/AMBARI-21373 > > > Repository: ambari > > > Description > --- > > The clean repo logic is internal only, remove it in Apache Ambari as > community version does not have necessary Python API in recourse_management. > Also the code is not a must-have > > > Diffs > - > > > ambari-server/src/main/resources/stacks/BigInsights/4.0/hooks/before-INSTALL/scripts/repo_initialization.py > 1b0f5c6 > > ambari-server/src/main/resources/stacks/BigInsights/4.2.5/hooks/before-INSTALL/scripts/repo_initialization.py > 98fa131 > > > Diff: https://reviews.apache.org/r/60537/diff/1/ > > > Testing > --- > > Patched an IOP/HDP cluster post Ambari upgrade, before migration. > > > Thanks, > > Di Li > >
Re: Review Request 60509: AMBARI-21368 Update Ambari server rat exclude list with BI AMS files
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/60509/#review179146 --- Ship it! Ship It! - Tim Thorpe On June 28, 2017, 6:03 p.m., Di Li wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/60509/ > --- > > (Updated June 28, 2017, 6:03 p.m.) > > > Review request for Ambari, Dmytro Grinenko, Di Li, Dmitro Lisnichenko, > Jonathan Hurley, Sumit Mohanty, Sid Wagle, and Tim Thorpe. > > > Bugs: AMBARI-21368 > https://issues.apache.org/jira/browse/AMBARI-21368 > > > Repository: ambari > > > Description > --- > > Update Ambari server rat exclude list with BI AMS files > > > Diffs > - > > ambari-server/pom.xml 51759d4 > > > Diff: https://reviews.apache.org/r/60509/diff/1/ > > > Testing > --- > > mvn clean install -pl ambari-server > > > Thanks, > > Di Li > >
Re: Review Request 60481: AMBARI-21359 Add JavaVersionCheck check
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/60481/#review179117 --- Ship it! Ship It! - Tim Thorpe On June 28, 2017, 3:16 p.m., Di Li wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/60481/ > --- > > (Updated June 28, 2017, 3:16 p.m.) > > > Review request for Ambari, Dmytro Grinenko, Dmitro Lisnichenko, Jonathan > Hurley, Sumit Mohanty, Sid Wagle, and Tim Thorpe. > > > Bugs: AMBARI-21359 > https://issues.apache.org/jira/browse/AMBARI-21359 > > > Repository: ambari > > > Description > --- > > This is for the JavaVersionCheck check defined in the cross stack upgrade > pack EU xml file. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/checks/CheckDescription.java > 71c5857 > > ambari-server/src/main/java/org/apache/ambari/server/checks/JavaVersionCheck.java > PRE-CREATION > > > Diff: https://reviews.apache.org/r/60481/diff/3/ > > > Testing > --- > > to be tested later on with the integrated tests. > > > Thanks, > > Di Li > >
Re: Review Request 60489: Typo in stack advisor error message for yarn and mr queue config issues
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/60489/#review179089 --- Ship it! Ship It! - Tim Thorpe On June 28, 2017, 5:23 a.m., Sangeeta Ravindran wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/60489/ > --- > > (Updated June 28, 2017, 5:23 a.m.) > > > Review request for Ambari, Di Li and Tim Thorpe. > > > Bugs: AMBARI-21274 > https://issues.apache.org/jira/browse/AMBARI-21274 > > > Repository: ambari > > > Description > --- > > If there are issues with yarn service check queue or mapreduce job queue > configurations, service advisor throws an error with a typo. > > Queue is not exist or not corresponds to existing YARN leaf queue > > > Diffs > - > > ambari-server/src/main/resources/stacks/stack_advisor.py 3a39a34 > ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py > b6f1965 > ambari-server/src/test/python/stacks/2.5/common/test_stack_advisor.py > e62e00c > > contrib/management-packs/odpi-ambari-mpack/src/main/resources/stacks/ODPi/2.0/services/stack_advisor.py > 5a95fe3 > > > Diff: https://reviews.apache.org/r/60489/diff/1/ > > > Testing > --- > > Manual testing. > Ran python tests cleanly. > > > Thanks, > > Sangeeta Ravindran > >
Re: Review Request 60481: AMBARI-21359 Add JavaVersionCheck check
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/60481/#review179056 --- Ship it! Ship It! - Tim Thorpe On June 27, 2017, 9:02 p.m., Di Li wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/60481/ > --- > > (Updated June 27, 2017, 9:02 p.m.) > > > Review request for Ambari, Dmytro Grinenko, Dmitro Lisnichenko, Jonathan > Hurley, Sumit Mohanty, Sid Wagle, and Tim Thorpe. > > > Bugs: AMBARI-21359 > https://issues.apache.org/jira/browse/AMBARI-21359 > > > Repository: ambari > > > Description > --- > > This is for the JavaVersionCheck check defined in the cross stack upgrade > pack EU xml file. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/checks/CheckDescription.java > 71c5857 > > ambari-server/src/main/java/org/apache/ambari/server/checks/JavaVersionCheck.java > PRE-CREATION > > > Diff: https://reviews.apache.org/r/60481/diff/1/ > > > Testing > --- > > to be tested later on with the integrated tests. > > > Thanks, > > Di Li > >
Re: Review Request 60480: AMBARI-21351 Add Support For MariaDB to Python Scripts
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/60480/#review179055 --- Ship it! Ship It! - Tim Thorpe On June 27, 2017, 8:08 p.m., Di Li wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/60480/ > --- > > (Updated June 27, 2017, 8:08 p.m.) > > > Review request for Ambari, Dmytro Grinenko, Di Li, Dmitro Lisnichenko, > Jonathan Hurley, Sumit Mohanty, Sid Wagle, and Tim Thorpe. > > > Bugs: AMBARI-21351 > https://issues.apache.org/jira/browse/AMBARI-21351 > > > Repository: ambari > > > Description > --- > > Some stacks, such as BigInights, support MariaDB as the database for Hive. > When upgrading from BigInights to another stack, such as HDP, this causes the > following exception when starting the Hive service: > ```resource_management.core.exceptions.ExecutionFailed: Execution of 'service > mysqld stop' returned 5. Redirecting to /bin/systemctl stop mysqld.service > Failed to stop mysqld.service: Unit mysqld.service not loaded.``` > This looks to be caused by the hard-coded reliance of mysqld in the Python > scripts. > > > Diffs > - > > > ambari-common/src/main/python/resource_management/libraries/functions/constants.py > c74e029 > > ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/status_params.py > b7cb148 > > > Diff: https://reviews.apache.org/r/60480/diff/1/ > > > Testing > --- > > Tim and I patched a cluster with basically the same fix when migrating an IOP > 4.2.5 cluster to HDP 2.6/Ambari 2.5.1. THis patch has one additional change > that's to set StackFeature boolean in case this code gets merged back to > trunk later on. > > If the code only stays in this branch, the StackFeature boolean change can be > omitted. > > > Thanks, > > Di Li > >
Re: Review Request 60390: AMBARI-21327 Ambari server to print error messages if NN HA namenode services properties use diff FQDN (dual network cards) than FQDN in the HostComponentState table
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/60390/#review178892 --- Ship it! Ship It! - Tim Thorpe On June 23, 2017, 3:29 p.m., Di Li wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/60390/ > --- > > (Updated June 23, 2017, 3:29 p.m.) > > > Review request for Ambari and Tim Thorpe. > > > Bugs: AMBARI-21327 > https://issues.apache.org/jira/browse/AMBARI-21327 > > > Repository: ambari > > > Description > --- > > For hosts with dual network cards and configured with two FQDN, it's possible > to manually config NN HA with alternative FQDN than the ones recorded in the > Ambari database. This caused an issue for EU/RU when upon kicking off the > upgrade via UI, the UI immediately posts the following error: > "An internal system exception occurred: Request specifies host X but it > is not a valid host based on the target service=null and component=null" > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/stack/MasterHostResolver.java > 427a5f5 > > > Diff: https://reviews.apache.org/r/60390/diff/1/ > > > Testing > --- > > existing unit tests. > > > Thanks, > > Di Li > >
Re: Review Request 60206: AMBARI-21281 Code refactoring: rename RU_TASKS Stagewrapper type to upgrade tasks
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/60206/#review178317 --- Ship it! Ship It! - Tim Thorpe On June 19, 2017, 8:37 p.m., Di Li wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/60206/ > --- > > (Updated June 19, 2017, 8:37 p.m.) > > > Review request for Ambari and Tim Thorpe. > > > Bugs: AMBARI-21281 > https://issues.apache.org/jira/browse/AMBARI-21281 > > > Repository: ambari > > > Description > --- > > Code refactoring: rename RU_TASKS Stagewrapper type to upgrade tasks > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java > 2f6ffc1 > > ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ClusterGrouping.java > 8e59602 > > ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/ExecuteTask.java > 960a04c > > ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/Grouping.java > 51bd168 > > ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/StageWrapper.java > 288f98a > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UpgradeResourceProviderTest.java > 014ab42 > > > Diff: https://reviews.apache.org/r/60206/diff/1/ > > > Testing > --- > > existing unit tests. > > > Thanks, > > Di Li > >
Re: Review Request 59838: Stack advisor needs to recommend dependency for slaves and masters
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59838/ --- (Updated June 13, 2017, 9:56 p.m.) Review request for Ambari, Alejandro Fernandez, Di Li, Jaimin Jetly, Jonathan Hurley, Jayush Luniya, and Nate Cole. Changes --- Updated diff to fix the UI issue: File "/var/lib/ambari-server/resources/scripts/../stacks/stack_advisor.py", line 901, in sortServicesByDependencies for component in service["components"]: TypeError: 'NoneType' object is unsubscriptable Bugs: AMBARI-20122 https://issues.apache.org/jira/browse/AMBARI-20122 Repository: ambari Description --- After resolution of AMBARI-19685, stack advisor validates if stack defined dependency is not satisfied but recommendation API does not account for this. Stack defined dependencies are service/component based and has a scope CLUSTER|HOST. During recommendation the services to install have already been selected. We can't really utilize the cluster scope because either the dependent service was selected or it was not. If it was not selected it will be caught during validation. We can only recommend based on HOST scope. This JIRA is also limited to only handling those which don't have conditional dependencies. Diffs (updated) - ambari-server/src/main/resources/stacks/stack_advisor.py 4a81dc6 ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py 41c57f6 Diff: https://reviews.apache.org/r/59838/diff/4/ Changes: https://reviews.apache.org/r/59838/diff/3-4/ Testing --- Added new test case to test the recommendation of dependencies for slaves and masters mvn clean test -DskipSurefireTests New trunk without my change Total run:1168 Total errors:13 Total failures:0 New trunk with my change Total run:1169 Total errors:13 Total failures:0 Thanks, Tim Thorpe
Re: Review Request 59838: Stack advisor needs to recommend dependency for slaves and masters
> On June 13, 2017, 7:09 p.m., Nate Cole wrote: > > Ship It! Hi Nate, still working on this to fix a deploy with UI issue will hopefully have another patch up soon. Thanks - Tim --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59838/#review177782 --- On June 7, 2017, 12:26 p.m., Tim Thorpe wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/59838/ > --- > > (Updated June 7, 2017, 12:26 p.m.) > > > Review request for Ambari, Alejandro Fernandez, Di Li, Jaimin Jetly, Jonathan > Hurley, Jayush Luniya, and Nate Cole. > > > Bugs: AMBARI-20122 > https://issues.apache.org/jira/browse/AMBARI-20122 > > > Repository: ambari > > > Description > --- > > After resolution of AMBARI-19685, stack advisor validates if stack defined > dependency is not satisfied but recommendation API does not account for this. > > Stack defined dependencies are service/component based and has a scope > CLUSTER|HOST. > > During recommendation the services to install have already been selected. We > can't really utilize the cluster scope because either the dependent service > was selected or it was not. If it was not selected it will be caught during > validation. We can only recommend based on HOST scope. > > This JIRA is also limited to only handling those which don't have conditional > dependencies. > > > Diffs > - > > ambari-server/src/main/resources/stacks/stack_advisor.py 4a81dc6 > ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py > 41c57f6 > > > Diff: https://reviews.apache.org/r/59838/diff/3/ > > > Testing > --- > > Added new test case to test the recommendation of dependencies for slaves and > masters > > mvn clean test -DskipSurefireTests > > > New trunk without my change > > Total run:1168 > Total errors:13 > Total failures:0 > > New trunk with my change > > Total run:1169 > Total errors:13 > Total failures:0 > > > Thanks, > > Tim Thorpe > >
Review Request 60011: Remove Zookeeper as a required service from YARN
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/60011/ --- Review request for Ambari, Alejandro Fernandez, Di Li, Jonathan Hurley, and Nate Cole. Bugs: AMBARI-21206 https://issues.apache.org/jira/browse/AMBARI-21206 Repository: ambari Description --- YARN has an implicit dependency on ZOOKEEPER. It currently didn't declare an explicit dependency because HDFS had one. Enabling the following settings in YARN requires ZOOKEEPER: yarn.resourcemanager.recovery.enabled yarn.nodemanager.recovery.enabled yarn.resourcemanager.ha.enabled hadoop.registry.rm.enabled In addition, these properties are related to ZOOKEEPER and should be empty if ZOOKEEPER is not installed: yarn.resourcemanager.zk-address hadoop.registry.zk.quorum Diffs - ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml 0769ae8 ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/metainfo.xml 3e1a7ae ambari-server/src/main/resources/common-services/YARN/3.0.0.3.0/configuration/yarn-site.xml 64e0bcb ambari-server/src/main/resources/common-services/YARN/3.0.0.3.0/metainfo.xml a802795 ambari-server/src/main/resources/common-services/YARN/3.0.0.3.0/service_advisor.py 0fb538d ambari-server/src/main/resources/stacks/HDP/2.2/services/stack_advisor.py 726514b ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py 571ff26 ambari-server/src/test/python/stacks/2.6/common/test_stack_advisor.py d4d28c9 Diff: https://reviews.apache.org/r/60011/diff/1/ Testing --- mvn clean test -DskipSurefireTests mvn clean test -DskipSurefireTests -Dpython.test.mask=test_stack_advisor.py Thanks, Tim Thorpe
Re: Review Request 59955: AMBARI-21218 Enable Kerberos UI is blank the first time opens it
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59955/#review177515 --- Ship it! Ship It! - Tim Thorpe On June 9, 2017, 8:07 p.m., Di Li wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/59955/ > --- > > (Updated June 9, 2017, 8:07 p.m.) > > > Review request for Ambari and Tim Thorpe. > > > Bugs: AMBARI-21218 > https://issues.apache.org/jira/browse/AMBARI-21218 > > > Repository: ambari > > > Description > --- > > Enable Kerberos UI is blank the first time opens it (see screenshot attached > for details). Closing it and reopening it will see the Enable Kerberos wizard > UI populated correctly with UI elements. This was due to a NPE during loading > cluster data during initial loading of the UI. The NPE caused > "isComponentsStateLoaded" not to be set with the proper value. > > > Diffs > - > > ambari-web/app/controllers/main/admin/stack_and_upgrade_controller.js > a676f7429 > > > Diff: https://reviews.apache.org/r/59955/diff/1/ > > > Testing > --- > > unit tests. patch a trunk cluster and verify the issue is fixed. > > > Thanks, > > Di Li > >
Re: Review Request 59953: AMBARI-21213 Consolidate UI for the "Loading..." text
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59953/#review177507 --- Ship it! Ship It! - Tim Thorpe On June 9, 2017, 7:01 p.m., Di Li wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/59953/ > --- > > (Updated June 9, 2017, 7:01 p.m.) > > > Review request for Ambari and Tim Thorpe. > > > Bugs: AMBARI-21213 > https://issues.apache.org/jira/browse/AMBARI-21213 > > > Repository: ambari > > > Description > --- > > Consolidate UI for the "Loading..." text to use one label from message.js > > > Diffs > - > > > ambari-web/app/controllers/main/admin/highAvailability/progress_popup_controller.js > 7231e5c > ambari-web/app/messages.js 6f317a3 > ambari-web/app/templates/main.hbs bf6fa56 > ambari-web/app/utils/ajax/ajax.js 492a216 > ambari-web/app/views/common/configs/service_config_container_view.js > a7cee0c > ambari-web/app/views/loading.js 81d4a60 > ambari-web/app/views/main/alerts/definition_details_view.js 5c8efff > ambari-web/test/views/main/alerts/definition_details_view_test.js b3c4712 > > > Diff: https://reviews.apache.org/r/59953/diff/1/ > > > Testing > --- > > unit tests. patch trunk cluster with changes. > > > Thanks, > > Di Li > >
Review Request 59919: YARN service advisor has spelling error in yarn-site
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59919/
---
Review request for Ambari, Alejandro Fernandez, Di Li, and Jonathan Hurley.
Bugs: AMBARI-21202
https://issues.apache.org/jira/browse/AMBARI-21202
Repository: ambari
Description
---
self.validators = [("yarn-site", self.validateYARNSiteConfigurationsFromHDP206),
("yarn-site", self.validateYARNSiteConfigurationsFromHDP25),
("yarn-ste" , self.validateYarnSiteConfigurationsFromHDP26),
The HDP26 validator should be for yarn-site not yarn-ste.
Diffs
-
ambari-server/src/main/resources/common-services/YARN/3.0.0.3.0/service_advisor.py
3dd39e5
Diff: https://reviews.apache.org/r/59919/diff/1/
Testing
---
mvn clean test -DskipSurefireTests
Thanks,
Tim Thorpe
Re: Review Request 59913: AMBARI-21197 Disable Kerberos UI step by step panel misalign with the Complete button at the bottom of the page
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59913/#review177310 --- Ship it! Ship It! - Tim Thorpe On June 8, 2017, 12:45 p.m., Di Li wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/59913/ > --- > > (Updated June 8, 2017, 12:45 p.m.) > > > Review request for Ambari, Sangeeta Ravindran and Tim Thorpe. > > > Bugs: AMBARI-21197 > https://issues.apache.org/jira/browse/AMBARI-21197 > > > Repository: ambari > > > Description > --- > > Disable Kerberos UI step by step panel misalign with the Complete button at > the bottom of the page. See screenshot for details. > > > Diffs > - > > ambari-web/app/templates/common/progress.hbs ab59328 > ambari-web/app/views/main/admin/kerberos/disable_view.js f7b3527 > > > Diff: https://reviews.apache.org/r/59913/diff/1/ > > > Testing > --- > > existing unit tests. patch a trunk cluster, enable kerberos, verify the > common progress pages have the correct width (via the new if/else logic), run > disable kerberos, verify the progress page now has the correct width (via the > new if/else logic) > > > Thanks, > > Di Li > >
Re: Review Request 59727: AMBARI-21161 Able to safe guard Ambari server against circular dependencies in role command order
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59727/#review177169 --- Ship it! Ship It! - Tim Thorpe On June 6, 2017, 8:23 p.m., Di Li wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/59727/ > --- > > (Updated June 6, 2017, 8:23 p.m.) > > > Review request for Ambari and Tim Thorpe. > > > Bugs: AMBARI-21161 > https://issues.apache.org/jira/browse/AMBARI-21161 > > > Repository: ambari > > > Description > --- > > It is important to safe guard against cycle graphs, when Ambari supports > mpacks, custom services and service level role command order. Cycle graphs > indicate circular dependencies that can cause Ambari enter an infinite loop > while building stages, causing Ambari server to take up 100% CPU and large > amount of memory. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/stageplanner/RoleGraph.java > 024a5e1 > > ambari-server/src/test/java/org/apache/ambari/server/stageplanner/TestStagePlanner.java > 271ee89 > ambari-server/src/test/resources/stacks/HDP/2.0.6.1/role_command_order.json > 6154004 > > > Diff: https://reviews.apache.org/r/59727/diff/3/ > > > Testing > --- > > unit tests. patch a trunk cluster with code change and install HDP cluster, > intentionally add circular reference to make NN start depends on DN start > and DN start depends on NN start. Verify UI shows an error message when try > to start HDFS service, and does not really kick off the HDFS start. (without > the fix, Ambari server will try to start HDFS and gets itself into an > infinite loop and uses up 100% CPU and large amount of RAM) > > > Thanks, > > Di Li > >
Re: Review Request 59838: Stack advisor needs to recommend dependency for slaves and masters
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59838/ --- (Updated June 7, 2017, 12:26 p.m.) Review request for Ambari, Alejandro Fernandez, Di Li, Jaimin Jetly, Jonathan Hurley, Jayush Luniya, and Nate Cole. Changes --- Removed white space Bugs: AMBARI-20122 https://issues.apache.org/jira/browse/AMBARI-20122 Repository: ambari Description --- After resolution of AMBARI-19685, stack advisor validates if stack defined dependency is not satisfied but recommendation API does not account for this. Stack defined dependencies are service/component based and has a scope CLUSTER|HOST. During recommendation the services to install have already been selected. We can't really utilize the cluster scope because either the dependent service was selected or it was not. If it was not selected it will be caught during validation. We can only recommend based on HOST scope. This JIRA is also limited to only handling those which don't have conditional dependencies. Diffs (updated) - ambari-server/src/main/resources/stacks/stack_advisor.py 4a81dc6 ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py 41c57f6 Diff: https://reviews.apache.org/r/59838/diff/3/ Changes: https://reviews.apache.org/r/59838/diff/2-3/ Testing --- Added new test case to test the recommendation of dependencies for slaves and masters mvn clean test -DskipSurefireTests New trunk without my change Total run:1168 Total errors:13 Total failures:0 New trunk with my change Total run:1169 Total errors:13 Total failures:0 Thanks, Tim Thorpe
Re: Review Request 59838: Stack advisor needs to recommend dependency for slaves and masters
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59838/ --- (Updated June 7, 2017, 12:23 p.m.) Review request for Ambari, Alejandro Fernandez, Di Li, Jaimin Jetly, Jonathan Hurley, Jayush Luniya, and Nate Cole. Changes --- Updated patch to include documenting new functions. Bugs: AMBARI-20122 https://issues.apache.org/jira/browse/AMBARI-20122 Repository: ambari Description --- After resolution of AMBARI-19685, stack advisor validates if stack defined dependency is not satisfied but recommendation API does not account for this. Stack defined dependencies are service/component based and has a scope CLUSTER|HOST. During recommendation the services to install have already been selected. We can't really utilize the cluster scope because either the dependent service was selected or it was not. If it was not selected it will be caught during validation. We can only recommend based on HOST scope. This JIRA is also limited to only handling those which don't have conditional dependencies. Diffs (updated) - ambari-server/src/main/resources/stacks/stack_advisor.py 4a81dc6 ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py 41c57f6 Diff: https://reviews.apache.org/r/59838/diff/2/ Changes: https://reviews.apache.org/r/59838/diff/1-2/ Testing --- Added new test case to test the recommendation of dependencies for slaves and masters mvn clean test -DskipSurefireTests New trunk without my change Total run:1168 Total errors:13 Total failures:0 New trunk with my change Total run:1169 Total errors:13 Total failures:0 Thanks, Tim Thorpe
Re: Review Request 59095: AMBARI-20967 - Remove Zookeeper as a required service from HDFS
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59095/ --- (Updated June 6, 2017, 8:16 p.m.) Review request for Ambari, Alejandro Fernandez, Di Li, Jonathan Hurley, Jayush Luniya, and Nate Cole. Changes --- Updated patch to include both HDFS 2.1.* and 3.0. Also added the zookeeper dependency at least temporarily to YARN. YARN has an implicit dependency on ZOOKEEPER. It didn't need to explicitely state the dependency because HDFS had one. Bugs: AMBARI-20967 https://issues.apache.org/jira/browse/AMBARI-20967 Repository: ambari Description --- HDFS has a static dependency on Zookeeper even though it is only needed when NameNode HA is enabled. The check for Zookeeper should be done before enabling HA rather than enforcing it at installation time. Diffs (updated) - ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml 0769ae8 ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/metainfo.xml 3e1a7ae ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/metainfo.xml b72f309 ambari-server/src/main/resources/common-services/YARN/3.0.0.3.0/metainfo.xml a802795 Diff: https://reviews.apache.org/r/59095/diff/2/ Changes: https://reviews.apache.org/r/59095/diff/1-2/ Testing --- mvn clean test Manual testing - installed HDFS only cluster and attempted to enable NameNode HA. This is prevented with the same error you'd see if ZooKeeper was enabled but you had fewer than 3 ZooKeeper servers configured. Thanks, Tim Thorpe
Re: Review Request 59838: Stack advisor needs to recommend dependency for slaves and masters
> On June 6, 2017, 5:38 p.m., Alejandro Fernandez wrote: > > ambari-server/src/main/resources/stacks/stack_advisor.py > > Lines 759 (patched) > > <https://reviews.apache.org/r/59838/diff/1/?file=1742546#file1742546line759> > > > > Can you provide an example of why this has to be sorted? Hi Alejandro, In the function def getFilteredHostsBasedOnDependencies, you can see that it uses the hostsComponentsMap to determine which hosts a given component will be installed on. If the services are not processed in dependency order then you might process HDFS before ZOOKEEPER and there would be no hosts where ZOOKEEPER will be installed. for host, hostComponents in hostsComponentsMap.iteritems(): isRequiredIncluded = False for component in hostComponents: currentComponentName = None if "name" not in component else component["name"] if requiredComponentName == currentComponentName: isRequiredIncluded = True if not isRequiredIncluded: removeHosts.append(host) - Tim --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59838/#review177055 ------- On June 6, 2017, 11:33 a.m., Tim Thorpe wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/59838/ > --- > > (Updated June 6, 2017, 11:33 a.m.) > > > Review request for Ambari, Alejandro Fernandez, Di Li, Jaimin Jetly, Jonathan > Hurley, Jayush Luniya, and Nate Cole. > > > Bugs: AMBARI-20122 > https://issues.apache.org/jira/browse/AMBARI-20122 > > > Repository: ambari > > > Description > --- > > After resolution of AMBARI-19685, stack advisor validates if stack defined > dependency is not satisfied but recommendation API does not account for this. > > Stack defined dependencies are service/component based and has a scope > CLUSTER|HOST. > > During recommendation the services to install have already been selected. We > can't really utilize the cluster scope because either the dependent service > was selected or it was not. If it was not selected it will be caught during > validation. We can only recommend based on HOST scope. > > This JIRA is also limited to only handling those which don't have conditional > dependencies. > > > Diffs > - > > ambari-server/src/main/resources/stacks/stack_advisor.py 4a81dc6 > ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py > 41c57f6 > > > Diff: https://reviews.apache.org/r/59838/diff/1/ > > > Testing > --- > > Added new test case to test the recommendation of dependencies for slaves and > masters > > mvn clean test -DskipSurefireTests > > > New trunk without my change > > Total run:1168 > Total errors:13 > Total failures:0 > > New trunk with my change > > Total run:1169 > Total errors:13 > Total failures:0 > > > Thanks, > > Tim Thorpe > >
Review Request 59838: Stack advisor needs to recommend dependency for slaves and masters
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59838/ --- Review request for Ambari, Alejandro Fernandez, Di Li, Jaimin Jetly, Jonathan Hurley, Jayush Luniya, and Nate Cole. Bugs: AMBARI-20122 https://issues.apache.org/jira/browse/AMBARI-20122 Repository: ambari Description --- After resolution of AMBARI-19685, stack advisor validates if stack defined dependency is not satisfied but recommendation API does not account for this. Stack defined dependencies are service/component based and has a scope CLUSTER|HOST. During recommendation the services to install have already been selected. We can't really utilize the cluster scope because either the dependent service was selected or it was not. If it was not selected it will be caught during validation. We can only recommend based on HOST scope. This JIRA is also limited to only handling those which don't have conditional dependencies. Diffs - ambari-server/src/main/resources/stacks/stack_advisor.py 4a81dc6 ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py 41c57f6 Diff: https://reviews.apache.org/r/59838/diff/1/ Testing --- Added new test case to test the recommendation of dependencies for slaves and masters mvn clean test -DskipSurefireTests New trunk without my change Total run:1168 Total errors:13 Total failures:0 New trunk with my change Total run:1169 Total errors:13 Total failures:0 Thanks, Tim Thorpe
Re: Review Request 59095: AMBARI-20967 - Remove Zookeeper as a required service from HDFS
> On June 1, 2017, 5:01 p.m., Alejandro Fernandez wrote: > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml > > Line 351 (original) > > <https://reviews.apache.org/r/59095/diff/1/?file=1711501#file1711501line351> > > > > If we remove this block, it will create a situation in which a user can > > deploy HDFS without ZK, then attempt to enable HA and then fail. > > > > For this reason, I think we should retract this patch until we have a > > good way of annotating this dependency on the stack (ideally, the NameNode > > HA workflow in the stack would check that requirement). Hi Alejandro, the UI currently has a check to make sure that there are at least 3 zookeeper servers installed. This check applies even if Zookeeper is installed. For example I may have installed HDFS and ZK but only put ZK on one node. This will still pass the dependency check because ZK is installed but it still fails the HA enabling because that requires 3 ZK servers. I have tested this with both HDFS and ZK on only one node and just with HDFS. In both instances I get the same error message. There is one issue with my patch. YARN currently has an undeclared dependency on ZK. I will modify my patch to add ZK as a dependency for YARN. Later I will work on removing the ZK dependency for YARN. - Tim --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59095/#review176625 ------- On May 9, 2017, 2:39 p.m., Tim Thorpe wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/59095/ > --- > > (Updated May 9, 2017, 2:39 p.m.) > > > Review request for Ambari, Alejandro Fernandez, Di Li, Jonathan Hurley, > Jayush Luniya, and Nate Cole. > > > Bugs: AMBARI-20967 > https://issues.apache.org/jira/browse/AMBARI-20967 > > > Repository: ambari > > > Description > --- > > HDFS has a static dependency on Zookeeper even though it is only needed when > NameNode HA is enabled. The check for Zookeeper should be done before > enabling HA rather than enforcing it at installation time. > > > Diffs > - > > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml > 3026938 > > > Diff: https://reviews.apache.org/r/59095/diff/1/ > > > Testing > --- > > mvn clean test > > Manual testing - installed HDFS only cluster and attempted to enable NameNode > HA. This is prevented with the same error you'd see if ZooKeeper was enabled > but you had fewer than 3 ZooKeeper servers configured. > > > Thanks, > > Tim Thorpe > >
Re: Review Request 59440: Part One: Specify the script directly in alert target for script-based alert dispatchers
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59440/#review176601 --- Ship it! Ship It! - Tim Thorpe On June 1, 2017, 1:37 a.m., yao lei wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/59440/ > --- > > (Updated June 1, 2017, 1:37 a.m.) > > > Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, > and Tim Thorpe. > > > Bugs: AMBARI-21122 > https://issues.apache.org/jira/browse/AMBARI-21122 > > > Repository: ambari > > > Description > --- > > This patch will support using property > ambari.dispatch-property.script.filename in alert target to tell > AlertScriptDispatcher to lookup script by filename,default in directory > /var/lib/ambari-server/resources/scripts. > > We can also change this directory in ambari.properties by > notification.dispatch.alert.script.directory property. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java > 114046f > > ambari-server/src/main/java/org/apache/ambari/server/notifications/dispatchers/AlertScriptDispatcher.java > 84bfe52 > > ambari-server/src/test/java/org/apache/ambari/server/notifications/dispatchers/AlertScriptDispatcherTest.java > f1f320d > > > Diff: https://reviews.apache.org/r/59440/diff/6/ > > > Testing > --- > > Tested in a cluster > > > Thanks, > > yao lei > >
Re: Review Request 59095: AMBARI-20967 - Remove Zookeeper as a required service from HDFS
> On May 30, 2017, 9:42 p.m., Jonathan Hurley wrote: > > I think that the overall problem here is how the UI determines that before > > you can enable NN HA, you must first add ZK to the cluster. Alejandro's > > concern is that the UI shouldn't have this responsibility - it should be > > declared somewhere on the stack. This way, the UI can query to determine if > > any services are missing. > > > > Is there a way which we can accomplish this? It seems to be a general > > workflow issue. > > > > I saw that NN has a general dependency on ZK if HA is enabled via a > > property condition: > > > > > > ZOOKEEPER/ZOOKEEPER_SERVER > > cluster > > > > false > > > > > > > > hdfs-site > > dfs.nameservices > > > > > > > > > > So this is currently modeled in the metainfo.xml which is good. The > > question is what currently uses this information to enforce that ZK is a > > part of the cluster _PRIOR_ to enabling NN HA. As far as I know those dependencies are only used by the new stack advisor code in AMBARI-19685 (to validate the layout) and will be in AMBARI-20122 (to recommend the layout). Those conditions should also be honoured by the stack advisor recommendation and validation of the configurations. They may be used by blueprints but I'm not all that familiar with that code. For special processes like enabling NN HA, the whole process is already hard coded into the UI. This includes a check on whether there are at least 3 zookeeper nodes. My option 3 (the one Alejandro liked) is actually redundant with what is already there in the conditional dependencies AND it doesn't actually help with the problem in question. If the UI needs to know what operations it can run then we'd need to do something with the custom commands section. We could potentially add something like this: ENABLE_NAMENODE_HA scripts/namenode.py PYTHON 600 ZOOKEEPER/ZOOKEEPER_SERVER service cluster 3+ I'm not sure how many other services would actually be able to make use of something like this. Currently enabling HA is completely hard coded in the UI and there is nothing about it in the metainfo.xml (this is true for both HDFS and YARN). I just don't see this extra work as really being of much benefit. If we can actually find at least one place where this would help with the current services then I'd be fine with adding it. - Tim --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59095/#review176366 --- On May 9, 2017, 2:39 p.m., Tim Thorpe wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/59095/ > --- > > (Updated May 9, 2017, 2:39 p.m.) > > > Review request for Ambari, Alejandro Fernandez, Di Li, Jonathan Hurley, > Jayush Luniya, and Nate Cole. > > > Bugs: AMBARI-20967 > https://issues.apache.org/jira/browse/AMBARI-20967 > > > Repository: ambari > > > Description > --- > > HDFS has a static dependency on Zookeeper even though it is only needed when > NameNode HA is enabled. The check for Zookeeper should be done before > enabling HA rather than enforcing it at installation time. > > > Diffs > - > > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml > 3026938 > > > Diff: https://reviews.apache.org/r/59095/diff/1/ > > > Testing > --- > > mvn clean test > > Manual testing - installed HDFS only cluster and attempted to enable NameNode > HA. This is prevented with the same error you'd see if ZooKeeper was enabled > but you had fewer than 3 ZooKeeper servers configured. > > > Thanks, > > Tim Thorpe > >
Re: Review Request 59503: Create custom admin user during ambari server setup
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59503/#review175918 --- Ship it! Ship It! - Tim Thorpe On May 23, 2017, 11:26 p.m., Anita Jebaraj wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/59503/ > --- > > (Updated May 23, 2017, 11:26 p.m.) > > > Review request for Ambari, Alejandro Fernandez, Di Li, Jonathan Hurley, > Robert Levas, and Tim Thorpe. > > > Bugs: AMBARI-21083 > https://issues.apache.org/jira/browse/AMBARI-21083 > > > Repository: ambari > > > Description > --- > > Provide the ability to create custom admin user during ambari server setup, > so that the user will have the option to either create the admin user with > the username/password of their preference or can use the default username. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java > 114046f > > ambari-server/src/main/java/org/apache/ambari/server/controller/CreateUserHelper.java > PRE-CREATION > ambari-server/src/main/python/ambari_server/serverConfiguration.py 4780338 > ambari-server/src/main/python/ambari_server/serverSetup.py c6de088 > ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql 15670f3 > ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql 7e41399 > ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql 4d0274f > ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql cc933fa > ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 5fc14d4 > ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 12e66f9 > > ambari-server/src/test/java/org/apache/ambari/server/controller/CreateUserHelperTest.java > PRE-CREATION > > > Diff: https://reviews.apache.org/r/59503/diff/2/ > > > Testing > --- > > Added 1 test case > > > Ran mvn test > > > File Attachments > > > custom_admin.PNG > > https://reviews.apache.org/media/uploaded/files/2017/05/23/53d168bc-6c83-4acf-ae9b-74bf3fa005d3__custom_admin.PNG > > > Thanks, > > Anita Jebaraj > >
Re: Review Request 59232: AMBARI-20984 Be able to include stack repos in blueprint
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59232/#review174844
---
Ship it!
Ship It!
- Tim Thorpe
On May 12, 2017, 7:43 p.m., Di Li wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59232/
> ---
>
> (Updated May 12, 2017, 7:43 p.m.)
>
>
> Review request for Ambari, Robert Nettleton and Tim Thorpe.
>
>
> Bugs: AMBARI-20984
> https://issues.apache.org/jira/browse/AMBARI-20984
>
>
> Repository: ambari
>
>
> Description
> ---
>
> Currently, if a user wants to use custom HDP repos during a blueprint
> deployment, he has to update the repo URLs before hand via a REST API. The
> JIRA here proposes a way to include stack repos in the blueprint as optional
> fields so that when the fields exist, Ambari server will use the repos to
> deploy a cluster. This eliminates the need for user to run a separate rest
> api call before the actual blueprint deployment.
> Attachments
>
>
> Diffs
> -
>
>
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java
> c655c62
>
> ambari-server/src/main/java/org/apache/ambari/server/topology/Blueprint.java
> 139a1ee
>
> ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintImpl.java
> 826e4e5
>
> ambari-server/src/main/java/org/apache/ambari/server/topology/RepositorySetting.java
> PRE-CREATION
> ambari-server/src/main/java/org/apache/ambari/server/topology/Setting.java
> 555192c
>
> ambari-server/src/main/java/org/apache/ambari/server/topology/TopologyManager.java
> f5cf498
>
> ambari-server/src/test/java/org/apache/ambari/server/topology/ClusterDeployWithStartOnlyTest.java
> c8d4d55
>
> ambari-server/src/test/java/org/apache/ambari/server/topology/ClusterInstallWithoutStartOnComponentLevelTest.java
> 4c9815c
>
> ambari-server/src/test/java/org/apache/ambari/server/topology/ClusterInstallWithoutStartTest.java
> 1bdeb1b
>
> ambari-server/src/test/java/org/apache/ambari/server/topology/SettingTest.java
> 0007acf
>
> ambari-server/src/test/java/org/apache/ambari/server/topology/TopologyManagerTest.java
> 0378753
>
>
> Diff: https://reviews.apache.org/r/59232/diff/3/
>
>
> Testing
> ---
>
> unit tests. patch a trunk cluster with the change and use blueprint to
> install a cluster.
>
> The section to add to the bp for the repos is like the following.
>
> override_strategy - ALWAYS_APPLY will override the existing repo info stored
> in the metainfo table in the Ambari db. APPLY_WHEN_MISSING will only insert
> repo info to the metainfo table if there is no such entries in the table yet.
>
> "settings" : [
> {
> "repository_settings" : [
> {
> "override_strategy":"ALWAYS_APPLY",
> "operating_system":"redhat7",
> "repo_id":"HDP-2.6",
>
> "base_url":"http://s3.amazonaws.com/dev.hortonworks.com/HDP/centos7/2.x/BUILDS/2.6.0.3-8;
> },
> {
> "override_strategy":"APPLY_WHEN_MISSING",
> "operating_system":"redhat7",
> "repo_id": "HDP-UTILS-1.1.0.21",
> "base_url":
> "http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos7;
> }
> ]
> }
> ],
>
>
> Thanks,
>
> Di Li
>
>
Re: Review Request 59232: AMBARI-20984 Be able to include stack repos in blueprint
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/59232/#review174840
---
ambari-server/src/main/java/org/apache/ambari/server/topology/RepositorySetting.java
Lines 28 (patched)
<https://reviews.apache.org/r/59232/#comment248056>
should be OPERATING_SYSTEM not OPERATION_SYSYSTEM
ambari-server/src/main/java/org/apache/ambari/server/topology/TopologyManager.java
Lines 363 (patched)
<https://reviews.apache.org/r/59232/#comment248057>
"New repository setting" instead of "New RepoistorySetting".
- Tim Thorpe
On May 12, 2017, 6:47 p.m., Di Li wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/59232/
> ---
>
> (Updated May 12, 2017, 6:47 p.m.)
>
>
> Review request for Ambari, Robert Nettleton and Tim Thorpe.
>
>
> Bugs: AMBARI-20984
> https://issues.apache.org/jira/browse/AMBARI-20984
>
>
> Repository: ambari
>
>
> Description
> ---
>
> Currently, if a user wants to use custom HDP repos during a blueprint
> deployment, he has to update the repo URLs before hand via a REST API. The
> JIRA here proposes a way to include stack repos in the blueprint as optional
> fields so that when the fields exist, Ambari server will use the repos to
> deploy a cluster. This eliminates the need for user to run a separate rest
> api call before the actual blueprint deployment.
> Attachments
>
>
> Diffs
> -
>
>
> ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java
> c655c62
>
> ambari-server/src/main/java/org/apache/ambari/server/topology/Blueprint.java
> 139a1ee
>
> ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintImpl.java
> 826e4e5
>
> ambari-server/src/main/java/org/apache/ambari/server/topology/RepositorySetting.java
> PRE-CREATION
> ambari-server/src/main/java/org/apache/ambari/server/topology/Setting.java
> 555192c
>
> ambari-server/src/main/java/org/apache/ambari/server/topology/TopologyManager.java
> f5cf498
>
> ambari-server/src/test/java/org/apache/ambari/server/topology/ClusterDeployWithStartOnlyTest.java
> c8d4d55
>
> ambari-server/src/test/java/org/apache/ambari/server/topology/ClusterInstallWithoutStartOnComponentLevelTest.java
> 4c9815c
>
> ambari-server/src/test/java/org/apache/ambari/server/topology/ClusterInstallWithoutStartTest.java
> 1bdeb1b
>
> ambari-server/src/test/java/org/apache/ambari/server/topology/SettingTest.java
> 0007acf
>
> ambari-server/src/test/java/org/apache/ambari/server/topology/TopologyManagerTest.java
> 0378753
>
>
> Diff: https://reviews.apache.org/r/59232/diff/2/
>
>
> Testing
> ---
>
> unit tests. patch a trunk cluster with the change and use blueprint to
> install a cluster.
>
> The section to add to the bp for the repos is like the following.
>
> override_strategy - ALWAYS_APPLY will override the existing repo info stored
> in the metainfo table in the Ambari db. APPLY_WHEN_MISSING will only insert
> repo info to the metainfo table if there is no such entries in the table yet.
>
> "settings" : [
> {
> "repository_settings" : [
> {
> "override_strategy":"ALWAYS_APPLY",
> "operating_system":"redhat7",
> "repo_id":"HDP-2.6",
>
> "base_url":"http://s3.amazonaws.com/dev.hortonworks.com/HDP/centos7/2.x/BUILDS/2.6.0.3-8;
> },
> {
> "override_strategy":"APPLY_WHEN_MISSING",
> "operating_system":"redhat7",
> "repo_id": "HDP-UTILS-1.1.0.21",
> "base_url":
> "http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos7;
> }
> ]
> }
> ],
>
>
> Thanks,
>
> Di Li
>
>
Review Request 59189: AMBARI-20992 - HDFS metainfo.xml has the wrong scope for component dependencies
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59189/ --- Review request for Ambari, Alejandro Fernandez and Di Li. Bugs: AMBARI-20992 https://issues.apache.org/jira/browse/AMBARI-20992 Repository: ambari Description --- HDFS/ZKFC, ZOOKEEPER/ZOOKEEPER_SERVER and HDFS/JOURNALNODE all currently are listed with host. They should all have cluster scope. This affects the metainfo.xml from common-services/HDFS both 2.1.0.2.0 and 3.0.0.3.0. Diffs - ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml 3026938 ambari-server/src/main/resources/common-services/HDFS/3.0.0.3.0/metainfo.xml 967c974 Diff: https://reviews.apache.org/r/59189/diff/1/ Testing --- mvn clean test Thanks, Tim Thorpe
Re: Review Request 59095: AMBARI-20967 - Remove Zookeeper as a required service from HDFS
> On May 11, 2017, 4:35 p.m., Alejandro Fernandez wrote: > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml > > Line 351 (original) > > <https://reviews.apache.org/r/59095/diff/1/?file=1711501#file1711501line351> > > > > Let's go with option 3. > > We shouldn't rely on the UI to know about intricacies of Hadoop. > > > > > > > > ZOOKEEPER > > > > > > hdfs-site > > dfs.nameservices > > > > > > > > > > Tim Thorpe wrote: > I'm not exactly sure what your worry is here. Adding this new section > seems like duplication of the dependency that is listed at the component > level. The Add Services UI shouldn't need to know about the conditional > service depencency because the property should never be set in a way to > require the service to be installed. In this case, dfs.nameservices > shouldn't exist in hdfs-site. The configuration validation will keep users > from adding that setting until Zookeeper is installed (at least as soon as I > implement https://issues.apache.org/jira/browse/AMBARI-20993). > > I can definitely add in this information in the metainfo.xml, I'm just > not sure where it would actually get used. > > Maybe it would be better to introduce a simpler section like: > > > ZOOKEEPER > > > In this way the Add Services UI, can use that list to suggest but not > enforce the selection of the recommended services. Let me know what you > think. > > Alejandro Fernandez wrote: > Hi Tim, my worry is that by removing ZK as a dependency of HDFS, are are > relying on UI logic to provide that dependency. The practice of letting the > UI know about any complexity of Hadoop defeats the purpose of having configs > & dependencies be stack-driven, especially since any such information can > change between stacks and versions. > Yes, HDFS and ZK are a pretty simple scenario, but you get the idea. Hi Alejandro, I understand the general worry but I'm trying to figure out which UI page in particular you are worried about. In terms of during installation, the service selection page shouldn't be a problem. None of the configurations have been set yet at that point. The service advisor should be able to recommend the proper settings based on which services are installed. I don't see how this can be a problem for the Add Services page since the configuration properties should not be set in a manner to require the service. In general I can't set configuration X to value Y if component Z is not installed. This also wouldn't be a problem with the configuration pages because those would continue to use the same mechanism to validate the configuration properties. - Tim ------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59095/#review174676 --- On May 9, 2017, 2:39 p.m., Tim Thorpe wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/59095/ > --- > > (Updated May 9, 2017, 2:39 p.m.) > > > Review request for Ambari, Alejandro Fernandez, Di Li, Jonathan Hurley, > Jayush Luniya, and Nate Cole. > > > Bugs: AMBARI-20967 > https://issues.apache.org/jira/browse/AMBARI-20967 > > > Repository: ambari > > > Description > --- > > HDFS has a static dependency on Zookeeper even though it is only needed when > NameNode HA is enabled. The check for Zookeeper should be done before > enabling HA rather than enforcing it at installation time. > > > Diffs > - > > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml > 3026938 > > > Diff: https://reviews.apache.org/r/59095/diff/1/ > > > Testing > --- > > mvn clean test > > Manual testing - installed HDFS only cluster and attempted to enable NameNode > HA. This is prevented with the same error you'd see if ZooKeeper was enabled > but you had fewer than 3 ZooKeeper servers configured. > > > Thanks, > > Tim Thorpe > >
Re: Review Request 59095: AMBARI-20967 - Remove Zookeeper as a required service from HDFS
> On May 11, 2017, 4:35 p.m., Alejandro Fernandez wrote: > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml > > Line 351 (original) > > <https://reviews.apache.org/r/59095/diff/1/?file=1711501#file1711501line351> > > > > Let's go with option 3. > > We shouldn't rely on the UI to know about intricacies of Hadoop. > > > > > > > > ZOOKEEPER > > > > > > hdfs-site > > dfs.nameservices > > > > > > > > I'm not exactly sure what your worry is here. Adding this new section seems like duplication of the dependency that is listed at the component level. The Add Services UI shouldn't need to know about the conditional service depencency because the property should never be set in a way to require the service to be installed. In this case, dfs.nameservices shouldn't exist in hdfs-site. The configuration validation will keep users from adding that setting until Zookeeper is installed (at least as soon as I implement https://issues.apache.org/jira/browse/AMBARI-20993). I can definitely add in this information in the metainfo.xml, I'm just not sure where it would actually get used. Maybe it would be better to introduce a simpler section like: ZOOKEEPER In this way the Add Services UI, can use that list to suggest but not enforce the selection of the recommended services. Let me know what you think. - Tim --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59095/#review174676 --- On May 9, 2017, 2:39 p.m., Tim Thorpe wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/59095/ > --- > > (Updated May 9, 2017, 2:39 p.m.) > > > Review request for Ambari, Alejandro Fernandez, Di Li, Jonathan Hurley, > Jayush Luniya, and Nate Cole. > > > Bugs: AMBARI-20967 > https://issues.apache.org/jira/browse/AMBARI-20967 > > > Repository: ambari > > > Description > --- > > HDFS has a static dependency on Zookeeper even though it is only needed when > NameNode HA is enabled. The check for Zookeeper should be done before > enabling HA rather than enforcing it at installation time. > > > Diffs > - > > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml > 3026938 > > > Diff: https://reviews.apache.org/r/59095/diff/1/ > > > Testing > --- > > mvn clean test > > Manual testing - installed HDFS only cluster and attempted to enable NameNode > HA. This is prevented with the same error you'd see if ZooKeeper was enabled > but you had fewer than 3 ZooKeeper servers configured. > > > Thanks, > > Tim Thorpe > >
Re: Review Request 59095: AMBARI-20967 - Remove Zookeeper as a required service from HDFS
> On May 9, 2017, 5:18 p.m., Alejandro Fernandez wrote: > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml > > Line 351 (original) > > <https://reviews.apache.org/r/59095/diff/1/?file=1711501#file1711501line351> > > > > How do we capture this dependency only when HA is enabled? > > Tim Thorpe wrote: > There are several options: > > 1) We don't. The UI already has an explicit check for ZOOKEEPER when > attempting to enable NameNode HA. This is the option I implemented. > 2) We could add in a conditional dependency section. > > > ZOOKEEPER > > > 3) We could add the conditional dependency section but be more explicit > about how/when it applies: > > > > ZOOKEEPER > > > hdfs-site > dfs.nameservices > > > > > > 4) We add this into the stack/service advisor code. Either this could be > added by using the information from 3 above or by hard coding it. By hard > coding it, I mean that during configuration validation we'd check to see if > the dfs.nameservices exists and if so whether zookeeper is installed. > > Mostly I opened this JIRA and review board to get opinions on the issue > and possible solutions. > > As a side note, I just attempted to add YARN to my HDFS only cluster and > noticed that there were 2 configuration issues because > yarn.resourcemanager.zk-address and hadoop.registry.zk.quorum are both blank. > Both of these settings should be fine so long as > yarn.resourcemanager.ha.enabled and hadoop.registry.rm.enabled respectively > are set to false. YARN doesn't specify a dependency on ZOOKEEPER because it > has a dependency on HDFS and relies on HDFS's dependency on ZOOKEEPER. > > Alejandro Fernandez wrote: > Hi Tim, I think this should be stack-driven. I like option #3 After looking at this a little more. It seems like the actual conditional dependency is already declared for the NameNode. https://github.ibm.com/BigInsights/ambari/blob/trunk/ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml#L49 ZOOKEEPER/ZOOKEEPER_SERVER host false hdfs-site dfs.nameservices The only thing that is really missing is the enforcement at the stack/service advisor level. The initial implementation for enforcing dependencies in the stack/service advisor started with https://issues.apache.org/jira/browse/AMBARI-19685. I created another JIRA to continue the word to handle conditional dependencies: https://issues.apache.org/jira/browse/AMBARI-20993 I opened another JIRA to fix the scope of the dependencies which mistakenly have them set to host rather than cluster scope: https://issues.apache.org/jira/browse/AMBARI-20992 - Tim --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59095/#review174341 --- On May 9, 2017, 2:39 p.m., Tim Thorpe wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/59095/ > --- > > (Updated May 9, 2017, 2:39 p.m.) > > > Review request for Ambari, Alejandro Fernandez, Di Li, Jonathan Hurley, > Jayush Luniya, and Nate Cole. > > > Bugs: AMBARI-20967 > https://issues.apache.org/jira/browse/AMBARI-20967 > > > Repository: ambari > > > Description > --- > > HDFS has a static dependency on Zookeeper even though it is only needed when > NameNode HA is enabled. The check for Zookeeper should be done before > enabling HA rather than enforcing it at installation time. > > > Diffs > - > > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml > 3026938 > > > Diff: https://reviews.apache.org/r/59095/diff/1/ > > > Testing > --- > > mvn clean test > > Manual testing - installed HDFS only cluster and attempted to enable NameNode > HA. This is prevented with the same error you'd see if ZooKeeper was enabled > but you had fewer than 3 ZooKeeper servers configured. > > > Thanks, > > Tim Thorpe > >
Re: Review Request 59095: AMBARI-20967 - Remove Zookeeper as a required service from HDFS
> On May 9, 2017, 5:18 p.m., Alejandro Fernandez wrote: > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml > > Line 351 (original) > > <https://reviews.apache.org/r/59095/diff/1/?file=1711501#file1711501line351> > > > > How do we capture this dependency only when HA is enabled? There are several options: 1) We don't. The UI already has an explicit check for ZOOKEEPER when attempting to enable NameNode HA. This is the option I implemented. 2) We could add in a conditional dependency section. ZOOKEEPER 3) We could add the conditional dependency section but be more explicit about how/when it applies: ZOOKEEPER hdfs-site dfs.nameservices 4) We add this into the stack/service advisor code. Either this could be added by using the information from 3 above or by hard coding it. By hard coding it, I mean that during configuration validation we'd check to see if the dfs.nameservices exists and if so whether zookeeper is installed. Mostly I opened this JIRA and review board to get opinions on the issue and possible solutions. As a side note, I just attempted to add YARN to my HDFS only cluster and noticed that there were 2 configuration issues because yarn.resourcemanager.zk-address and hadoop.registry.zk.quorum are both blank. Both of these settings should be fine so long as yarn.resourcemanager.ha.enabled and hadoop.registry.rm.enabled respectively are set to false. YARN doesn't specify a dependency on ZOOKEEPER because it has a dependency on HDFS and relies on HDFS's dependency on ZOOKEEPER. - Tim --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59095/#review174341 ------- On May 9, 2017, 2:39 p.m., Tim Thorpe wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/59095/ > --- > > (Updated May 9, 2017, 2:39 p.m.) > > > Review request for Ambari, Alejandro Fernandez, Di Li, Jonathan Hurley, > Jayush Luniya, and Nate Cole. > > > Bugs: AMBARI-20967 > https://issues.apache.org/jira/browse/AMBARI-20967 > > > Repository: ambari > > > Description > --- > > HDFS has a static dependency on Zookeeper even though it is only needed when > NameNode HA is enabled. The check for Zookeeper should be done before > enabling HA rather than enforcing it at installation time. > > > Diffs > - > > > ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml > 3026938 > > > Diff: https://reviews.apache.org/r/59095/diff/1/ > > > Testing > --- > > mvn clean test > > Manual testing - installed HDFS only cluster and attempted to enable NameNode > HA. This is prevented with the same error you'd see if ZooKeeper was enabled > but you had fewer than 3 ZooKeeper servers configured. > > > Thanks, > > Tim Thorpe > >
Review Request 59095: AMBARI-20967 - Remove Zookeeper as a required service from HDFS
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59095/ --- Review request for Ambari, Alejandro Fernandez, Di Li, Jonathan Hurley, Jayush Luniya, and Nate Cole. Bugs: AMBARI-20967 https://issues.apache.org/jira/browse/AMBARI-20967 Repository: ambari Description --- HDFS has a static dependency on Zookeeper even though it is only needed when NameNode HA is enabled. The check for Zookeeper should be done before enabling HA rather than enforcing it at installation time. Diffs - ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml 3026938 Diff: https://reviews.apache.org/r/59095/diff/1/ Testing --- mvn clean test Manual testing - installed HDFS only cluster and attempted to enable NameNode HA. This is prevented with the same error you'd see if ZooKeeper was enabled but you had fewer than 3 ZooKeeper servers configured. Thanks, Tim Thorpe
Re: Review Request 59053: Review for AMBARI-20946
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59053/#review174188 --- Ship it! Ship It! - Tim Thorpe On May 8, 2017, 4:31 p.m., Diego Santesteban wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/59053/ > --- > > (Updated May 8, 2017, 4:31 p.m.) > > > Review request for Ambari and Tim Thorpe. > > > Bugs: AMBARI-20946 > https://issues.apache.org/jira/browse/AMBARI-20946 > > > Repository: ambari > > > Description > --- > > I added an extra case in resourceFilesKeeper.py to recreate the archive.zip > file if it doesn't exist, even when the saved and current directory hashes > are matching. > > I added one new test case to make sure this case is working, and modified one > existing test case to make it specific for the other case (where the > archive.zip file does exist and the hashes match). > > > Diffs > - > > ambari-server/src/main/python/ambari_server/resourceFilesKeeper.py > dba6833924 > ambari-server/src/test/python/TestResourceFilesKeeper.py 4f7dc525d3 > > > Diff: https://reviews.apache.org/r/59053/diff/1/ > > > Testing > --- > > I created a new test case and successfully ran all tests with 'mvn clean > test'. > > I also applied this change to an existing cluster to ensure that it does fix > the issue in a real usage case. > > > Thanks, > > Diego Santesteban > >
Re: Review Request 59053: Review for AMBARI-20946
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/59053/#review174187 --- ambari-server/src/main/python/ambari_server/resourceFilesKeeper.py Line 168 (original), 168 (patched) <https://reviews.apache.org/r/59053/#comment247309> Should add a comment in here saying the archive will be created if it doesn't exist even if there is a hash file with equal to the directories hash. - Tim Thorpe On May 8, 2017, 4:31 p.m., Diego Santesteban wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/59053/ > --- > > (Updated May 8, 2017, 4:31 p.m.) > > > Review request for Ambari and Tim Thorpe. > > > Bugs: AMBARI-20946 > https://issues.apache.org/jira/browse/AMBARI-20946 > > > Repository: ambari > > > Description > --- > > I added an extra case in resourceFilesKeeper.py to recreate the archive.zip > file if it doesn't exist, even when the saved and current directory hashes > are matching. > > I added one new test case to make sure this case is working, and modified one > existing test case to make it specific for the other case (where the > archive.zip file does exist and the hashes match). > > > Diffs > - > > ambari-server/src/main/python/ambari_server/resourceFilesKeeper.py > dba6833924 > ambari-server/src/test/python/TestResourceFilesKeeper.py 4f7dc525d3 > > > Diff: https://reviews.apache.org/r/59053/diff/1/ > > > Testing > --- > > I created a new test case and successfully ran all tests with 'mvn clean > test'. > > I also applied this change to an existing cluster to ensure that it does fix > the issue in a real usage case. > > > Thanks, > > Diego Santesteban > >
Re: Review Request 58845: AMBARI-20891 - Allow extensions to auto-link with supported stack versions
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58845/ --- (Updated May 5, 2017, 6:30 p.m.) Review request for Ambari, Alejandro Fernandez, Di Li, Jonathan Hurley, Jayush Luniya, and Nate Cole. Bugs: AMBARI-20891 https://issues.apache.org/jira/browse/AMBARI-20891 Repository: ambari Description --- It would possible to link extensions to supported stack versions while parsing the stacks, extensions and common-services directories. This would allow extensions to avoid making rest API calls to set up the link. Diffs - ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java 751ce08 ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementHelper.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/orm/dao/ExtensionLinkDAO.java 3df1cd5 ambari-server/src/main/java/org/apache/ambari/server/orm/entities/ExtensionLinkEntity.java 12b3ce0 ambari-server/src/main/java/org/apache/ambari/server/stack/ExtensionModule.java ce92ecf ambari-server/src/main/java/org/apache/ambari/server/stack/StackManager.java 20e28fc ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java d3ad351 ambari-server/src/main/java/org/apache/ambari/server/state/ExtensionInfo.java 656a8ce ambari-server/src/main/java/org/apache/ambari/server/state/StackInfo.java 353dd86 ambari-server/src/main/java/org/apache/ambari/server/state/stack/ExtensionMetainfoXml.java 77a7250 ambari-server/src/test/java/org/apache/ambari/server/stack/StackManagerCommonServicesTest.java 4e20c2b ambari-server/src/test/java/org/apache/ambari/server/stack/StackManagerExtensionTest.java 9900eb5 ambari-server/src/test/java/org/apache/ambari/server/stack/StackManagerMiscTest.java ca24cd9 ambari-server/src/test/java/org/apache/ambari/server/stack/StackManagerMock.java 0fe1573 ambari-server/src/test/java/org/apache/ambari/server/stack/StackManagerTest.java d8c02dc ambari-server/src/test/resources/extensions/EXT/0.1/metainfo.xml 77a832c ambari-server/src/test/resources/extensions/EXT/0.2/metainfo.xml 04f733c ambari-server/src/test/resources/extensions/EXT/0.3/metainfo.xml PRE-CREATION ambari-server/src/test/resources/extensions/EXT/0.3/services/OOZIE2/metainfo.xml PRE-CREATION ambari-server/src/test/resources/extensions/EXT/0.3/services/OOZIE2/themes/broken_theme.json PRE-CREATION ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/metainfo.xml PRE-CREATION ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/repos/repoinfo.xml PRE-CREATION ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/services/HBASE/metainfo.xml PRE-CREATION ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/services/HDFS/configuration/global.xml PRE-CREATION ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/services/HDFS/configuration/hadoop-env.xml PRE-CREATION ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/services/HDFS/configuration/hbase-site.xml PRE-CREATION ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/services/HDFS/configuration/hdfs-log4j.xml PRE-CREATION ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/services/HDFS/configuration/hdfs-site.xml PRE-CREATION ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/services/HDFS/metainfo.xml PRE-CREATION ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/services/HDFS/package/dummy-script.py PRE-CREATION ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/services/HIVE/metainfo.xml PRE-CREATION ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/services/MAPREDUCE/metainfo.xml PRE-CREATION ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/services/ZOOKEEPER/metainfo.xml PRE-CREATION Diff: https://reviews.apache.org/r/58845/diff/2/ Testing (updated) --- mvn clean test mvn clean test -Dtest=StackManagerExtensionTest -DskipPythonTests mvn clean test -Dtest=StackManagerExtensionTest,StackManagerCommonServicesTest,StackManagerMiscTest,StackManagerTest,AmbariManagementControllerImplTest,AmbariManagementControllerTest -DskipPythonTests Thanks, Tim Thorpe
Re: Review Request 58845: AMBARI-20891 - Allow extensions to auto-link with supported stack versions
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58845/ --- (Updated May 5, 2017, 6:26 p.m.) Review request for Ambari, Alejandro Fernandez, Di Li, Jonathan Hurley, Jayush Luniya, and Nate Cole. Changes --- Fixed issue reported by Jonathan Hurley Bugs: AMBARI-20891 https://issues.apache.org/jira/browse/AMBARI-20891 Repository: ambari Description --- It would possible to link extensions to supported stack versions while parsing the stacks, extensions and common-services directories. This would allow extensions to avoid making rest API calls to set up the link. Diffs (updated) - ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java 751ce08 ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementHelper.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/orm/dao/ExtensionLinkDAO.java 3df1cd5 ambari-server/src/main/java/org/apache/ambari/server/orm/entities/ExtensionLinkEntity.java 12b3ce0 ambari-server/src/main/java/org/apache/ambari/server/stack/ExtensionModule.java ce92ecf ambari-server/src/main/java/org/apache/ambari/server/stack/StackManager.java 20e28fc ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java d3ad351 ambari-server/src/main/java/org/apache/ambari/server/state/ExtensionInfo.java 656a8ce ambari-server/src/main/java/org/apache/ambari/server/state/StackInfo.java 353dd86 ambari-server/src/main/java/org/apache/ambari/server/state/stack/ExtensionMetainfoXml.java 77a7250 ambari-server/src/test/java/org/apache/ambari/server/stack/StackManagerCommonServicesTest.java 4e20c2b ambari-server/src/test/java/org/apache/ambari/server/stack/StackManagerExtensionTest.java 9900eb5 ambari-server/src/test/java/org/apache/ambari/server/stack/StackManagerMiscTest.java ca24cd9 ambari-server/src/test/java/org/apache/ambari/server/stack/StackManagerMock.java 0fe1573 ambari-server/src/test/java/org/apache/ambari/server/stack/StackManagerTest.java d8c02dc ambari-server/src/test/resources/extensions/EXT/0.1/metainfo.xml 77a832c ambari-server/src/test/resources/extensions/EXT/0.2/metainfo.xml 04f733c ambari-server/src/test/resources/extensions/EXT/0.3/metainfo.xml PRE-CREATION ambari-server/src/test/resources/extensions/EXT/0.3/services/OOZIE2/metainfo.xml PRE-CREATION ambari-server/src/test/resources/extensions/EXT/0.3/services/OOZIE2/themes/broken_theme.json PRE-CREATION ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/metainfo.xml PRE-CREATION ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/repos/repoinfo.xml PRE-CREATION ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/services/HBASE/metainfo.xml PRE-CREATION ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/services/HDFS/configuration/global.xml PRE-CREATION ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/services/HDFS/configuration/hadoop-env.xml PRE-CREATION ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/services/HDFS/configuration/hbase-site.xml PRE-CREATION ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/services/HDFS/configuration/hdfs-log4j.xml PRE-CREATION ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/services/HDFS/configuration/hdfs-site.xml PRE-CREATION ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/services/HDFS/metainfo.xml PRE-CREATION ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/services/HDFS/package/dummy-script.py PRE-CREATION ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/services/HIVE/metainfo.xml PRE-CREATION ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/services/MAPREDUCE/metainfo.xml PRE-CREATION ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/services/ZOOKEEPER/metainfo.xml PRE-CREATION Diff: https://reviews.apache.org/r/58845/diff/2/ Changes: https://reviews.apache.org/r/58845/diff/1-2/ Testing --- mvn clean test mvn clean test -Dtest=StackManagerExtensionTest -DskipPythonTests Thanks, Tim Thorpe
Re: Review Request 58845: AMBARI-20891 - Allow extensions to auto-link with supported stack versions
> On May 4, 2017, 3:07 p.m., Jonathan Hurley wrote: > > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementHelper.java > > Lines 43 (patched) > > <https://reviews.apache.org/r/58845/diff/1/?file=1703102#file1703102line43> > > > > This class can be made a Singleton and be injected as a dependency. I was kind of debatting removing all the dao parameters to the StackManager constructor and move some of the code that uses them to the helper... Haven't done that so far. - Tim --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58845/#review173909 ------- On April 28, 2017, 5:48 p.m., Tim Thorpe wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/58845/ > --- > > (Updated April 28, 2017, 5:48 p.m.) > > > Review request for Ambari, Alejandro Fernandez, Di Li, Jonathan Hurley, > Jayush Luniya, and Nate Cole. > > > Bugs: AMBARI-20891 > https://issues.apache.org/jira/browse/AMBARI-20891 > > > Repository: ambari > > > Description > --- > > It would possible to link extensions to supported stack versions while > parsing the stacks, extensions and common-services directories. > > This would allow extensions to avoid making rest API calls to set up the link. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java > 751ce08 > > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementHelper.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/orm/dao/ExtensionLinkDAO.java > 3df1cd5 > > ambari-server/src/main/java/org/apache/ambari/server/orm/entities/ExtensionLinkEntity.java > 12b3ce0 > > ambari-server/src/main/java/org/apache/ambari/server/stack/ExtensionModule.java > ce92ecf > > ambari-server/src/main/java/org/apache/ambari/server/stack/StackManager.java > 20e28fc > ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java > d3ad351 > > ambari-server/src/main/java/org/apache/ambari/server/state/ExtensionInfo.java > 656a8ce > ambari-server/src/main/java/org/apache/ambari/server/state/StackInfo.java > 353dd86 > > ambari-server/src/main/java/org/apache/ambari/server/state/stack/ExtensionMetainfoXml.java > 77a7250 > > ambari-server/src/test/java/org/apache/ambari/server/stack/StackManagerExtensionTest.java > 9900eb5 > ambari-server/src/test/resources/extensions/EXT/0.1/metainfo.xml 77a832c > ambari-server/src/test/resources/extensions/EXT/0.2/metainfo.xml 04f733c > ambari-server/src/test/resources/extensions/EXT/0.3/metainfo.xml > PRE-CREATION > > ambari-server/src/test/resources/extensions/EXT/0.3/services/OOZIE2/metainfo.xml > PRE-CREATION > > ambari-server/src/test/resources/extensions/EXT/0.3/services/OOZIE2/themes/broken_theme.json > PRE-CREATION > > ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/metainfo.xml > PRE-CREATION > > ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/repos/repoinfo.xml > PRE-CREATION > > ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/services/HBASE/metainfo.xml > PRE-CREATION > > ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/services/HDFS/configuration/global.xml > PRE-CREATION > > ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/services/HDFS/configuration/hadoop-env.xml > PRE-CREATION > > ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/services/HDFS/configuration/hbase-site.xml > PRE-CREATION > > ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/services/HDFS/configuration/hdfs-log4j.xml > PRE-CREATION > > ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/services/HDFS/configuration/hdfs-site.xml > PRE-CREATION > > ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/services/HDFS/metainfo.xml > PRE-CREATION > > ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/services/HDFS/package/dummy-script.py > PRE-CREATION > > ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/services/HIVE/metainfo.xml > PRE-CREATION > > ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/services/MAPREDUCE/metainfo.xml > PRE-CREATION > > ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/services/ZOOKEEPER/metainfo.xml > PRE-CREATION > > > Diff: https://reviews.apache.org/r/58845/diff/1/ > > > Testing > --- > > mvn clean test > > mvn clean test -Dtest=StackManagerExtensionTest -DskipPythonTests > > > Thanks, > > Tim Thorpe > >
Re: Review Request 58845: AMBARI-20891 - Allow extensions to auto-link with supported stack versions
> On May 4, 2017, 3:07 p.m., Jonathan Hurley wrote: > > ambari-server/src/main/java/org/apache/ambari/server/stack/StackManager.java > > Line 229 (original), 240-263 (patched) > > <https://reviews.apache.org/r/58845/diff/1/?file=1703106#file1703106line240> > > > > Documentation on this. Also, what about the exception being logged if > > the extension could not be created. Should this be thrown as well? During auto linking, I don't want to throw an exception because that would actually stop ambari server from starting. - Tim --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58845/#review173909 ----------- On April 28, 2017, 5:48 p.m., Tim Thorpe wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/58845/ > --- > > (Updated April 28, 2017, 5:48 p.m.) > > > Review request for Ambari, Alejandro Fernandez, Di Li, Jonathan Hurley, > Jayush Luniya, and Nate Cole. > > > Bugs: AMBARI-20891 > https://issues.apache.org/jira/browse/AMBARI-20891 > > > Repository: ambari > > > Description > --- > > It would possible to link extensions to supported stack versions while > parsing the stacks, extensions and common-services directories. > > This would allow extensions to avoid making rest API calls to set up the link. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java > 751ce08 > > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementHelper.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/orm/dao/ExtensionLinkDAO.java > 3df1cd5 > > ambari-server/src/main/java/org/apache/ambari/server/orm/entities/ExtensionLinkEntity.java > 12b3ce0 > > ambari-server/src/main/java/org/apache/ambari/server/stack/ExtensionModule.java > ce92ecf > > ambari-server/src/main/java/org/apache/ambari/server/stack/StackManager.java > 20e28fc > ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java > d3ad351 > > ambari-server/src/main/java/org/apache/ambari/server/state/ExtensionInfo.java > 656a8ce > ambari-server/src/main/java/org/apache/ambari/server/state/StackInfo.java > 353dd86 > > ambari-server/src/main/java/org/apache/ambari/server/state/stack/ExtensionMetainfoXml.java > 77a7250 > > ambari-server/src/test/java/org/apache/ambari/server/stack/StackManagerExtensionTest.java > 9900eb5 > ambari-server/src/test/resources/extensions/EXT/0.1/metainfo.xml 77a832c > ambari-server/src/test/resources/extensions/EXT/0.2/metainfo.xml 04f733c > ambari-server/src/test/resources/extensions/EXT/0.3/metainfo.xml > PRE-CREATION > > ambari-server/src/test/resources/extensions/EXT/0.3/services/OOZIE2/metainfo.xml > PRE-CREATION > > ambari-server/src/test/resources/extensions/EXT/0.3/services/OOZIE2/themes/broken_theme.json > PRE-CREATION > > ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/metainfo.xml > PRE-CREATION > > ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/repos/repoinfo.xml > PRE-CREATION > > ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/services/HBASE/metainfo.xml > PRE-CREATION > > ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/services/HDFS/configuration/global.xml > PRE-CREATION > > ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/services/HDFS/configuration/hadoop-env.xml > PRE-CREATION > > ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/services/HDFS/configuration/hbase-site.xml > PRE-CREATION > > ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/services/HDFS/configuration/hdfs-log4j.xml > PRE-CREATION > > ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/services/HDFS/configuration/hdfs-site.xml > PRE-CREATION > > ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/services/HDFS/metainfo.xml > PRE-CREATION > > ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/services/HDFS/package/dummy-script.py > PRE-CREATION > > ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/services/HIVE/metainfo.xml > PRE-CREATION > > ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/services/MAPREDUCE/metainfo.xml > PRE-CREATION > > ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/services/ZOOKEEPER/metainfo.xml > PRE-CREATION > > > Diff: https://reviews.apache.org/r/58845/diff/1/ > > > Testing > --- > > mvn clean test > > mvn clean test -Dtest=StackManagerExtensionTest -DskipPythonTests > > > Thanks, > > Tim Thorpe > >
Re: Review Request 58968: Server Error in Ambari UI, when trying to login as a pam user due to user name conflict
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58968/#review173791 --- Ship it! Ship It! - Tim Thorpe On May 3, 2017, 6:24 p.m., Anita Jebaraj wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/58968/ > --- > > (Updated May 3, 2017, 6:24 p.m.) > > > Review request for Ambari, Attila Doroszlai, Di Li, Robert Levas, and Tim > Thorpe. > > > Bugs: AMBARI-20909 > https://issues.apache.org/jira/browse/AMBARI-20909 > > > Repository: ambari > > > Description > --- > > Create a local user "test" in Ambari > > Create a system user "test" with different password > > Trying to authenticate via pam in Ambari UI as user "test" throws Server > Error in Ambari UI, without any error in Ambari-server logs > > Also the UI gets stalled and not even able to login as admin user unless the > browser cache is removed or Ambari UI is opened in a new browser page > > Ambari doesn't allow creating users with same user name but different > types(Local/pam), We reach the pam authentication only when the local user > authentication failed due to non-existing userid or incorrect password, So if > local user exists do not attempt to authenticate via PAM, This lets Ambari to > avoid importing duplicate userid into the database > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/security/authorization/AmbariPamAuthenticationProvider.java > b3fb861 > > ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AmbariPamAuthenticationProviderTest.java > b7272c5 > > > Diff: https://reviews.apache.org/r/58968/diff/2/ > > > Testing > --- > > Updated the related test cases > > > Thanks, > > Anita Jebaraj > >
Re: Review Request 58968: Server Error in Ambari UI, when trying to login as a pam user due to user name conflict
> On May 3, 2017, 5:08 p.m., Robert Levas wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/security/authorization/AmbariPamAuthenticationProvider.java
> > Lines 79 (patched)
> > <https://reviews.apache.org/r/58968/diff/1/?file=1707168#file1707168line79>
> >
> > There are other User types - LDAP and JWT. You should check to see if
> > any user exists with the username. If the existing user is not a PAM user
> > the failure should occur.
> >
> > For example:
> > ```
> > UserEntitiy foundUser = userDAO.findUserByName(username)'
> > if((foundUser != null) && (foundUser.getUserType != UserType.PAM)) {
> > ... Fail ...
> > }
> > ```
>
> Tim Thorpe wrote:
> Hi Robert, I'm not sure I'm following your logic here. My interpretation
> is that the PAM users will not be found using the
> UserDAO.findLocalUserByName(String userName) method. So basically the
> getUserType() != UserType.PAM is unnecessary. Although it wouldn't hurt.
>
> Anita Jebaraj wrote:
> Hi Tim, Robert means that I should look for all user types like
> (Local/ldap/jwt), I am aware ldap cannot be enabled when pam is enabled, but
> jwt users can exist when pam is enabled. That should be the reason why he
> wants me to look for all the users.
Sorry missed the part where you changed findLocalUserByName to findUserByName.
That makes sense now.
- Tim
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58968/#review173752
---
On May 3, 2017, 4:48 p.m., Anita Jebaraj wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58968/
> -------
>
> (Updated May 3, 2017, 4:48 p.m.)
>
>
> Review request for Ambari, Attila Doroszlai, Di Li, Robert Levas, and Tim
> Thorpe.
>
>
> Bugs: AMBARI-20909
> https://issues.apache.org/jira/browse/AMBARI-20909
>
>
> Repository: ambari
>
>
> Description
> ---
>
> Create a local user "test" in Ambari
>
> Create a system user "test" with different password
>
> Trying to authenticate via pam in Ambari UI as user "test" throws Server
> Error in Ambari UI, without any error in Ambari-server logs
>
> Also the UI gets stalled and not even able to login as admin user unless the
> browser cache is removed or Ambari UI is opened in a new browser page
>
> Ambari doesn't allow creating users with same user name but different
> types(Local/pam), We reach the pam authentication only when the local user
> authentication failed due to non-existing userid or incorrect password, So if
> local user exists do not attempt to authenticate via PAM, This lets Ambari to
> avoid importing duplicate userid into the database
>
>
> Diffs
> -
>
>
> ambari-server/src/main/java/org/apache/ambari/server/security/authorization/AmbariPamAuthenticationProvider.java
> b3fb861
>
> ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AmbariPamAuthenticationProviderTest.java
> b7272c5
>
>
> Diff: https://reviews.apache.org/r/58968/diff/1/
>
>
> Testing
> ---
>
> Updated the related test cases
>
>
> Thanks,
>
> Anita Jebaraj
>
>
Re: Review Request 58968: Server Error in Ambari UI, when trying to login as a pam user due to user name conflict
> On May 3, 2017, 5:08 p.m., Robert Levas wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/security/authorization/AmbariPamAuthenticationProvider.java
> > Lines 79 (patched)
> > <https://reviews.apache.org/r/58968/diff/1/?file=1707168#file1707168line79>
> >
> > There are other User types - LDAP and JWT. You should check to see if
> > any user exists with the username. If the existing user is not a PAM user
> > the failure should occur.
> >
> > For example:
> > ```
> > UserEntitiy foundUser = userDAO.findUserByName(username)'
> > if((foundUser != null) && (foundUser.getUserType != UserType.PAM)) {
> > ... Fail ...
> > }
> > ```
Hi Robert, I'm not sure I'm following your logic here. My interpretation is
that the PAM users will not be found using the
UserDAO.findLocalUserByName(String userName) method. So basically the
getUserType() != UserType.PAM is unnecessary. Although it wouldn't hurt.
- Tim
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58968/#review173752
---
On May 3, 2017, 4:48 p.m., Anita Jebaraj wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58968/
> -------
>
> (Updated May 3, 2017, 4:48 p.m.)
>
>
> Review request for Ambari, Attila Doroszlai, Di Li, Robert Levas, and Tim
> Thorpe.
>
>
> Bugs: AMBARI-20909
> https://issues.apache.org/jira/browse/AMBARI-20909
>
>
> Repository: ambari
>
>
> Description
> ---
>
> Create a local user "test" in Ambari
>
> Create a system user "test" with different password
>
> Trying to authenticate via pam in Ambari UI as user "test" throws Server
> Error in Ambari UI, without any error in Ambari-server logs
>
> Also the UI gets stalled and not even able to login as admin user unless the
> browser cache is removed or Ambari UI is opened in a new browser page
>
> Ambari doesn't allow creating users with same user name but different
> types(Local/pam), We reach the pam authentication only when the local user
> authentication failed due to non-existing userid or incorrect password, So if
> local user exists do not attempt to authenticate via PAM, This lets Ambari to
> avoid importing duplicate userid into the database
>
>
> Diffs
> -
>
>
> ambari-server/src/main/java/org/apache/ambari/server/security/authorization/AmbariPamAuthenticationProvider.java
> b3fb861
>
> ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AmbariPamAuthenticationProviderTest.java
> b7272c5
>
>
> Diff: https://reviews.apache.org/r/58968/diff/1/
>
>
> Testing
> ---
>
> Updated the related test cases
>
>
> Thanks,
>
> Anita Jebaraj
>
>
Re: Review Request 58919: AMBARI-20903 RepoVersion parsing error should be more specific
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58919/#review173598 --- Ship it! Ship It! - Tim Thorpe On May 2, 2017, 3:37 p.m., Di Li wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/58919/ > --- > > (Updated May 2, 2017, 3:37 p.m.) > > > Review request for Ambari and Tim Thorpe. > > > Bugs: AMBARI-20903 > https://issues.apache.org/jira/browse/AMBARI-20903 > > > Repository: ambari > > > Description > --- > > This is a minor JIRA on updating an error message in case the repoinfo.xml > misses any of the three required fields - repo_name, repo_id, and base url. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/orm/entities/RepositoryVersionEntity.java > d3705f3 > > > Diff: https://reviews.apache.org/r/58919/diff/1/ > > > Testing > --- > > existing unit tests. patch a cluster with the change. > > > Thanks, > > Di Li > >
Re: Review Request 58429: After pam setup- Hive View user home test fails
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58429/#review173354
---
Ship it!
Ship It!
- Tim Thorpe
On April 28, 2017, 5:45 p.m., Anita Jebaraj wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/58429/
> ---
>
> (Updated April 28, 2017, 5:45 p.m.)
>
>
> Review request for Ambari, Attila Doroszlai, Di Li, Vishal Ghugare, Henning
> Kropp, Robert Levas, and Tim Thorpe.
>
>
> Bugs: AMBARI-20760
> https://issues.apache.org/jira/browse/AMBARI-20760
>
>
> Repository: ambari
>
>
> Description
> ---
>
> After setting up PAM, tried to login as PAM user and access hive view, user
> home test fails with the error as in screen shot.
>
> This issue was pointed out by Henning Kropp in the jira AMBARI-12263, but was
> not incorporated in the code. Pasting the comment from Henning below.
>
> Something we noticed is that in a secured cluster we have issues with the
> views, getting the following exception for the Hive view as an example:
>
> Struct:TOpenSessionResp(status:TStatus(statusCode:ERROR_STATUS,
> infoMessages:[*org.apache.hive.service.cli.HiveSQLException:Failed to
> validate proxy privilege of ambari for
> org.apache.ambari.server.security.authorization.AmbariPamAuthenticationProvider$1@3459:33:32,
> .
> sqlState:08S01, errorCode:0, errorMessage:Failed to validate proxy privilege
> of ambari for
> org.apache.ambari.server.security.authorization.AmbariPamAuthenticationProvider$1@3459),
> serverProtocolVersion:null)
>
> As you can see it tries to impersonte
> "org.apache.ambari.server.security.authorization.AmbariPamAuthenticationProvider$1@3459:33:32".
> Changing the UsernamePasswordAuthenticationToken from Principal to username
> fixes this.
>
> So instead of :
>
> UsernamePasswordAuthenticationToken token = new
> UsernamePasswordAuthenticationToken(principal, null, userAuthorities);
>
> We use:
>
> UsernamePasswordAuthenticationToken token = new
> UsernamePasswordAuthenticationToken(user.getUserName(), null,
> userAuthorities);
>
> What could potential also work is, overriding toString of the principal like:
>
> Principal principal = new Principal() {
> @Override
> public String getName()
> { return user.getUserName(); }
>
> @Override
> public String toString()
> { return user.getUserName().toString(); }
>
> };
>
> We did not test this!
>
>
> Diffs
> -
>
>
> ambari-server/src/main/java/org/apache/ambari/server/security/authorization/AmbariPamAuthenticationProvider.java
> ca7cd31
>
>
> Diff: https://reviews.apache.org/r/58429/diff/1/
>
>
> Testing
> ---
>
> Ran mvn test and also manually tested the scenario
>
>
> File Attachments
>
>
> error screenshot
>
> https://reviews.apache.org/media/uploaded/files/2017/04/13/4a43b897-e030-41a7-b702-f711432b03b9__error.PNG
>
>
> Thanks,
>
> Anita Jebaraj
>
>
Re: Review Request 58845: AMBARI-20891 - Allow extensions to auto-link with supported stack versions
> On April 28, 2017, 5:23 p.m., Alejandro Fernandez wrote: > > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java > > Line 5456 (original), 5457 (patched) > > <https://reviews.apache.org/r/58845/diff/1/?file=1703101#file1703101line5461> > > > > Please include Jonathan Hurley, Nate Cole, and Jayush Luniya in this > > review. Added Jonathan and Nate. Jayush was already added. - Tim --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58845/#review173348 ----------- On April 28, 2017, 5:48 p.m., Tim Thorpe wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/58845/ > --- > > (Updated April 28, 2017, 5:48 p.m.) > > > Review request for Ambari, Alejandro Fernandez, Di Li, Jonathan Hurley, > Jayush Luniya, and Nate Cole. > > > Bugs: AMBARI-20891 > https://issues.apache.org/jira/browse/AMBARI-20891 > > > Repository: ambari > > > Description > --- > > It would possible to link extensions to supported stack versions while > parsing the stacks, extensions and common-services directories. > > This would allow extensions to avoid making rest API calls to set up the link. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java > 751ce08 > > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementHelper.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/orm/dao/ExtensionLinkDAO.java > 3df1cd5 > > ambari-server/src/main/java/org/apache/ambari/server/orm/entities/ExtensionLinkEntity.java > 12b3ce0 > > ambari-server/src/main/java/org/apache/ambari/server/stack/ExtensionModule.java > ce92ecf > > ambari-server/src/main/java/org/apache/ambari/server/stack/StackManager.java > 20e28fc > ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java > d3ad351 > > ambari-server/src/main/java/org/apache/ambari/server/state/ExtensionInfo.java > 656a8ce > ambari-server/src/main/java/org/apache/ambari/server/state/StackInfo.java > 353dd86 > > ambari-server/src/main/java/org/apache/ambari/server/state/stack/ExtensionMetainfoXml.java > 77a7250 > > ambari-server/src/test/java/org/apache/ambari/server/stack/StackManagerExtensionTest.java > 9900eb5 > ambari-server/src/test/resources/extensions/EXT/0.1/metainfo.xml 77a832c > ambari-server/src/test/resources/extensions/EXT/0.2/metainfo.xml 04f733c > ambari-server/src/test/resources/extensions/EXT/0.3/metainfo.xml > PRE-CREATION > > ambari-server/src/test/resources/extensions/EXT/0.3/services/OOZIE2/metainfo.xml > PRE-CREATION > > ambari-server/src/test/resources/extensions/EXT/0.3/services/OOZIE2/themes/broken_theme.json > PRE-CREATION > > ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/metainfo.xml > PRE-CREATION > > ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/repos/repoinfo.xml > PRE-CREATION > > ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/services/HBASE/metainfo.xml > PRE-CREATION > > ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/services/HDFS/configuration/global.xml > PRE-CREATION > > ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/services/HDFS/configuration/hadoop-env.xml > PRE-CREATION > > ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/services/HDFS/configuration/hbase-site.xml > PRE-CREATION > > ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/services/HDFS/configuration/hdfs-log4j.xml > PRE-CREATION > > ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/services/HDFS/configuration/hdfs-site.xml > PRE-CREATION > > ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/services/HDFS/metainfo.xml > PRE-CREATION > > ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/services/HDFS/package/dummy-script.py > PRE-CREATION > > ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/services/HIVE/metainfo.xml > PRE-CREATION > > ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/services/MAPREDUCE/metainfo.xml > PRE-CREATION > > ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/services/ZOOKEEPER/metainfo.xml > PRE-CREATION > > > Diff: https://reviews.apache.org/r/58845/diff/1/ > > > Testing > --- > > mvn clean test > > mvn clean test -Dtest=StackManagerExtensionTest -DskipPythonTests > > > Thanks, > > Tim Thorpe > >
Review Request 58845: AMBARI-20891 - Allow extensions to auto-link with supported stack versions
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58845/ --- Review request for Ambari, Alejandro Fernandez, Di Li, and Jayush Luniya. Bugs: AMBARI-20891 https://issues.apache.org/jira/browse/AMBARI-20891 Repository: ambari Description --- It would possible to link extensions to supported stack versions while parsing the stacks, extensions and common-services directories. This would allow extensions to avoid making rest API calls to set up the link. Diffs - ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java 751ce08 ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementHelper.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/orm/dao/ExtensionLinkDAO.java 3df1cd5 ambari-server/src/main/java/org/apache/ambari/server/orm/entities/ExtensionLinkEntity.java 12b3ce0 ambari-server/src/main/java/org/apache/ambari/server/stack/ExtensionModule.java ce92ecf ambari-server/src/main/java/org/apache/ambari/server/stack/StackManager.java 20e28fc ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java d3ad351 ambari-server/src/main/java/org/apache/ambari/server/state/ExtensionInfo.java 656a8ce ambari-server/src/main/java/org/apache/ambari/server/state/StackInfo.java 353dd86 ambari-server/src/main/java/org/apache/ambari/server/state/stack/ExtensionMetainfoXml.java 77a7250 ambari-server/src/test/java/org/apache/ambari/server/stack/StackManagerExtensionTest.java 9900eb5 ambari-server/src/test/resources/extensions/EXT/0.1/metainfo.xml 77a832c ambari-server/src/test/resources/extensions/EXT/0.2/metainfo.xml 04f733c ambari-server/src/test/resources/extensions/EXT/0.3/metainfo.xml PRE-CREATION ambari-server/src/test/resources/extensions/EXT/0.3/services/OOZIE2/metainfo.xml PRE-CREATION ambari-server/src/test/resources/extensions/EXT/0.3/services/OOZIE2/themes/broken_theme.json PRE-CREATION ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/metainfo.xml PRE-CREATION ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/repos/repoinfo.xml PRE-CREATION ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/services/HBASE/metainfo.xml PRE-CREATION ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/services/HDFS/configuration/global.xml PRE-CREATION ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/services/HDFS/configuration/hadoop-env.xml PRE-CREATION ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/services/HDFS/configuration/hbase-site.xml PRE-CREATION ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/services/HDFS/configuration/hdfs-log4j.xml PRE-CREATION ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/services/HDFS/configuration/hdfs-site.xml PRE-CREATION ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/services/HDFS/metainfo.xml PRE-CREATION ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/services/HDFS/package/dummy-script.py PRE-CREATION ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/services/HIVE/metainfo.xml PRE-CREATION ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/services/MAPREDUCE/metainfo.xml PRE-CREATION ambari-server/src/test/resources/stacks_with_extensions/HDP/0.3/services/ZOOKEEPER/metainfo.xml PRE-CREATION Diff: https://reviews.apache.org/r/58845/diff/1/ Testing --- mvn clean test mvn clean test -Dtest=StackManagerExtensionTest -DskipPythonTests Thanks, Tim Thorpe
Re: Review Request 58480: Files View: In non-HA environment, Files view executes with error when "List of NameNodes" property is cleared
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58480/#review173090 --- Ship it! Ship It! - Tim Thorpe On April 26, 2017, 8:18 p.m., Sangeeta Ravindran wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/58480/ > --- > > (Updated April 26, 2017, 8:18 p.m.) > > > Review request for Ambari, Alexandr Antonenko, Di Li, DIPAYAN BHOWMICK, and > Tim Thorpe. > > > Bugs: AMBARI-20354 > https://issues.apache.org/jira/browse/AMBARI-20354 > > > Repository: ambari > > > Description > --- > > 1. In a multi-node cluster where namenode HA is not enabled, create a Files > View. Select Custom Configuration and enter the WebHDFS FileSystem URI*. In > the "List of NameNodes", enter the namenode information. Save and execute > view. There are no errors. > 2. Edit the view and delete the values entered in the "List of NameNodes" > field. Save and execute the view. > The following error is thrown although Namenode HA is not enabled. > Service 'hdfs' check failed: HDFS080 webhdfs.ha.namenodes.list namenodes > count is not exactly 2 > > > Diffs > - > > > contrib/views/utils/src/main/java/org/apache/ambari/view/utils/hdfs/ConfigurationBuilder.java > 714e229 > > > Diff: https://reviews.apache.org/r/58480/diff/2/ > > > Testing > --- > > Manual testing. > > > Thanks, > > Sangeeta Ravindran > >
Re: Review Request 58480: Files View: In non-HA environment, Files view executes with error when "List of NameNodes" property is cleared
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58480/#review173059 --- contrib/views/utils/src/main/java/org/apache/ambari/view/utils/hdfs/ConfigurationBuilder.java Lines 270 (patched) <https://reviews.apache.org/r/58480/#comment246134> !StringUtils.isEmpty(namenodeIDs) should be enough. That will check whether the string is null or empty. - Tim Thorpe On April 20, 2017, 8:53 p.m., Sangeeta Ravindran wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/58480/ > --- > > (Updated April 20, 2017, 8:53 p.m.) > > > Review request for Ambari, Alexandr Antonenko, Di Li, DIPAYAN BHOWMICK, and > Tim Thorpe. > > > Bugs: AMBARI-20354 > https://issues.apache.org/jira/browse/AMBARI-20354 > > > Repository: ambari > > > Description > --- > > 1. In a multi-node cluster where namenode HA is not enabled, create a Files > View. Select Custom Configuration and enter the WebHDFS FileSystem URI*. In > the "List of NameNodes", enter the namenode information. Save and execute > view. There are no errors. > 2. Edit the view and delete the values entered in the "List of NameNodes" > field. Save and execute the view. > The following error is thrown although Namenode HA is not enabled. > Service 'hdfs' check failed: HDFS080 webhdfs.ha.namenodes.list namenodes > count is not exactly 2 > > > Diffs > - > > > contrib/views/utils/src/main/java/org/apache/ambari/view/utils/hdfs/ConfigurationBuilder.java > 7e174a7 > > > Diff: https://reviews.apache.org/r/58480/diff/1/ > > > Testing > --- > > Manual testing. > > > Thanks, > > Sangeeta Ravindran > >
Re: Review Request 58590: Provide an optional attribute for custom commands to specify title used in operations window title
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58590/#review172908 --- Ship it! Ship It! - Tim Thorpe On April 25, 2017, 4:51 a.m., Sangeeta Ravindran wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/58590/ > --- > > (Updated April 25, 2017, 4:51 a.m.) > > > Review request for Ambari, Di Li and Tim Thorpe. > > > Bugs: AMBARI-20616 > https://issues.apache.org/jira/browse/AMBARI-20616 > > > Repository: ambari > > > Description > --- > > Some services in our stack have custom commands where the name is a > combination of an operation and a component name. > > For e.g. Remove_Logical_myservice_mycomponent > > When the custom command is being executed, the operations window title is > displayed as "Remove Logical myservice mycomponent mycomponent" > > It would be useful to provide an optional attribute e.g. opsDisplayName for > custom commands to allow specifying the string for the title displayed in the > operations window. > > > Remove_Logical_myservice_mycomponent > true > > scripts/mycomponent.py > PYTHON > 10800 > > > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/actionmanager/HostRoleCommand.java > 87a6edf > > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java > 17bc718 > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/TaskResourceProvider.java > 20b0417 > > ambari-server/src/main/java/org/apache/ambari/server/orm/entities/HostRoleCommandEntity.java > 86feceb > > ambari-server/src/main/java/org/apache/ambari/server/state/CustomCommandDefinition.java > 280a59b > > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog300.java > 633d837 > ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql fd49b94 > ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql 9fc3209 > ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql 310208d > ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql c052104 > ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 5a58ef8 > ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 8c7a31d > ambari-server/src/main/resources/properties.json c2545fe > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/TaskResourceProviderTest.java > 8a52b8a > > ambari-server/src/test/java/org/apache/ambari/server/orm/entities/HostRoleCommandEntityTest.java > PRE-CREATION > > ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog300Test.java > fec041c > ambari-web/app/controllers/wizard/step9_controller.js 4697dad > ambari-web/app/utils/ajax/ajax.js 5d7108f > ambari-web/app/utils/helper.js 03a2e82 > ambari-web/app/utils/host_progress_popup.js c615cae > ambari-web/app/views/wizard/step9/hostLogPopupBody_view.js d983252 > ambari-web/test/utils/helper_test.js 4b9ec36 > > > Diff: https://reviews.apache.org/r/58590/diff/3/ > > > Testing > --- > > Manual Testing. > Added new test cases. > > > Thanks, > > Sangeeta Ravindran > >
Re: Review Request 58665: AMBARI-20822 Ambari Agent needs to handle custom version number like 1.2.3.4_Company in its Controller code
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58665/#review172840 --- Ship it! Ship It! - Tim Thorpe On April 24, 2017, 7:28 p.m., Di Li wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/58665/ > --- > > (Updated April 24, 2017, 7:28 p.m.) > > > Review request for Ambari and Tim Thorpe. > > > Bugs: AMBARI-20822 > https://issues.apache.org/jira/browse/AMBARI-20822 > > > Repository: ambari > > > Description > --- > > Ambari agent printed the following error (albeit as a logger.info) during > start up when its version is 1.2.3.0_MyAgent. > INFO 2017-04-21 06:17:19,510 Controller.py:545 - Exception in > move_data_dir_mount_file(). Error: invalid literal for int() with base 10: > '0_MyAgent' > > > Diffs > - > > ambari-agent/src/main/python/ambari_agent/Controller.py 78b5c0c > ambari-agent/src/test/python/ambari_agent/TestController.py 7f5d451 > > > Diff: https://reviews.apache.org/r/58665/diff/2/ > > > Testing > --- > > exiting unit tests, new unit test, patched a cluster with the fix and verify > the error message message was gone. > > > Thanks, > > Di Li > >
Re: Review Request 58665: AMBARI-20822 Ambari Agent needs to handle custom version number like 1.2.3.4_Company in its Controller code
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58665/#review172780 --- Ship it! Ship It! - Tim Thorpe On April 24, 2017, 12:41 p.m., Di Li wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/58665/ > --- > > (Updated April 24, 2017, 12:41 p.m.) > > > Review request for Ambari and Tim Thorpe. > > > Bugs: AMBARI-20822 > https://issues.apache.org/jira/browse/AMBARI-20822 > > > Repository: ambari > > > Description > --- > > Ambari agent printed the following error (albeit as a logger.info) during > start up when its version is 1.2.3.0_MyAgent. > INFO 2017-04-21 06:17:19,510 Controller.py:545 - Exception in > move_data_dir_mount_file(). Error: invalid literal for int() with base 10: > '0_MyAgent' > > > Diffs > - > > ambari-agent/src/main/python/ambari_agent/Controller.py 78b5c0c > ambari-agent/src/test/python/ambari_agent/TestController.py 7f5d451 > > > Diff: https://reviews.apache.org/r/58665/diff/1/ > > > Testing > --- > > exiting unit tests, new unit test, patched a cluster with the fix and verify > the error message message was gone. > > > Thanks, > > Di Li > >
Re: Review Request 58590: Provide an optional attribute for custom commands to allow using command name in operations window title
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58590/#review172542 --- Has upgrade been tested? - Tim Thorpe On April 20, 2017, 7:53 p.m., Sangeeta Ravindran wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/58590/ > --- > > (Updated April 20, 2017, 7:53 p.m.) > > > Review request for Ambari, Di Li and Tim Thorpe. > > > Bugs: AMBARI-20616 > https://issues.apache.org/jira/browse/AMBARI-20616 > > > Repository: ambari > > > Description > --- > > Some services in our stack have custom commands where the name is a > combination of an operation and a component name. > > For e.g. Remove_Logical_myservice_mycomponent > > When the custom command is being executed, the operations window title is > displayed as "Remove Logical myservice mycomponent mycomponent" > > It would be useful to provide an optional attribute e.g. useNameInOps for > custom commands to allow using the command name itself as the title displayed > in the operations window. > > > Remove_Logical_myservice_mycomponent > true > > scripts/mycomponent.py > PYTHON > 10800 > > > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/actionmanager/HostRoleCommand.java > 87a6edf > > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java > 17bc718 > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/TaskResourceProvider.java > 20b0417 > > ambari-server/src/main/java/org/apache/ambari/server/orm/entities/HostRoleCommandEntity.java > 86feceb > > ambari-server/src/main/java/org/apache/ambari/server/state/CustomCommandDefinition.java > 280a59b > > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog300.java > 633d837 > ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql fd49b94 > ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql 9fc3209 > ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql 310208d > ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql c052104 > ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 5a58ef8 > ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 8c7a31d > ambari-server/src/main/resources/properties.json c2545fe > > ambari-server/src/test/java/org/apache/ambari/server/controller/internal/TaskResourceProviderTest.java > 8a52b8a > > ambari-server/src/test/java/org/apache/ambari/server/orm/entities/HostRoleCommandEntityTest.java > PRE-CREATION > > ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog300Test.java > fec041c > ambari-web/app/controllers/wizard/step9_controller.js 4697dad > ambari-web/app/utils/ajax/ajax.js 2b5e52a > ambari-web/app/utils/helper.js 03a2e82 > ambari-web/app/utils/host_progress_popup.js c615cae > ambari-web/app/views/wizard/step9/hostLogPopupBody_view.js d983252 > ambari-web/test/utils/helper_test.js 4b9ec36 > > > Diff: https://reviews.apache.org/r/58590/diff/1/ > > > Testing > --- > > Manual Testing. > Added new test cases. > > > Thanks, > > Sangeeta Ravindran > >
Re: Review Request 58534: AMBARI-20784 Able to hide the "Why public repo not selected" when no stacks have repoinfo.xml with the "latest" section
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58534/#review172355 --- Ship it! Ship It! - Tim Thorpe On April 19, 2017, 4:22 p.m., Di Li wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/58534/ > --- > > (Updated April 19, 2017, 4:22 p.m.) > > > Review request for Ambari, Jaimin Jetly, Sangeeta Ravindran, and Tim Thorpe. > > > Bugs: Ambari-20784 > https://issues.apache.org/jira/browse/Ambari-20784 > > > Repository: ambari > > > Description > --- > > Able to hide the "Why public repo not selected" when no stacks have > repoinfo.xml with the "latest" section. There is no point of display the > error based on the stackDefault checking when no stacks even offer the > "latest" section in their repoinfo.xml. > > > Diffs > - > > ambari-web/app/controllers/wizard/step1_controller.js 2ac295b > ambari-web/app/mappers/stack_mapper.js 4b6a6f8 > ambari-web/app/models/stack.js dbc2d72 > ambari-web/app/templates/wizard/step1.hbs 769aca3 > ambari-web/app/utils/ajax/ajax.js 2d157fe > > > Diff: https://reviews.apache.org/r/58534/diff/1/ > > > Testing > --- > > existing unit tests > Build Ambari server and agent RPMs off trunk code with the changes, run > install. Modify repoinfo.xml in 2.x and 3.0 stacks to remove > section, verify the "Why not selected" is hidden. and when some stacks have > the section, the link shows up if no stacks have repo with > stackDefault=False. > > > Thanks, > > Di Li > >
Re: Review Request 58423: AMBARI-20757 - Selecting ignore service checks results in blank dialog box when hitting an error
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58423/ --- (Updated April 13, 2017, 6:19 p.m.) Review request for Ambari, Di Li and Sangeeta Ravindran. Bugs: AMBARI-20757 https://issues.apache.org/jira/browse/AMBARI-20757 Repository: ambari Description --- In order to see what has gone wrong, you need to drill down into the last step until you find the service check that failed. Ideally there would be a message above where they normally appear. Diffs (updated) - ambari-web/app/messages.js 4513775 ambari-web/app/templates/main/admin/stack_upgrade/stack_upgrade_wizard.hbs f6be5d1 Diff: https://reviews.apache.org/r/58423/diff/3/ Changes: https://reviews.apache.org/r/58423/diff/2-3/ Testing --- This was tested manually while testing upgrade. Thanks, Tim Thorpe
Re: Review Request 58423: AMBARI-20757 - Selecting ignore service checks results in blank dialog box when hitting an error
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58423/ --- (Updated April 13, 2017, 6:07 p.m.) Review request for Ambari, Di Li and Sangeeta Ravindran. Bugs: AMBARI-20757 https://issues.apache.org/jira/browse/AMBARI-20757 Repository: ambari Description --- In order to see what has gone wrong, you need to drill down into the last step until you find the service check that failed. Ideally there would be a message above where they normally appear. Diffs (updated) - ambari-web/app/messages.js 4513775 ambari-web/app/templates/main/admin/stack_upgrade/stack_upgrade_wizard.hbs f6be5d1 Diff: https://reviews.apache.org/r/58423/diff/2/ Changes: https://reviews.apache.org/r/58423/diff/1-2/ Testing --- This was tested manually while testing upgrade. Thanks, Tim Thorpe
Re: Review Request 58425: Remove Server attribute from the HTTP response headers
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58425/#review171903 --- Ship it! Ship It! - Tim Thorpe On April 13, 2017, 5:16 p.m., Sangeeta Ravindran wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/58425/ > --- > > (Updated April 13, 2017, 5:16 p.m.) > > > Review request for Ambari, Di Li, Robert Levas, and Tim Thorpe. > > > Bugs: AMBARI-20650 > https://issues.apache.org/jira/browse/AMBARI-20650 > > > Repository: ambari > > > Description > --- > > Currently the HTTP response header contains the Jetty version. > Server:Jetty(8.1.19.v20160209) > To follow best security practices, it would be good to avoid including this > information in the response headers. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java > 4e7af0c > > > Diff: https://reviews.apache.org/r/58425/diff/1/ > > > Testing > --- > > Manual Testing. Verified that "Server" header is removed from the HTTP > response headers. > > > Thanks, > > Sangeeta Ravindran > >
Review Request 58423: AMBARI-20757 - Selecting ignore service checks results in blank dialog box when hitting an error
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58423/ --- Review request for Ambari, Di Li and Sangeeta Ravindran. Bugs: AMBARI-20757 https://issues.apache.org/jira/browse/AMBARI-20757 Repository: ambari Description --- In order to see what has gone wrong, you need to drill down into the last step until you find the service check that failed. Ideally there would be a message above where they normally appear. Diffs - ambari-web/app/templates/main/admin/stack_upgrade/stack_upgrade_wizard.hbs a86048e Diff: https://reviews.apache.org/r/58423/diff/1/ Testing --- This was tested manually while testing upgrade. Thanks, Tim Thorpe
Re: Review Request 58152: AMBARI-20666 - beginning or ending commas in dfs.cluster.administrators cause trouble for starting services
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58152/
---
(Updated April 13, 2017, 4:20 p.m.)
Review request for Ambari, Alejandro Fernandez and Di Li.
Changes
---
Fix for both 2.0.6 and 3.0 hooks
Bugs: AMBARI-20666
https://issues.apache.org/jira/browse/AMBARI-20666
Repository: ambari
Description
---
In order to avoid issues with starting and stopping services, it would be
useful to trim any commas from the beginning or ending of the user and group
lists
Diffs (updated)
-
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-ANY/scripts/shared_initialization.py
4d0de7f
ambari-server/src/main/resources/stacks/HDP/3.0/hooks/before-ANY/scripts/shared_initialization.py
320872e
Diff: https://reviews.apache.org/r/58152/diff/2/
Changes: https://reviews.apache.org/r/58152/diff/1-2/
Testing
---
mvn clean test
Also ran manual testing including running the code as a separate script:
import re
user_and_groups = ",users, group1,group2,"
parts = re.split('\s', user_and_groups)
if len(parts) == 1:
parts.append("")
users_list = parts[0].strip(",").split(",") if parts[0] else []
groups_list = parts[1].strip(",").split(",") if parts[1] else []
print users_list
print groups_list
Thanks,
Tim Thorpe
Re: Review Request 58420: AMBARI-20746 Change to warning log if Ambari doesn't have baseurls for Install Packages step before EU/RU
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/58420/#review171889 --- Ship it! Ship It! - Tim Thorpe On April 13, 2017, 3:59 p.m., Di Li wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/58420/ > --- > > (Updated April 13, 2017, 3:59 p.m.) > > > Review request for Ambari and Tim Thorpe. > > > Bugs: AMBARI-20746 > https://issues.apache.org/jira/browse/AMBARI-20746 > > > Repository: ambari > > > Description > --- > > Change to warning log if Ambari doesn't have baseurls for Install Packages > step before EU/RU. It currently prints out as an INFO that can be easily > ignored. Since we don't fail the Install Packages step when the baseurls are > empty, we should at least change the logging level to warning. > > > Diffs > - > > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProvider.java > 92e72ed > > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostStackVersionResourceProvider.java > 8f2d4e6 > ambari-server/src/main/resources/custom_actions/scripts/install_packages.py > 112abe3 > > > Diff: https://reviews.apache.org/r/58420/diff/2/ > > > Testing > --- > > patch a trunk cluster with the change, run Install step before the EU/RU > > > Thanks, > > Di Li > >
