----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45347/ -----------------------------------------------------------
(Updated March 29, 2016, 7:17 p.m.) Review request for Ambari, Ajit Kumar, Jonathan Hurley, Nate Cole, Sumit Mohanty, Sebastian Toader, and Sid Wagle. Changes ------- Renamed blueprint_settings table to blueprint_setting and renamed the backing classes. Bugs: AMBARI-15592 https://issues.apache.org/jira/browse/AMBARI-15592 Repository: ambari Description ------- AMBARI-15592: Auto-start services - support blueprint deployment. ** Issue: ** Define a JSON structure to specify settings for auto start in a blueprint and use that information to enable or disable auto start for components during deployment. ** Changes ** Added a new section in the blueprint called "settings" which is a collection of various setting names to collection of properties. For auto start, the following setting names are used: *recovery_settings*: {"recovery_enabled" : "true" } specifies that auto start should be enabled for all components. Can also be specified within a collection, to support a uniform schema. *service_settings*: Collection of service names and the recovery values. [{"name":"HDFS", "recovery_enabled":"false"},{...}] overrides cluster level recovery settings. *component_settings*: Collection of component names and the corresponding recovery values. [{"name":"HDFS_CLIENT", "recovery_enabled":"true"},{...}]. Overrides both service settings and recovery settings. Diffs (updated) ----- ambari-server/src/main/java/org/apache/ambari/server/orm/entities/BlueprintEntity.java 8578d6beca91bf411d0c3dafeaee42d2dd23caea ambari-server/src/main/java/org/apache/ambari/server/orm/entities/BlueprintSettingEntity.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/topology/AmbariContext.java a5e2fa1c7a2adaadc8bbe9de15b913cd9a7ca456 ambari-server/src/main/java/org/apache/ambari/server/topology/Blueprint.java 11311dba0f1174248d24276a41837d7284e41607 ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintFactory.java cca28ca1529d6bccdf7a870dab7c317c1a334b1d ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintImpl.java bea036421c64b70b4bceffcbd0d13c26020a7ed1 ambari-server/src/main/java/org/apache/ambari/server/topology/Setting.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/topology/SettingFactory.java PRE-CREATION ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java eb7d750702bfe83913cea92f1e1ef978ed0e6189 ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql 4a0479eb3b749f332fda306ecc9b9698cde0ac92 ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql e40165d72d5d97be28882bbdf61cae5f5be67c9b ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql e85ea496696603b20b0efd8ffd6fb5cd2e9246ea ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 8bcda317c70cce320fa69127e52a786fdd372eac ambari-server/src/main/resources/Ambari-DDL-Postgres-EMBEDDED-CREATE.sql c7d7cff85e1aa502976db3f7902c3d920a9e5a02 ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 4bd7a7a9209d4d102d6d0612450280214f9b32b0 ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 063db3aa6f4e3f14563180be2b20664f9351d852 ambari-server/src/main/resources/META-INF/persistence.xml 513035f5baf6eacfcc69507069d311418546a09c ambari-server/src/main/resources/properties.json 01c15f2b1c3564c37e8203ec70f2d2b812135b13 ambari-server/src/test/java/org/apache/ambari/server/orm/entities/BlueprintEntityTest.java c660d19aee1727fd4734c07c0e5130f5bbe3c3cf ambari-server/src/test/java/org/apache/ambari/server/orm/entities/BlueprintSettingEntityTest.java PRE-CREATION ambari-server/src/test/java/org/apache/ambari/server/topology/SettingFactoryTest.java PRE-CREATION ambari-server/src/test/java/org/apache/ambari/server/topology/SettingTest.java PRE-CREATION ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog240Test.java ef2a96e4e376bf2fa7a10b2d6c5c4c40f8fcc663 Diff: https://reviews.apache.org/r/45347/diff/ Testing (updated) ------- ** 1. mvn clean install ** [INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary: [INFO] [INFO] Ambari Main ....................................... SUCCESS [4.429s] [INFO] Apache Ambari Project POM ......................... SUCCESS [0.035s] [INFO] Ambari Web ........................................ SUCCESS [30.123s] [INFO] Ambari Views ...................................... SUCCESS [1.231s] [INFO] Ambari Admin View ................................. SUCCESS [7.878s] [INFO] ambari-metrics .................................... SUCCESS [0.360s] [INFO] Ambari Metrics Common ............................. SUCCESS [0.497s] [INFO] Ambari Metrics Hadoop Sink ........................ SUCCESS [1.041s] [INFO] Ambari Metrics Flume Sink ......................... SUCCESS [0.567s] [INFO] Ambari Metrics Kafka Sink ......................... SUCCESS [0.638s] [INFO] Ambari Metrics Storm Sink ......................... SUCCESS [1.506s] [INFO] Ambari Metrics Collector .......................... SUCCESS [6.252s] [INFO] Ambari Metrics Monitor ............................ SUCCESS [2.092s] [INFO] Ambari Metrics Grafana ............................ SUCCESS [0.909s] [INFO] Ambari Metrics Assembly ........................... SUCCESS [1:13.560s] [INFO] Ambari Server ..................................... SUCCESS [2:28.882s] [INFO] Ambari Functional Tests ........................... SUCCESS [2.142s] [INFO] Ambari Agent ...................................... SUCCESS [24.309s] [INFO] Ambari Client ..................................... SUCCESS [0.047s] [INFO] Ambari Python Client .............................. SUCCESS [1.069s] [INFO] Ambari Groovy Client .............................. SUCCESS [2.121s] [INFO] Ambari Shell ...................................... SUCCESS [0.040s] [INFO] Ambari Python Shell ............................... SUCCESS [0.710s] [INFO] Ambari Groovy Shell ............................... SUCCESS [1.158s] [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 5:12.200s [INFO] Finished at: Tue Mar 29 19:11:37 PDT 2016 [INFO] Final Memory: 256M/1137M [INFO] ------------------------------------------------------------------------ ** 2. mvn test -DskipPythonTests -Dtest=BlueprintEntityTest,UpgradeCatalog240Test,BlueprintSettingEntityTest,SettingFactoryTest,SettingTest ** ------------------------------------------------------- T E S T S ------------------------------------------------------- Picked up _JAVA_OPTIONS: -Xmx2048m -XX:MaxPermSize=512m -Djava.awt.headless=true Running org.apache.ambari.server.orm.entities.BlueprintEntityTest Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.004 sec - in org.apache.ambari.server.orm.entities.BlueprintEntityTest Picked up _JAVA_OPTIONS: -Xmx2048m -XX:MaxPermSize=512m -Djava.awt.headless=true Running org.apache.ambari.server.orm.entities.BlueprintSettingEntityTest Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.014 sec - in org.apache.ambari.server.orm.entities.BlueprintSettingEntityTest Picked up _JAVA_OPTIONS: -Xmx2048m -XX:MaxPermSize=512m -Djava.awt.headless=true Running org.apache.ambari.server.topology.SettingFactoryTest Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.008 sec - in org.apache.ambari.server.topology.SettingFactoryTest Picked up _JAVA_OPTIONS: -Xmx2048m -XX:MaxPermSize=512m -Djava.awt.headless=true Running org.apache.ambari.server.topology.SettingTest Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.008 sec - in org.apache.ambari.server.topology.SettingTest Picked up _JAVA_OPTIONS: -Xmx2048m -XX:MaxPermSize=512m -Djava.awt.headless=true Running org.apache.ambari.server.upgrade.UpgradeCatalog240Test Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9.873 sec - in org.apache.ambari.server.upgrade.UpgradeCatalog240Test Results : Tests run: 17, Failures: 0, Errors: 0, Skipped: 0 [INFO] Rat check: Summary of files. Unapproved: 0 unknown: 0 generated: 0 approved: 4300 licence. [INFO] [INFO] --- exec-maven-plugin:1.2.1:exec (python-test) @ ambari-server --- [INFO] skipping execute as per configuraion [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 37.273s [INFO] Finished at: Tue Mar 29 19:03:27 PDT 2016 [INFO] Final Memory: 55M/1005M [INFO] ------------------------------------------------------------------------ ** 3. Manual tests ** Deployed a 2-node VM and replaced the ambari-server JAR on the server node and made a blueprint based deployment. Also tested upgrade: [root@c6403 /]# curl -u admin:admin -H "X-Requested-By: ambari" -X POST -d @/vagrant/blueprints1.json http://localhost:8080/api/v1/blueprints/bp1 [root@c6403 /]# curl -u admin:admin -H "X-Requested-By: ambari" -X POST -d @/vagrant/hg1.json http://localhost:8080/api/v1/clusters/c1 { "href" : "http://localhost:8080/api/v1/clusters/c1/requests/1", "Requests" : { "id" : 1, "status" : "Accepted" } } ** Blueprint "settings" ** "settings" : [ { "recovery_settings" : [{ "recovery_enabled" : "true" } ]}, { "service_settings" : [ { "name" : "HDFS", "recovery_enabled" : "false" }, { "name" : "TEZ", "recovery_enabled" : "false" } ]}, { "component_settings" : [ { "name" : "DATANODE", "recovery_enabled" : "true" } ] } ] ** Database: blueprint_setting table ** ambari=> select * from blueprint_setting; blueprint_name | setting_name | setting_data ----------------+--------------------+---------------------------------------------------------------------------------------- bp1 | component_settings | [{"recovery_enabled":"true","name":"DATANODE"}] bp1 | recovery_settings | [{"recovery_enabled":"true"}] bp1 | service_settings | [{"recovery_enabled":"false","name":"HDFS"},{"recovery_enabled":"false","name":"TEZ"}] (3 rows) ** Database: servicecomponentdesiredstate table ** ambari=> select * from servicecomponentdesiredstate; id | component_name | cluster_id | desired_stack_id | desired_version | desired_state | service_name | recovery_enabled ----+---------------------+------------+------------------+-----------------+---------------+----------------+----------------- 3 | TEZ_CLIENT | 2 | 7 | UNKNOWN | INSTALLED | TEZ | 0 5 | MAPREDUCE2_CLIENT | 2 | 7 | UNKNOWN | INSTALLED | MAPREDUCE2 | 1 12 | HDFS_CLIENT | 2 | 7 | UNKNOWN | INSTALLED | HDFS | 0 13 | YARN_CLIENT | 2 | 7 | UNKNOWN | INSTALLED | YARN | 1 15 | ZOOKEEPER_CLIENT | 2 | 7 | UNKNOWN | INSTALLED | ZOOKEEPER | 1 1 | DATANODE | 2 | 7 | UNKNOWN | STARTED | HDFS | 1 2 | METRICS_COLLECTOR | 2 | 7 | UNKNOWN | STARTED | AMBARI_METRICS | 1 4 | HISTORYSERVER | 2 | 7 | UNKNOWN | STARTED | MAPREDUCE2 | 1 6 | ZOOKEEPER_SERVER | 2 | 7 | UNKNOWN | STARTED | ZOOKEEPER | 1 7 | APP_TIMELINE_SERVER | 2 | 7 | UNKNOWN | STARTED | YARN | 1 8 | METRICS_MONITOR | 2 | 7 | UNKNOWN | STARTED | AMBARI_METRICS | 1 9 | NAMENODE | 2 | 7 | UNKNOWN | STARTED | HDFS | 0 10 | SECONDARY_NAMENODE | 2 | 7 | UNKNOWN | STARTED | HDFS | 0 11 | RESOURCEMANAGER | 2 | 7 | UNKNOWN | STARTED | YARN | 1 14 | METRICS_GRAFANA | 2 | 7 | UNKNOWN | STARTED | AMBARI_METRICS | 1 16 | NODEMANAGER | 2 | 7 | UNKNOWN | STARTED | YARN | 1 (16 rows) Thanks, Nahappan Somasundaram