> On March 28, 2016, 11:44 a.m., Alejandro Fernandez wrote: > > ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql, line 424 > > <https://reviews.apache.org/r/45347/diff/2/?file=1315827#file1315827line424> > > > > Let's use an ID here for the PK, so we can reference it in other tables > > as a FK. > > We can make blueprint_name a UQ > > Alejandro Fernandez wrote: > Also, let's give names to the FK and UK. > > Nahappan Somasundaram wrote: > Blueprint name per row is not unique. Depending on the numbe of settings, > the blueprint_name will be repeated. A sequential ID column can be added for > future use as a FK lookup.
The blueprint_configuration and blueprint_settings tables are read only once during the template deployment. If we want to reference the blueprint name using an UQ id, then the blueprint table needs an ID column. Then blueprint_configuration and blueprint_settings table need to be modified, which is a lot of ask for one read. In the blueprint_settings table, blueprint_name and setting_name form the composite PK. Adding an ID column in this table does not add much value unless we plan to use it later for quick lookup or search. - Nahappan ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/45347/#review125704 ----------------------------------------------------------- On March 27, 2016, 2:51 p.m., Nahappan Somasundaram wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/45347/ > ----------------------------------------------------------- > > (Updated March 27, 2016, 2:51 p.m.) > > > Review request for Ambari, Jonathan Hurley, Nate Cole, Sumit Mohanty, > Sebastian Toader, and Sid Wagle. > > > Bugs: AMBARI-15592 > https://issues.apache.org/jira/browse/AMBARI-15592 > > > Repository: ambari > > > Description > ------- > > AMBARI-15592: Auto-start services - support blueprint deployment. > > ** Issue: ** > Define a JSON structure to specify settings for auto start in a blueprint and > use that information to enable or disable auto start for components during > deployment. > > ** Changes ** > Added a new section in the blueprint called "settings" which is a collection > of various setting names to collection of properties. For auto start, the > following setting names are used: > *recovery_settings*: {"recovery_enabled" : "true" } specifies that auto start > should be enabled for all components. Can also be specified within a > collection, to support a uniform schema. > *service_settings*: Collection of service names and the recovery values. > [{"name":"HDFS", "recovery_enabled":"false"},{...}] overrides cluster level > recovery settings. > *component_settings*: Collection of component names and the corresponding > recovery values. [{"name":"HDFS_CLIENT", "recovery_enabled":"true"},{...}]. > Overrides both service settings and recovery settings. > > > Diffs > ----- > > > ambari-server/src/main/java/org/apache/ambari/server/orm/entities/BlueprintEntity.java > 8578d6beca91bf411d0c3dafeaee42d2dd23caea > > ambari-server/src/main/java/org/apache/ambari/server/orm/entities/BlueprintSettingsEntity.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/orm/entities/BlueprintSettingsEntityPK.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/topology/AmbariContext.java > 87225adc0afafc111b865120fd71742dc4e096f0 > > ambari-server/src/main/java/org/apache/ambari/server/topology/Blueprint.java > 11311dba0f1174248d24276a41837d7284e41607 > > ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintFactory.java > cca28ca1529d6bccdf7a870dab7c317c1a334b1d > > ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintImpl.java > bea036421c64b70b4bceffcbd0d13c26020a7ed1 > ambari-server/src/main/java/org/apache/ambari/server/topology/Settings.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/topology/SettingsFactory.java > PRE-CREATION > > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java > 38a3614ae3543ae1de0ea0bfd4191951c6d5c5ff > ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql > f8c97ca11040bff414626189591d86d4781be478 > ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql > b306c0ab7687c6fe05c818acefa2d063e5b111ed > ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql > 37744f8bc729e374a1db76f6509a6e85bffb37d0 > ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql > eba1745d8c7a3e1b677cb8b8fc33d9ae09d0cf68 > ambari-server/src/main/resources/Ambari-DDL-Postgres-EMBEDDED-CREATE.sql > cad0a39d824fdbf03b1ec6cd2a99dec79fa916d2 > ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql > 346af50961fa1b2a41bd0a81121a08ec8ed70a2f > ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql > e238a7665dff5a52426be9e334e4a48f53c7dbee > ambari-server/src/main/resources/META-INF/persistence.xml > 513035f5baf6eacfcc69507069d311418546a09c > ambari-server/src/main/resources/properties.json > 01c15f2b1c3564c37e8203ec70f2d2b812135b13 > > ambari-server/src/test/java/org/apache/ambari/server/orm/entities/BlueprintEntityTest.java > c660d19aee1727fd4734c07c0e5130f5bbe3c3cf > > ambari-server/src/test/java/org/apache/ambari/server/orm/entities/BlueprintSettingsEntityPKTest.java > PRE-CREATION > > ambari-server/src/test/java/org/apache/ambari/server/orm/entities/BlueprintSettingsEntityTest.java > PRE-CREATION > > ambari-server/src/test/java/org/apache/ambari/server/topology/SettingsFactoryTest.java > PRE-CREATION > > ambari-server/src/test/java/org/apache/ambari/server/topology/SettingsTest.java > PRE-CREATION > > ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog240Test.java > 7b2d797486061a1377622806f7680176fa8ad458 > > Diff: https://reviews.apache.org/r/45347/diff/ > > > Testing > ------- > > ** 1. mvn clean install ** > > [INFO] > ------------------------------------------------------------------------ > [INFO] Reactor Summary: > [INFO] > [INFO] Ambari Main ....................................... SUCCESS [6.402s] > [INFO] Apache Ambari Project POM ......................... SUCCESS [0.039s] > [INFO] Ambari Web ........................................ SUCCESS [24.133s] > [INFO] Ambari Views ...................................... SUCCESS [1.280s] > [INFO] Ambari Admin View ................................. SUCCESS [5.725s] > [INFO] ambari-metrics .................................... SUCCESS [0.346s] > [INFO] Ambari Metrics Common ............................. SUCCESS [0.459s] > [INFO] Ambari Metrics Hadoop Sink ........................ SUCCESS [1.077s] > [INFO] Ambari Metrics Flume Sink ......................... SUCCESS [0.574s] > [INFO] Ambari Metrics Kafka Sink ......................... SUCCESS [0.582s] > [INFO] Ambari Metrics Storm Sink ......................... SUCCESS [1.459s] > [INFO] Ambari Metrics Collector .......................... SUCCESS [6.518s] > [INFO] Ambari Metrics Monitor ............................ SUCCESS [2.164s] > [INFO] Ambari Metrics Grafana ............................ SUCCESS [0.847s] > [INFO] Ambari Metrics Assembly ........................... SUCCESS [1:15.164s] > [INFO] Ambari Server ..................................... SUCCESS [2:20.006s] > [INFO] Ambari Functional Tests ........................... SUCCESS [1.010s] > [INFO] Ambari Agent ...................................... SUCCESS [22.882s] > [INFO] Ambari Client ..................................... SUCCESS [0.043s] > [INFO] Ambari Python Client .............................. SUCCESS [0.839s] > [INFO] Ambari Groovy Client .............................. SUCCESS [2.054s] > [INFO] Ambari Shell ...................................... SUCCESS [0.040s] > [INFO] Ambari Python Shell ............................... SUCCESS [0.676s] > [INFO] Ambari Groovy Shell ............................... SUCCESS [1.182s] > [INFO] > ------------------------------------------------------------------------ > [INFO] BUILD SUCCESS > [INFO] > ------------------------------------------------------------------------ > [INFO] Total time: 4:56.150s > [INFO] Finished at: Fri Mar 25 17:24:57 PDT 2016 > [INFO] Final Memory: 279M/1202M > [INFO] > ------------------------------------------------------------------------ > > ** 2. mvn test -DskipPythonTests > -Dtest=BlueprintEntityTest,BlueprintSettingsEntityPKTest,BlueprintSettingsEntityTest,SettingsFactoryTest,SettingsTest > ** > ------------------------------------------------------- > T E S T S > ------------------------------------------------------- > Picked up _JAVA_OPTIONS: -Xmx2048m -XX:MaxPermSize=512m > -Djava.awt.headless=true > Running org.apache.ambari.server.orm.entities.BlueprintEntityTest > Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.001 sec - > in org.apache.ambari.server.orm.entities.BlueprintEntityTest > Picked up _JAVA_OPTIONS: -Xmx2048m -XX:MaxPermSize=512m > -Djava.awt.headless=true > Running org.apache.ambari.server.orm.entities.BlueprintSettingsEntityPKTest > Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.009 sec - > in org.apache.ambari.server.orm.entities.BlueprintSettingsEntityPKTest > Picked up _JAVA_OPTIONS: -Xmx2048m -XX:MaxPermSize=512m > -Djava.awt.headless=true > Running org.apache.ambari.server.orm.entities.BlueprintSettingsEntityTest > Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.014 sec - > in org.apache.ambari.server.orm.entities.BlueprintSettingsEntityTest > Picked up _JAVA_OPTIONS: -Xmx2048m -XX:MaxPermSize=512m > -Djava.awt.headless=true > Running org.apache.ambari.server.topology.SettingsFactoryTest > Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.009 sec - > in org.apache.ambari.server.topology.SettingsFactoryTest > Picked up _JAVA_OPTIONS: -Xmx2048m -XX:MaxPermSize=512m > -Djava.awt.headless=true > Running org.apache.ambari.server.topology.SettingsTest > Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.007 sec - > in org.apache.ambari.server.topology.SettingsTest > > Results : > > Tests run: 16, Failures: 0, Errors: 0, Skipped: 0 > > [INFO] Rat check: Summary of files. Unapproved: 0 unknown: 0 generated: 0 > approved: 4290 licence. > [INFO] > [INFO] --- exec-maven-plugin:1.2.1:exec (python-test) @ ambari-server --- > [INFO] skipping execute as per configuraion > [INFO] > ------------------------------------------------------------------------ > [INFO] BUILD SUCCESS > [INFO] > ------------------------------------------------------------------------ > [INFO] Total time: 28.801s > [INFO] Finished at: Fri Mar 25 17:32:53 PDT 2016 > [INFO] Final Memory: 55M/1014M > [INFO] > ------------------------------------------------------------------------ > > > ** mvn test -DskipPythonTests -Dtest=UpgradeCatalog240Test ** > > > ------------------------------------------------------- > T E S T S > ------------------------------------------------------- > Picked up _JAVA_OPTIONS: -Xmx2048m -XX:MaxPermSize=512m > -Djava.awt.headless=true > Running org.apache.ambari.server.upgrade.UpgradeCatalog240Test > Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9.649 sec - > in org.apache.ambari.server.upgrade.UpgradeCatalog240Test > > Results : > > Tests run: 4, Failures: 0, Errors: 0, Skipped: 0 > > [INFO] > ------------------------------------------------------------------------ > [INFO] BUILD SUCCESS > [INFO] > ------------------------------------------------------------------------ > [INFO] Total time: 33.620s > [INFO] Finished at: Sun Mar 27 14:49:50 PDT 2016 > [INFO] Final Memory: 55M/1013M > [INFO] > ------------------------------------------------------------------------ > > ** 3. Manual tests ** > > Deployed a 2-node VM and replaced the ambari-server JAR on the server node > and made a blueprint based deployment: > > [root@c6403 /]# curl -u admin:admin -H "X-Requested-By: ambari" -X POST -d > @/vagrant/blueprints1.json http://localhost:8080/api/v1/blueprints/bp1 > > [root@c6403 /]# curl -u admin:admin -H "X-Requested-By: ambari" -X POST -d > @/vagrant/hg1.json http://localhost:8080/api/v1/clusters/c1 > { > "href" : "http://localhost:8080/api/v1/clusters/c1/requests/1";, > "Requests" : { > "id" : 1, > "status" : "Accepted" > } > } > > ** Blueprint "settings" ** > > "settings" : [ > { > "recovery_settings" : [{ > "recovery_enabled" : "true" > } > ]}, > { > "service_settings" : [ > { > "name" : "HDFS", > "recovery_enabled" : "false" > }, > { > "name" : "TEZ", > "recovery_enabled" : "false" > } > ]}, > { > "component_settings" : [ > { > "name" : "DATANODE", > "recovery_enabled" : "true" > } > ] > } > ] > > ** Database: blueprint_settings table ** > > ambari=> select * from blueprint_settings; > blueprint_name | setting_name | > setting_data > ----------------+--------------------+---------------------------------------------------------------------------------------- > bp1 | component_settings | > [{"recovery_enabled":"true","name":"DATANODE"}] > bp1 | recovery_settings | [{"recovery_enabled":"true"}] > bp1 | service_settings | > [{"recovery_enabled":"false","name":"HDFS"},{"recovery_enabled":"false","name":"TEZ"}] > (3 rows) > > ** Database: servicecomponentdesiredstate table ** > > ambari=> select * from servicecomponentdesiredstate; > id | component_name | cluster_id | desired_stack_id | desired_version | > desired_state | service_name | recovery_enabled > > ----+---------------------+------------+------------------+-----------------+---------------+----------------+----------------- > 3 | TEZ_CLIENT | 2 | 7 | UNKNOWN | > INSTALLED | TEZ | 0 > 5 | MAPREDUCE2_CLIENT | 2 | 7 | UNKNOWN | > INSTALLED | MAPREDUCE2 | 1 > 12 | HDFS_CLIENT | 2 | 7 | UNKNOWN | > INSTALLED | HDFS | 0 > 13 | YARN_CLIENT | 2 | 7 | UNKNOWN | > INSTALLED | YARN | 1 > 15 | ZOOKEEPER_CLIENT | 2 | 7 | UNKNOWN | > INSTALLED | ZOOKEEPER | 1 > 1 | DATANODE | 2 | 7 | UNKNOWN | > STARTED | HDFS | 1 > 2 | METRICS_COLLECTOR | 2 | 7 | UNKNOWN | > STARTED | AMBARI_METRICS | 1 > 4 | HISTORYSERVER | 2 | 7 | UNKNOWN | > STARTED | MAPREDUCE2 | 1 > 6 | ZOOKEEPER_SERVER | 2 | 7 | UNKNOWN | > STARTED | ZOOKEEPER | 1 > 7 | APP_TIMELINE_SERVER | 2 | 7 | UNKNOWN | > STARTED | YARN | 1 > 8 | METRICS_MONITOR | 2 | 7 | UNKNOWN | > STARTED | AMBARI_METRICS | 1 > 9 | NAMENODE | 2 | 7 | UNKNOWN | > STARTED | HDFS | 0 > 10 | SECONDARY_NAMENODE | 2 | 7 | UNKNOWN | > STARTED | HDFS | 0 > 11 | RESOURCEMANAGER | 2 | 7 | UNKNOWN | > STARTED | YARN | 1 > 14 | METRICS_GRAFANA | 2 | 7 | UNKNOWN | > STARTED | AMBARI_METRICS | 1 > 16 | NODEMANAGER | 2 | 7 | UNKNOWN | > STARTED | YARN | 1 > (16 rows) > > > Thanks, > > Nahappan Somasundaram > >
