[ https://issues.apache.org/jira/browse/AMBARI-22945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16383438#comment-16383438 ]
Swapan Shridhar edited comment on AMBARI-22945 at 3/2/18 10:53 AM: ------------------------------------------------------------------- POSTMAN Script : [^Ambari API v2.postman_collection_Updated] It creates the following structure: -- Note that it creates the same named Service Components and it s associated Host Components. - Two Service Groups (SG) : *prod*, *dev* - Two services in each Service Group : -- SG: prod having Services : *HIVE_Prod*, *ZOOKEEPER_Prod* -- SG: dev having Services : *HIVE_Dev*, *ZOOKEEPER_Dev* - Each Service having two Service Components (Total Service Components : 8) : -- / prod / HIVE_Prod --- Service Comps : ---- *HIVE_SERVER_Prod*, ---- *HIVE_CLIENT_Prod* -- / prod / ZOOKEEPER_Prod --- Service Comps : ---- *ZOOKEEPER_SERVER_Prod*, ---- *ZOOKEEPER_CLIENT_Prod* -- / dev / HIVE_Dev --- Service Comps : ---- *HIVE_SERVER_Dev*, ---- *HIVE_CLIENT_Dev* -- / dev / ZOOKEEPER_Dev --- Service Comps : ---- *ZOOKEEPER_Dev*, ---- *ZOOKEEPER_CLIENT_Dev* - Each Service Component having 2 Host Components associated (Total Host Components : 16) -- / prod / HIVE_Prod / *HIVE_SERVER_Prod* --- Host Components: ---- *HIVE_SERVER_Prod* -> On Host 1 ---- *HIVE_SERVER_Prod* -> On Host 2 -- / prod / HIVE_Prod / *HIVE_CLIENT_Prod* --- Host Components: ---- *HIVE_CLIENT_Prod* -> On Host 1 ---- *HIVE_CLIENT_Prod* -> On Host 2 -- / prod / ZOOKEEPER_Prod / *ZOOKEEPER_SERVER_Prod* --- Host Components: ---- *ZOOKEEPER_SERVER_Prod* -> On Host 1 ---- *ZOOKEEPER_SERVER_Prod* -> On Host 2 -- / prod / ZOOKEEPER_Prod / *ZOOKEEPER_CLIENT_Prod* --- Host Components: ---- *ZOOKEEPER_CLIENT_Prod* -> On Host 1 ---- *ZOOKEEPER_CLIENT_Prod* -> On Host 2 -- / dev / HIVE_Dev / *HIVE_SERVER_Dev* --- Host Components: ---- *HIVE_SERVER_Dev* -> On Host 1 ---- *HIVE_SERVER_Dev* -> On Host 2 -- / dev / HIVE_Dev / *HIVE_CLIENT_Dev* --- Host Components: ---- *HIVE_CLIENT_Dev* -> On Host 1 ---- *HIVE_CLIENT_Dev* -> On Host 2 -- / dev / ZOOKEEPER_Dev / *ZOOKEEPER_SERVER_Dev* --- Host Components: ---- *ZOOKEEPER_SERVER_Dev* -> On Host 1 ---- *ZOOKEEPER_SERVER_Dev* -> On Host 2 -- / dev / ZOOKEEPER_Dev / *ZOOKEEPER_CLIENT_Dev* --- Host Components: ---- *ZOOKEEPER_CLIENT_Dev* -> On Host 1 ---- *ZOOKEEPER_CLIENT_Dev* -> On Host 2 was (Author: swapanshridhar): It creates the following : - Two Service Groups (SG) : *prod*, *dev* - Two services in each Service Group : -- SG: prod having Services : *HIVE_Prod*, *ZOOKEEPER_Prod* -- SG: dev having Services : *HIVE_Dev*, *ZOOKEEPER_Dev* - Each Service having two Service Components (Total Service Components : 8) : -- / prod / HIVE_Prod --- Service Comps : ---- *HIVE_SERVER_Prod*, ---- *HIVE_CLIENT_Prod* -- / prod / ZOOKEEPER_Prod --- Service Comps : ---- *ZOOKEEPER_SERVER_Prod*, ---- *ZOOKEEPER_CLIENT_Prod* -- / dev / HIVE_Dev --- Service Comps : ---- *HIVE_SERVER_Dev*, ---- *HIVE_CLIENT_Dev* -- / dev / ZOOKEEPER_Dev --- Service Comps : ---- *ZOOKEEPER_Dev*, ---- *ZOOKEEPER_CLIENT_Dev* - Each Service Component having 2 Host Components associated (Total Host Components : 16) -- / prod / HIVE_Prod / *HIVE_SERVER_Prod* --- Host Components: ---- *HIVE_SERVER_Prod* -> On Host 1 ---- *HIVE_SERVER_Prod* -> On Host 2 -- / prod / HIVE_Prod / *HIVE_CLIENT_Prod* --- Host Components: ---- *HIVE_CLIENT_Prod* -> On Host 1 ---- *HIVE_CLIENT_Prod* -> On Host 2 -- / prod / ZOOKEEPER_Prod / *ZOOKEEPER_SERVER_Prod* --- Host Components: ---- *ZOOKEEPER_SERVER_Prod* -> On Host 1 ---- *ZOOKEEPER_SERVER_Prod* -> On Host 2 -- / prod / ZOOKEEPER_Prod / *ZOOKEEPER_CLIENT_Prod* --- Host Components: ---- *ZOOKEEPER_CLIENT_Prod* -> On Host 1 ---- *ZOOKEEPER_CLIENT_Prod* -> On Host 2 -- / dev / HIVE_Dev / *HIVE_SERVER_Dev* --- Host Components: ---- *HIVE_SERVER_Dev* -> On Host 1 ---- *HIVE_SERVER_Dev* -> On Host 2 -- / dev / HIVE_Dev / *HIVE_CLIENT_Dev* --- Host Components: ---- *HIVE_CLIENT_Dev* -> On Host 1 ---- *HIVE_CLIENT_Dev* -> On Host 2 -- / dev / ZOOKEEPER_Dev / *ZOOKEEPER_SERVER_Dev* --- Host Components: ---- *ZOOKEEPER_SERVER_Dev* -> On Host 1 ---- *ZOOKEEPER_SERVER_Dev* -> On Host 2 -- / dev / ZOOKEEPER_Dev / *ZOOKEEPER_CLIENT_Dev* --- Host Components: ---- *ZOOKEEPER_CLIENT_Dev* -> On Host 1 ---- *ZOOKEEPER_CLIENT_Dev* -> On Host 2 > Enhance host components API to support multiple host component instances. > ------------------------------------------------------------------------- > > Key: AMBARI-22945 > URL: https://issues.apache.org/jira/browse/AMBARI-22945 > Project: Ambari > Issue Type: Bug > Components: ambari-server > Reporter: Swapan Shridhar > Assignee: Swapan Shridhar > Priority: Critical > Labels: pull-request-available > Fix For: 3.0.0 > > Attachments: Ambari API v2.postman_collection_Updated > > Time Spent: 5h 20m > Remaining Estimate: 0h > > *BACKGROUND:* > Given that in Ambari, we refer components of a services in 2 ways: > - *1.* One from the Service component APIs : > > *http://<AmbariServerHost>:8080/api/v1/clusters/<clusterName>/services/<ServiceName>/components/<componentName>* > - *2.*From the host component APIs (which tell us the host on which the > current component is resides) > > *http://<AmbariServerHost>:8080/api/v1/clusters/<clusterName>/hosts/<hostName>/host_components/<hostComponentName>* > _Note that, we of now, we are referring both 1and 2 using the component names_ > But in the multi-component world for Ambari, we will come up with a situation > like this, as shown below: > - *A.* If *core* _Service Group_ has its *HDFS_CLIENT* type component named > as *HDFS_CLIENT* > *Core -> HDFS -> HDFS_CLIENT* > - *B.* And at the same time, we have *edw* _Service Group_ having its > *HDFS_CLIENT* type component named as *HDFS_CLIENT* : > *edw -> HDFS -> HDFS_CLIENT* > There is no good way to distinguish a given host component in *A* and *B*, > if we continue to refer them with the component name end point (No. *2* API > call). > However, No. *1* call is still fine as we will not allow 2 names for the same > component within a given service, thus making it unique and allowing us to > continue using the component names endpoint. > *****************************************************************WORK DONE > :******************************************************************* > In order to support multi-instance for components of a given service, we need > to enhance the *Host Components API* so that they can distinguish *one > component instance of a service* from *another component instance of a > service* with the same Name. > The way to achieve this is to move Host Components API to be *ID* (number) > based end point, compared to earlier being a name based endpoint. This will > allow us to distinguish *Core -> HDFS -> HDFS_CLIENT* from *edw -> HDFS -> > HDFS_CLIENT*, as an example. > Thus, following changes are required: > - New field : *component_type* (eg: HDFS_CLIENT, HIVE_SERVER, > ZOOKEEPER_SERVER etc) > - Existing field : *component_name* will now hold the actual name given for > the component at the time of install. For example: > - *HDFS_CLIENT_EDW* for component_type *HDFS_CLIENT* > - *HIVE_SERVER* FOR component_type *HIVE_SERVER* _(name can be same as > type also)_ > - *HIVE_SERVER1* for component_type *HIVE_SERVER* > - *ZOOKEEPER_SERVER_FOR_KAFKA* for component_type *ZOOKEEPER_SERVER*, > and so forth. > > - A way to identify the HOST component uniquely via API calls, by way of > referring them by *ID* instead of name. > *MODIFIED API CALLs:* > 1. POST SERVICE COMPONENT API call > > - CHANGE : New field *component_type* > - POST > http://{{AmbariServer}}:8080/api/v1/clusters/<clusterName>/servicegroups/core/services/HDFS/components > Body : > {code} > > [{"ServiceComponentInfo":{"component_name":"HDFS_CLIENT2","component_type":"HDFS_CLIENT"}}] > {code} > Response: > > {code} > { > "resources" : [ > { > "href" : > "http://{{AmbariServer}}:8080/api/v1/clusters/<clusterName>/servicegroups/core/services/HDFS/components/HDFS_CLIENT2", > "ServiceComponentInfo" : { > "category" : null, > "cluster_id" : 2, > "cluster_name" : "c1", > "component_name" : "HDFS_CLIENT2", > "component_type" : "HDFS_CLIENT", > "desired_stack" : "HDP-2.6", > "desired_version" : "NOT_REQUIRED", > "display_name" : "HDFS Client", > "id" : 51, > "recovery_enabled" : false, > "service_group_id" : 2, > "service_group_name" : "core", > "service_id" : 3, > "service_name" : "HDFS", > "service_type" : "HDFS", > "state" : "STARTED", > "total_count" : { > "installFailedCount" : 0, > "unknownCount" : 0, > "installedCount" : 0, > "initCount" : 0, > "installedAndMaintenanceOffCount" : 0, > "startedCount" : 0, > "totalCount" : 0 > } > } > } > ] > } > {code} > > 2. POST HOST COMPONENT API call > - CHANGE : New field *component_type* > - POST > http://{{AmbariServer}}:8080/api/v1/clusters/<clusterName>/hosts/<host1>/host_components > Body > {code} > {"HostRoles":{"cluster_name": > "c1","component_name":"HDFS_CLIENT2","component_type":"HDFS_CLIENT"}} > {code} > - Response: > {code} > { > "resources" : [ > { > "href" : > "http://{{AmbariServer}}:8080/api/v1/clusters/<clusterName>/hosts/<host1>/host_components/51", > "HostRoles" : { > "actual_configs" : { }, > "cluster_id" : 2, > "cluster_name" : "c1", > "component_name" : "HDFS_CLIENT2", > "component_type" : "HDFS_CLIENT", > "desired_admin_state" : null, > "desired_repository_version" : "2.6.4.0-91", > "desired_stack_id" : "HDP-2.6", > "display_name" : "HDFS_CLIENT2", > "host_name" : "<host1>", > "id" : 51, > "maintenance_state" : null, > "public_host_name" : "<host1>", > "service_group_id" : 2, > "service_group_name" : "core", > "service_id" : 3, > "service_name" : "HDFS", > "service_type" : "HDFS", > "stale_configs" : false, > "state" : "INIT", > "upgrade_state" : "NONE", > "version" : "UNKNOWN" > }, > "host" : { > "href" : > "http://{{AmbariServer}}:8080/api/v1/clusters/<clusterName>/hosts/<host1>" > } > } > ] > } > {code} > 3. GET SERVICE COMPONENT: > - CHANGE: *host_components* sub-resource will be referenced with end point > as ID one. > - GET > http://{{AmbariServer}}:8080/api/v1/clusters/<clusterName>/servicegroups/core/services/HDFS/components/HDFS_CLIENT2 > - Response: > {code} > { > "href" : > "http://{{AmbariServer}}:8080/api/v1/clusters/<clusterName>/servicegroups/core/services/HDFS/components/HDFS_CLIENT2", > "ServiceComponentInfo" : { > "category" : null, > "cluster_id" : 2, > "cluster_name" : "c1", > "component_name" : "HDFS_CLIENT2", > "component_type" : "HDFS_CLIENT", > "desired_stack" : "HDP-2.6", > "desired_version" : "2.6.4.0-91", > "display_name" : "HDFS Client", > "id" : 51, > "init_count" : 1, > "install_failed_count" : 0, > "installed_and_maintenance_off_count" : 0, > "installed_count" : 0, > "recovery_enabled" : "false", > "repository_state" : "NOT_REQUIRED", > "service_group_id" : 2, > "service_group_name" : "core", > "service_id" : 3, > "service_name" : "HDFS", > "service_type" : "HDFS", > "started_count" : 0, > "state" : "STARTED", > "total_count" : 1, > "unknown_count" : 0 > }, > "host_components" : [ > { > "href" : > "http://{{AmbariServer}}:8080/api/v1/clusters/<clusterName>/hosts/<host1>/host_components/51", > "HostRoles" : { > "cluster_name" : "c1", > "component_name" : "HDFS_CLIENT2", > "host_name" : "<host1>", > "id" : 51, > "service_group_name" : "core", > "service_name" : "HDFS" > } > } > ] > } > {code} > 4. GET HOST COMPONENT: > - CHANGE: Called with ID based end point > - GET > http://{{AmbariServer}}:8080/api/v1/clusters/<clusterName>/hosts/<host1>/host_components/51 > - Response: > {code} > { > "href" : > "http://{{AmbariServer}}:8080/api/v1/clusters/<clusterName>/hosts/<host1>/host_components/51", > "HostRoles" : { > "cluster_name" : "c1", > "component_name" : "HDFS_CLIENT2", > "component_type" : "HDFS_CLIENT", > "desired_repository_version" : "2.6.4.0-91", > "desired_stack_id" : "HDP-2.6", > "desired_state" : "INIT", > "display_name" : "HDFS_CLIENT2", > "host_name" : "<host1>", > "id" : 51, > "maintenance_state" : "OFF", > "public_host_name" : "<host1>", > "reload_configs" : false, > "service_group_name" : "core", > "service_name" : "HDFS", > "stale_configs" : false, > "state" : "INIT", > "upgrade_state" : "NONE", > "version" : "UNKNOWN", > "actual_configs" : { } > }, > "host" : { > "href" : > "http://{{AmbariServer}}:8080/api/v1/clusters/<clusterName>/hosts/<host1>" > }, > "processes" : [ ], > "component" : [ > { > "href" : > "http://{{AmbariServer}}:8080/api/v1/clusters/<clusterName>/servicegroups/core/services/HDFS/components/HDFS_CLIENT2", > "ServiceComponentInfo" : { > "cluster_name" : "c1", > "component_name" : "HDFS_CLIENT2", > "service_group_name" : "core", > "service_name" : "HDFS" > } > } > ] > } > {code} > 5. UPDATE SERVICE COMPONENTS > - NO CHANGE > 6. UPDATE HOST COMPONENTS > - CHANGE: API end point is ID based now. > 7. DELETE SERVICE COMPONENT: > - No CHANGE > 8. DELETE HOST COMPONENT: > - CHANGE: API end point is ID based now. -- This message was sent by Atlassian JIRA (v7.6.3#76005)