[
https://issues.apache.org/jira/browse/KNOX-1014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16163460#comment-16163460
]
Phil Zampino edited comment on KNOX-1014 at 9/12/17 6:42 PM:
-------------------------------------------------------------
KNOX-1014.patch provides the foundation for service discovery and topology
generation.
After applying the patch, if you have access to an Ambari cluster, you can try
the following:
# Locate or
[create|https://cwiki.apache.org/confluence/display/AMBARI/Quick+Start+Guide]
an Ambari cluster
# Extract the <gateway> element and its contents from
conf/topologies/sandbox.xml, and save it in
conf/shared-providers/sandbox-providers.xml
# Provision the username/password for your Ambari instance (bin/knoxcli.sh
create-alias AMBARI_CLUSTER_ADMIN_NAME --value AMBARI_CLUSTER_ADMIN_PASSWORD)
# Create a simple descriptor (JSON):
Replace YOUR_AMBARI_HOST(e.g., c6401.ambari.apache.org),
AMBARI_CLUSTER_ADMIN_NAME, YOUR_CLUSTER_NAME with the appropriate values
{
"discovery-type":"AMBARI",
"discovery-address":"http://YOUR_AMBARI_HOST:8080";,
"discovery-user":"AMBARI_CLUSTER_ADMIN_NAME",
"provider-config-ref":"sandbox-providers.xml",
"cluster":"YOUR_CLUSTER_NAME",
"services":[
{"name":"NAMENODE"},
{"name":"JOBTRACKER"},
{"name":"WEBHDFS"},
{"name":"WEBHCAT"},
{"name":"OOZIE"},
{"name":"WEBHBASE"},
{"name":"HIVE"},
{"name":"RESOURCEMANAGER"},
{"name":"AMBARI", "url":"http://c6401.ambari.apache.org:8080"},
{"name":"AMBARIUI", "url":"http://c6401.ambari.apache.org:8080"}
]
}
# Place this simple descriptor in conf/descriptors/YOUR_DESCRIPTOR_NAME.json
# Allow the TopologyService to notice the simple descriptor, generate and
deploy the full topology.
# Check conf/topologies, and notice YOUR_DESCRIPTOR_NAME.xml there
# cat conf/topologies/YOUR_DESCRIPTOR_NAME.xml to see that the service URLs
have been populated from the Ambari cluster details
# Verify the WEBHDFS service URL was correctly discovered
curl -ivku guest:guest-password
https://localhost:8443/gateway/YOUR_DESCRIPTOR_NAME/webhdfs/v1/tmp?op=LISTSTATUS
Now, you can try adding/modifying/removing combinations of provider
configurations and descriptions to see the results.
* If you modify a descriptor, it will update the full topology so the changes
are reflected.
* If you modify a shared provider configuration, it will update any referencing
descriptors, which will in turn update the associated full topology files.
* If you delete a simple descriptor, the associated topology will be
removed/undeployed.
* If you delete a topology file, the associated simple descriptor will also be
deleted.
* If you delete a shared provider configuration, any referencing descriptors
(and associated topology files) will also be deleted, and the topologies will
be undeployed.
was (Author: pzampino):
KNOX-1014.patch provides the foundation for service discovery and topology
generation.
After applying the patch, if you have access to an Ambari cluster, you can try
the following:
# Locate or
[create|https://cwiki.apache.org/confluence/display/AMBARI/Quick+Start+Guide]
an Ambari cluster
# Extract the <gateway> element and its contents from
conf/topologies/sandbox.xml, and save it in
conf/shared-providers/sandbox-providers.xml
# Provision the username/password for your Ambari instance (bin/knoxcli.sh
create-alias AMBARI_CLUSTER_ADMIN_NAME --value AMBARI_CLUSTER_ADMIN_PASSWORD)
# Create a simple descriptor (JSON):
Replace YOUR_AMBARI_HOST(e.g., c6401.ambari.apache.org),
AMBARI_CLUSTER_ADMIN_NAME, YOUR_CLUSTER_NAME with the appropriate values
{
"discovery-type":"AMBARI",
"discovery-address":"http://YOUR_AMBARI_HOST:8080";,
"discovery-user":"AMBARI_CLUSTER_ADMIN_NAME",
"provider-config-ref":"sandbox-providers.xml",
"cluster":"YOUR_CLUSTER_NAME",
"services":[
{"name":"NAMENODE"},
{"name":"JOBTRACKER"},
{"name":"WEBHDFS"},
{"name":"WEBHCAT"},
{"name":"OOZIE"},
{"name":"WEBHBASE"},
{"name":"HIVE"},
{"name":"RESOURCEMANAGER"},
{"name":"AMBARI", "url":"http://c6401.ambari.apache.org:8080"},
{"name":"AMBARIUI", "url":"http://c6401.ambari.apache.org:8080"}
]
}
# Place this simple descriptor in conf/descriptors/YOUR_DESCRIPTOR_NAME.json
# Allow the TopologyService to notice the simple descriptor, generate and
deploy the full topology.
# Check conf/topologies, and notice YOUR_DESCRIPTOR_NAME.xml there
# cat conf/topologies/YOUR_DESCRIPTOR_NAME.xml to see that the service URLs
have been populated from the Ambari cluster details
# Verify the WEBHDFS service URL was correctly discovered
curl -ivku guest:guest-password
https://localhost:8443/gateway/YOUR_DESCRIPTOR_NAME/webhdfs/v1/tmp?op=LISTSTATUS
Now, you can try adding/modifying/removing combinations of provider
configurations and descriptions to see the results.
* If you modify a descriptor, it will update the full topology so the changes
are reflected.
* If you modify a shared provider configuration, it will update any referencing
descriptors, which will in turn update the associated full topology files.
* If you delete a simple descriptor, the associated topology will be
removed/undeployed.
* If you delete a topology file, the associated simple descriptor will also be
deleted.
* If you delete a shared provider configuration, any referencing descriptors
(and associated topology files) will also be deleted, and the topologies will
be undeployed.
*
> Service Discovery and Topology Generation Framework
> ---------------------------------------------------
>
> Key: KNOX-1014
> URL: https://issues.apache.org/jira/browse/KNOX-1014
> Project: Apache Knox
> Issue Type: Sub-task
> Components: Server
> Reporter: Phil Zampino
> Assignee: Phil Zampino
> Labels: kip-8
> Attachments: KNOX-1014.patch
>
>
> Implement the foundation for Service Discovery and Topology Generation.
> * Define simple descriptor format (YAML, JSON, properties, etc...)
> * Local simple descriptor discovery (re-use existing FileAlterationMonitor?)
> * Ambari service discovery (REST API interactions and model construction)
> ** Configuration
> *** How to plug-in discovery implementations (service loader?)
> *** How to configure authentication (credentials/trust) with the service
> registries
> * Topology assembly from simple descriptor and discovery details
> * Topology deployment (something more than copy to the conf/topologies
> directory?)
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
