[
https://issues.apache.org/jira/browse/KNOX-1014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16163460#comment-16163460
]
Phil Zampino edited comment on KNOX-1014 at 9/14/17 4:59 PM:
-------------------------------------------------------------
KNOX-1014.patch (attached) provides the foundation for service discovery and
topology generation.
After applying the patch, if you have access to an Ambari cluster, you can try
the following:
# Locate or
[create|https://cwiki.apache.org/confluence/display/AMBARI/Quick+Start+Guide]
an Ambari cluster
# Extract the <gateway> element and its contents from
conf/topologies/sandbox.xml, and save it in
conf/shared-providers/sandbox-providers.xml
# Provision the username/password for your Ambari instance
_bin/knoxcli.sh create-alias AMBARI_USERNAME --value AMBARI_PASSWORD
# Create a simple descriptor (JSON):
_Replace YOUR_AMBARI_HOST(e.g., c6401.ambari.apache.org), AMBARI_USERNAME,
YOUR_CLUSTER_NAME with the appropriate values_
{noformat}
{
"discovery-type":"AMBARI",
"discovery-address":"http://YOUR_AMBARI_HOST:8080";,
"discovery-user":"AMBARI_USERNAME",
"provider-config-ref":"sandbox-providers.xml",
"cluster":"YOUR_CLUSTER_NAME",
"services":[
{"name":"NAMENODE"},
{"name":"JOBTRACKER"},
{"name":"WEBHDFS"},
{"name":"WEBHCAT"},
{"name":"OOZIE"},
{"name":"WEBHBASE"},
{"name":"HIVE"},
{"name":"RESOURCEMANAGER"},
{"name":"AMBARI", "urls":"[http://YOUR_AMBARI_HOST:8080"]},
{"name":"AMBARIUI", "urls":["http://YOUR_AMBARI_HOST:8080"]}
]
}
{noformat}
# Place this simple descriptor in conf/descriptors/YOUR_DESCRIPTOR_NAME.json
# Allow the TopologyService to notice the simple descriptor, generate and
deploy the full topology.
# Check conf/topologies, and notice YOUR_DESCRIPTOR_NAME.xml there
# Review the contents of conf/topologies/YOUR_DESCRIPTOR_NAME.xml to see that
the service URLs have been populated from the Ambari cluster details
# Verify the WEBHDFS service URL was correctly discovered
curl -ivku guest:guest-password
https://localhost:8443/gateway/YOUR_DESCRIPTOR_NAME/webhdfs/v1/tmp?op=LISTSTATUS
Now, you can try adding/modifying/removing combinations of provider
configurations and descriptions to see the results.
* If you modify a descriptor, it will update the full topology so the changes
are reflected.
* If you modify a shared provider configuration, it will update any referencing
descriptors, which will in turn update the associated full topology files.
* If you delete a simple descriptor, the associated topology will be
removed/undeployed.
* If you delete a topology file, the associated simple descriptor will also be
deleted.
-* If you delete a shared provider configuration, any referencing descriptors
(and associated topology files) will also be deleted, and the topologies will
be undeployed.-
was (Author: pzampino):
KNOX-1014.patch (attached) provides the foundation for service discovery and
topology generation.
After applying the patch, if you have access to an Ambari cluster, you can try
the following:
# Locate or
[create|https://cwiki.apache.org/confluence/display/AMBARI/Quick+Start+Guide]
an Ambari cluster
# Extract the <gateway> element and its contents from
conf/topologies/sandbox.xml, and save it in
conf/shared-providers/sandbox-providers.xml
# Provision the username/password for your Ambari instance
_bin/knoxcli.sh create-alias AMBARI_USERNAME --value AMBARI_PASSWORD
# Create a simple descriptor (JSON):
_Replace YOUR_AMBARI_HOST(e.g., c6401.ambari.apache.org), AMBARI_USERNAME,
YOUR_CLUSTER_NAME with the appropriate values_
{noformat}
{
"discovery-type":"AMBARI",
"discovery-address":"http://YOUR_AMBARI_HOST:8080";,
"discovery-user":"AMBARI_CLUSTER_ADMIN_NAME",
"provider-config-ref":"sandbox-providers.xml",
"cluster":"YOUR_CLUSTER_NAME",
"services":[
{"name":"NAMENODE"},
{"name":"JOBTRACKER"},
{"name":"WEBHDFS"},
{"name":"WEBHCAT"},
{"name":"OOZIE"},
{"name":"WEBHBASE"},
{"name":"HIVE"},
{"name":"RESOURCEMANAGER"},
{"name":"AMBARI", "url":"http://YOUR_AMBARI_HOST:8080"},
{"name":"AMBARIUI", "url":"http://YOUR_AMBARI_HOST:8080"}
]
}
{noformat}
# Place this simple descriptor in conf/descriptors/YOUR_DESCRIPTOR_NAME.json
# Allow the TopologyService to notice the simple descriptor, generate and
deploy the full topology.
# Check conf/topologies, and notice YOUR_DESCRIPTOR_NAME.xml there
# Review the contents of conf/topologies/YOUR_DESCRIPTOR_NAME.xml to see that
the service URLs have been populated from the Ambari cluster details
# Verify the WEBHDFS service URL was correctly discovered
curl -ivku guest:guest-password
https://localhost:8443/gateway/YOUR_DESCRIPTOR_NAME/webhdfs/v1/tmp?op=LISTSTATUS
Now, you can try adding/modifying/removing combinations of provider
configurations and descriptions to see the results.
* If you modify a descriptor, it will update the full topology so the changes
are reflected.
* If you modify a shared provider configuration, it will update any referencing
descriptors, which will in turn update the associated full topology files.
* If you delete a simple descriptor, the associated topology will be
removed/undeployed.
* If you delete a topology file, the associated simple descriptor will also be
deleted.
* If you delete a shared provider configuration, any referencing descriptors
(and associated topology files) will also be deleted, and the topologies will
be undeployed.
> Service Discovery and Topology Generation Framework
> ---------------------------------------------------
>
> Key: KNOX-1014
> URL: https://issues.apache.org/jira/browse/KNOX-1014
> Project: Apache Knox
> Issue Type: Sub-task
> Components: Server
> Reporter: Phil Zampino
> Assignee: Phil Zampino
> Labels: kip-8
> Fix For: 0.14.0
>
> Attachments: KNOX-1014.patch
>
>
> Implement the foundation for Service Discovery and Topology Generation.
> * Define simple descriptor format (YAML, JSON, etc...)
> * Local simple descriptor and shared provider configuration discovery
> ** Monitor conf/shared-providers, conf/descriptors similar to the way
> conf/topologies is currently monitored.
> * Ambari service discovery (REST API interactions and model construction)
> ** Configuration
> *** How to plug-in discovery implementations
> *** How to configure authentication (credentials/trust) with the service
> registries
> * Topology assembly from simple descriptor and discovery details
> * Topology deployment
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
