That's actually a different error than you posted before. That error is
Ambari telling you that the blueprint that Ansible submitted is invalid.
Likely a result of a required hostgroup not being defined. Can you (re?)
post what you've put in amazon-ec2/playbook.yml?
-D...
On Mon, May 29, 2017 at 12:02 PM, Laurens Vets <laur...@daemon.be> wrote:
> Same problem as all the previous ones...
>
> TASK [ambari_config : Install python-requests]
> *********************************
> ok: [ec2-34-209-23-182.us-west-2.compute.amazonaws.com] => {"attempts":
> 1, "changed": false, "msg": "", "rc": 0, "results":
> ["python-requests-2.6.0-3.el6.noarch providing python-requests is already
> installed"]}
>
> TASK [ambari_config : check if ambari-server is up on
> ec2-34-209-23-182.us-west-2.compute.amazonaws.com:8080] ***
> ok: [ec2-34-209-23-182.us-west-2.compute.amazonaws.com] => {"changed":
> false, "elapsed": 120, "path": null, "port": 8080, "search_regex": null,
> "state": "started"}
>
> TASK [ambari_config : Deploy cluster with Ambari;
> http://ec2-34-209-23-182.us-west-2.compute.amazonaws.com:8080] ***
> fatal: [ec2-34-209-23-182.us-west-2.compute.amazonaws.com]: FAILED! =>
> {"changed": false, "failed": true, "msg": "Ambari client exception
> occurred: Could not create cluster: request code 400,
> request message {\n \"status\" : 400,\n \"message\" : \"Topology
> validation failed: org.apache.ambari.server.topology.InvalidTopologyException:
> The following hosts are mapped to multiple host groups: [
> ec2-34-208-74-17.us-west-2.compute.amazonaws.com]. Be aware that host
> names are converted to lowercase, case differences do not matter in Ambari
> deployments.\"\n}"}
> to retry, use: --limit @/root/metron/metron-deploymen
> t/amazon-ec2/playbook.retry
>
> PLAY RECAP ************************************************************
> *********
> ec2-34-208-74-17.us-west-2.compute.amazonaws.com : ok=47 changed=31
> unreachable=0 failed=0
> ec2-34-209-23-182.us-west-2.compute.amazonaws.com : ok=48 changed=28
> unreachable=0 failed=1
> ec2-35-163-138-4.us-west-2.compute.amazonaws.com : ok=41 changed=27
> unreachable=0 failed=0
> ec2-35-163-150-6.us-west-2.compute.amazonaws.com : ok=41 changed=27
> unreachable=0 failed=0
> ec2-35-167-207-80.us-west-2.compute.amazonaws.com : ok=17 changed=8
> unreachable=0 failed=0
> ec2-54-70-75-61.us-west-2.compute.amazonaws.com : ok=41 changed=27
> unreachable=0 failed=0
> localhost : ok=18 changed=13 unreachable=0 failed=0
>
> root@ip-10-0-10-6:~/metron/metron-deployment/amazon-ec2#
>
>
> On 2017-05-27 04:45, David Lyle wrote:
>
>> For some reason, kibana_hosts is undefined, so the curl command that loads
>> the Kibana dashboard doesn't contain the URL for Kibana. If you look at
>> small_cluster.yml, you'll see that the Kibana should have been installed
>> on
>> the first host in the "web" hostgroup. Verify that you placed a host in
>> that hostgroup (it's the 5th line in Mike's example). If you did, check
>> that host on Ambari and verify that Kibana was installed, if not, you'll
>> want to dig through the install log to find out why.
>>
>> Also- now that I've looked at the example carefully, I noticed that the
>> Zeppelin hostgroup was duplicated. That's not a problem, because the 2nd
>> host will be ignored, maybe you caught it when you created your own.
>>
>> I would try something like (for a 6 node host cluster):
>>
>> - include: tasks/create-hosts.yml host_count=1
>> host_type=sensors,ambari_
>> master,ec2,monit
>> - include: tasks/create-hosts.yml host_count=1
>> host_type=pcap_server,monit,ec2
>> - include: tasks/create-hosts.yml host_count=1 host_type=ambari_slave,
>> enrichment,metron,ec2,zeppelin
>> - include: tasks/create-hosts.yml host_count=2
>> host_type=ambari_slave,search,ec2
>> - include: tasks/create-hosts.yml host_count=1
>> host_type=ambari_slave,web,ec2
>>
>> -D...
>>
>>
>> On Fri, May 26, 2017 at 8:15 PM, Laurens Vets <laur...@daemon.be> wrote:
>>
>> Different error than before, but it still fails:
>>>
>>> TASK [ambari_gather_facts : set_fact] ******************************
>>> ************
>>> ok: [ec2-34-210-207-190.us-west-2.compute.amazonaws.com] =>
>>> {"ansible_facts": {"kibana_hosts": []}, "changed": false}
>>>
>>> TASK [ambari_gather_facts : debug] ******************************
>>> ***************
>>> ok: [ec2-34-210-207-190.us-west-2.compute.amazonaws.com] => {
>>> "msg": "zookeeper_port = 2181, zookeeper_hosts = [u'
>>> ec2-52-43-143-124.us-west-2.compute.amazonaws.com'], zookeeper_url =
>>> ec2-52-43-143-124.us-west-2.compute.amazonaws.com:2181,
>>> kafka_broker_port
>>> = 6667, kafka_broker_hosts = [u'ec2-34-210-18-244.us-west-2
>>> .compute.amazonaws.com', u'ec2-35-164-236-192.us-west-2
>>> .compute.amazonaws.com', u'ec2-52-41-30-101.us-west-2.c
>>> ompute.amazonaws.com'], kafka_broker_url =
>>> ec2-34-210-18-244.us-west-2.co
>>> mpute.amazonaws.com:6667,ec2-35-164-236-192.us-west-2.compu
>>> te.amazonaws.com:6667,ec2-52-41-30-101.us-west-2.compute.
>>> amazonaws.com:6667, metron_hosts = [u'ec2-35-164-236-192.us-west-
>>> 2.compute.amazonaws.com']"
>>> }
>>>
>>> TASK [load_web_templates : Load ES Templates]
>>> **********************************
>>> changed: [ec2-34-210-207-190.us-west-2.compute.amazonaws.com] =>
>>> {"changed": true, "cmd": ["curl", "-s", "-w", "%{http_code}", "-u",
>>> "admin:admin", "-H", "X-Requested-By: ambari", "-X", "POST", "-d", "{
>>> \"RequestInfo\": { \"context\": \"Install ES Template from REST\",
>>> \"command\": \"ELASTICSEARCH_TEMPLATE_INSTALL\"},\"Requests/resource_
>>> filters\":
>>> [{\"service_name\": \"METRON\",\"component_name\":
>>> \"METRON_INDEXING\",\"hosts\" : \"ec2-35-164-236-192.us-west-2
>>> .compute.amazonaws.com\"}]}", "http://ec2-34-210-207-190.us-
>>> west-2.compute.amazonaws.com:8080/api/v1/clusters/metron/requests"],
>>> "delta": "0:00:00.049367", "end": "2017-05-26 21:33:34.767106", "failed":
>>> false, "failed_when_result": false, "rc": 0, "start": "2017-05-26
>>> 21:33:34.717739", "stderr": "", "stdout": "{\n \"href\" : \"
>>> http://ec2-34-210-207-190.us-west-2.compute.amazonaws.com:
>>> 8080/api/v1/clusters/metron/requests/14\",\n \"Requests\" : {\n
>>> \"id\" : 14,\n \"status\" : \"Accepted\"\n }\n}202", "stdout_lines":
>>> ["{", " \"href\" : \"http://ec2-34-210-207-190.us
>>> -west-2.compute.amazonaws.com:8080/api/v1/clusters/metron/requests/14\
>>> <http://west-2.compute.amazonaws.com:8080/api/v1/clusters/metron/requests/14%5C>
>>> ",",
>>> " \"Requests\" : {", " \"id\" : 14,", " \"status\" :
>>> \"Accepted\"",
>>> " }", "}202"], "warnings": []}
>>>
>>> TASK [load_web_templates : Load Kibana Dashboard]
>>> ******************************
>>> fatal: [ec2-34-210-207-190.us-west-2.compute.amazonaws.com]: FAILED! =>
>>> {"failed": true, "msg": "the field 'args' has an invalid value, which
>>> appears to include a variable that is undefined. The error was: list
>>> object
>>> has no element 0\n\nThe error appears to have been in
>>> '/root/metron/metron-deployment/roles/load_web_templates/
>>> tasks/main.yml':
>>> line 26, column 3, but may\nbe elsewhere in the file depending on the
>>> exact
>>> syntax problem.\n\nThe offending line appears to be:\n\n\n- name: Load
>>> Kibana Dashboard\n ^ here\n"}
>>> to retry, use: --limit @/root/metron/metron-deploymen
>>> t/amazon-ec2/playbook.retry
>>>
>>> PLAY RECAP ************************************************************
>>> *********
>>> ec2-34-210-18-244.us-west-2.compute.amazonaws.com : ok=41 changed=27
>>> unreachable=0 failed=0
>>> ec2-34-210-207-190.us-west-2.compute.amazonaws.com : ok=80 changed=31
>>> unreachable=0 failed=1
>>> ec2-34-211-19-168.us-west-2.compute.amazonaws.com : ok=41 changed=27
>>> unreachable=0 failed=0
>>> ec2-35-164-236-192.us-west-2.compute.amazonaws.com : ok=47 changed=31
>>> unreachable=0 failed=0
>>> ec2-52-41-30-101.us-west-2.compute.amazonaws.com : ok=41 changed=27
>>> unreachable=0 failed=0
>>> ec2-52-43-143-124.us-west-2.compute.amazonaws.com : ok=41 changed=27
>>> unreachable=0 failed=0
>>> localhost : ok=16 changed=11 unreachable=0
>>> failed=0
>>>
>>> root@ip-10-0-10-6:~/metron/metron-deployment/amazon-ec2#
>>>
>>>
>>> On 2017-05-26 11:59, Michael Miklavcic wrote:
>>>
>>> Maybe you could re-purpose the playbook.yml for the 10-node install -
>>>> comment out some of the host creation lines and move the host types
>>>> where
>>>> you want them installed, e.g.
>>>>
>>>> From this
>>>>
>>>> tasks:
>>>> - include: tasks/create-keypair.yml
>>>> - include: tasks/create-vpc.yml
>>>> - include: tasks/create-open-inbound-security-group.yml
>>>> - include: tasks/create-open-outbound-security-group.yml
>>>> - include: tasks/create-hosts.yml host_count=1
>>>> host_type=sensors,ambari_master,ec2,monit
>>>> - include: tasks/create-hosts.yml host_count=4
>>>> host_type=ambari_slave,ec2
>>>> - include: tasks/create-hosts.yml host_count=1
>>>> host_type=pcap_server,monit,ec2
>>>> - include: tasks/create-hosts.yml host_count=1
>>>> host_type=ambari_slave,enrichment,metron,ec2,zeppelin
>>>> - include: tasks/create-hosts.yml host_count=2
>>>> host_type=ambari_slave,search,ec2
>>>> - include: tasks/create-hosts.yml host_count=1
>>>> host_type=ambari_slave,web,ec2
>>>>
>>>> to this (6-node)
>>>>
>>>> tasks:
>>>> - include: tasks/create-keypair.yml
>>>> - include: tasks/create-vpc.yml
>>>> - include: tasks/create-open-inbound-security-group.yml
>>>> - include: tasks/create-open-outbound-security-group.yml
>>>> - include: tasks/create-hosts.yml host_count=1
>>>> host_type=sensors,ambari_master,ec2,monit,zeppelin,search,web
>>>> - include: tasks/create-hosts.yml host_count=3
>>>> host_type=ambari_slave,ec2
>>>> - include: tasks/create-hosts.yml host_count=1 host_type=
>>>> ambari_slave
>>>> ,pcap_server,monit,ec2
>>>> - include: tasks/create-hosts.yml host_count=1
>>>> host_type=ambari_slave,enrichment,metron,ec2,zeppelin
>>>>
>>>> Of course, this is just to demonstrate re-locating items, not
>>>> necessarily
>>>> how you would want to lay them out.
>>>>
>>>> Best,
>>>> Mike Miklavcic
>>>>
>>>>
>>>> On Fri, May 26, 2017 at 11:28 AM, Laurens Vets <laur...@daemon.be>
>>>> wrote:
>>>>
>>>> I'm trying to install Metron in AWS but with fewer hosts than the
>>>> default
>>>>
>>>>> 10 node cluster install... So far no luck. Whatever I do, it always
>>>>> ends
>>>>> in
>>>>> errors...
>>>>>
>>>>>
>>>>> On 2017-05-26 10:16, David Lyle wrote:
>>>>>
>>>>> I'm gonna to venture a 'no' simply because I don't know what that is.
>>>>>
>>>>>> Afaik, single_node_vm.yml was never used with AWS. What are you trying
>>>>>> to
>>>>>> do?
>>>>>>
>>>>>> -D...
>>>>>>
>>>>>>
>>>>>> On Thu, May 25, 2017 at 8:50 PM, Laurens Vets <laur...@daemon.be>
>>>>>> wrote:
>>>>>>
>>>>>> Is AWS single_node_vm still supported?
>>>>>>
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
